Duolingo’s VP of Growth Gina Gotthilf examines the four A/B tests that have been most crucial to Duolingo’s growth and shares lessons and cautionary tales.
A/B TEST #1: Delayed Sign-Up
- What was causing them to lose or fail to retain users?
- Implement hard and soft walls regarding signing up.
- Test early users to determine if signing up before using your product or after is better.
A/B TEST #2: Streaks
- How do they get people to do it every day or every other day for languages to stick?
- Streaks keep track of how many days in a row a user has used the product.
- When possible, have users set goals that will help them form habits.
A/B TEST #3: Badges
- How do they get users to complete more lessons?
- Badges are rewards that represent meaningful accomplishments.
- Successful testing demands balance; at the core of the minimum viable product (MVP) is “viable,” so try to avoid getting caught up in the “minimum” aspect.
A/B Test #4: The In-App Coach
- How do they encourage people based on their ability to improve their skills rather than their natural intelligence?
- Duo is Duolingo’s supportive in-app owl mascot who encourages users using a growth-mindset.
- Sometimes you have to put your assumptions and personal tastes aside and just try it.
The Tenets of Testing
- Rake in and rank tests continuously.
- How many people will be affected by this change?
- How many hours will go into running the test?
- Refine and repeat.
- Remember to design with restraint and stand firm on what you stand for because mission trumps metrics.