Understand Why You Are Running a Beta
- Understand the problem that you’re trying to solve
- What do you want to learn as early as possible?
- Not every feature needs a beta.
- E.g., Intercom decided it would be easier to release and learn for a new feature because it required considerable volume, and developers were working on tight deadlines.
Paint a Picture of a Successful Beta
Use success metrics to have something tangible to compare progress to.
Consider Who Should Get Access
- Start a conversation with customers who have requested the feature.
- Start with a small rollout and then increase the audience based on adoption and feedback.
- Stress to the customer that it is a beta to set expectations.
- Describe the problem.
- Describe the solution with visuals.
- Ask for feedback.
Get Direct Feedback
- Call customers early on.
- Before the call, review what they have and haven’t done, and jot down the qualitative questions you care about them answering.
Make Sense of Beta Feedback
Create a spreadsheet to track feedback. Include:
1. Original feedback
2. Type of feedback
1. E.g., Improvement, comprehension, or usability
3. Summary of feedback
4. Reason for an issue
Decide When to Ship
- Set a timeline as a forcing function for actions.
- Do you feel confident that you have solved the problem?
- Is it replacing something that exists already?
- Does your marketing team want to announce at a particular time?