Beta Testing With Customers

A problem that frequently plagues most startups is not getting feedback both often and early enough. A lot of times, I think that I’ll know what is the best solution, but someone will provide some feedback, and I’ll be surprised by how great their idea was. Most of the times these great ideas are not even close to what I was thinking of. This is the reason why many successful entrepreneurs advocate speaking to customers often and early in the ideation process to develop new products.

But how do you go about finding these early customers?

Customer Development Options

1. Your network. An indie developer only needs to get ~10 early adopters to start collecting meaningful feedback on a new product. You can easily start sourcing this from your personal network. Look for people who might fit the characteristics of your target market.

2. Social media. You can enlist people on facebook and twitter if you have an existing presence on those platforms.

3. Build a mailing list. If you were like me and didn’t know what you were doing at all when you first started. Just launch something and see if you get any users at all. If people actually sign up and show an preference for using your product, then build an email feature. Start capturing those emails and ask some of users if they would be interested in beta testing new updates.

Customer Interviews

Once you’ve identified several customers that fit your target profile, the purpose of a customer interview is to (1) understand the customer pain points, (2) validate your idea assumptions, and (3) test how you app functions out in the wild as compared to competitors.

Customer interviews are preferentially always in-person, but a video chat (eg. Skype) is a workable substitute. I always like to start out with the phrase “Please be very blunt with me. You won’t be hurting my feelings at all…” to create an environment of open and honest dialogue. I generally ask permission to record the conversation using a program like Jing, so that I can also reference it at a later point.

I try to keep these conversations fairly unstructured because I’m more interested in hearing the customers opinions. A general framework is:

  • Present some basic background information
  • Listen to the customer, paying close attention to what the customer describes as their problem and how severe the problem is.
  • Learn if they are using any existing solutions and what their opinions are of them. Try to ascertain why certain solution are preferable.
  • Share broadly how you are approaching the problem and ask for their feedback.
  • Better yet, send them a test version of your app, or if you’ve already sent it to them, ask them to open the app and tell you their opinion as you watch them use it. Note any confusing and sticking points in the UX.
  • Close out the interview by thanking them, and delivering on any expectations that have been set. I’ve seen some people offer gift cards and other people nothing. This part is completely based on your own budget constraints. From personal experience, most people that are enthusiastic enough to sign up as beta testers generally aren’t doing this for financial gain, and will happily help out if only to be a part of solving their own problem. I usually end the interview asking them if they’d be willing to help out again in the future and if we can notify them on the app’s progress via our mailing list.

After a video interview is complete, synthesize your notes for that particular customer. I’ve gotten lazy a few times and lost some valuable insights because too much time had passed between the interview and when I finally had time to review my notes.

Improve Your Product

Lastly once your customer interviews are all complete, revist your existing product and product roadmap. Repriortize and make necessary changes. When is it a good time to stop talking to customers? Rule of thumb is when you start hearing the same feedback over and over again.


Leave a Reply

Your email address will not be published. Required fields are marked *