User Feedback: Getting Honest Insights Via Interviews

The CFO of Netigate, Henrik Ceder, says about user feedback:

“Feedback is the compass for greatness; it tells you what to avoid, what to learn, and where to excel.”

Talking to users is a great predictor of success. To prove it, let’s look at Steven Blank’s The Lean LaunchPad training for the I-Corps@NIH program. The major point of it is:

“Lean startups use a ‘get out of the building’ approach called customer development to test their hypotheses. They go out and ask potential users, purchasers, and partners for feedback…”

Out of 2,500 teams that received this entrepreneurial training, more than half went on to launch their startup and raise substantial funding. The core part of the training for future founders – conduct at least 100 customer interviews. 

YCombinator Startup School also underscores the importance of user interviews taught by a former product lead of Airbnb. Airbnb is the company that maintains a strong culture of staying connected with its customers and hosts. In its early days, Airbnb founders Brian Chesky and Joe Gebbia shared their personal cell numbers with the hosts and still occasionally get calls. It’s even been speculated that Airbnb would not exist today without the honest feedback from its hosts.

In this post, we’ll dive into why, as a founder, you should talk to your users, what questions to ask, as well as the essentials of getting honest user feedback. 

Honest User Feedback Throughout Startup Lifecycle

Wherever you are in the startup life cycle, conducting a user interview brings invaluable feedback. It either helps you to determine whether this problem really exists or the perceived value of your solution. 

For instance, a fair share of SaaS solutions end up failing by overcomplicating what users already have workarounds for with spreadsheets for free. There is no other way to know beforehand, apart from talking to users directly. 

Another thing is that it’s common for startups to start developing a solution in conditions of high uncertainty. This is the default state for all early-stage startups. In fact, research into 631 engineering events of startups revealed that 70% were unplanned, made to react to things revealed over time. The surest way to reduce uncertainty is to talk to users. 

In the chart below, you can see the key topics of user interviews for every stage of a startup’s life cycle. 

Conducting a Customer Interview at Ideation

The key here is to start with the problem you think users have. Going into the interviews, do not think about your potential solutions yet. Your goals should be:

• Is it a problem people care about at organizations?
• Existing behaviors around it;
• Is it something they are willing to pay for? 

At this stage, the major point is that at no point should you be disclosing anything about your solution. Maybe only mentioning something briefly at the end of a customer interview. 

Try to ask open-ended questions as much as possible. If your startup wanted to solve a problem of employee well-being, the questions could be as follows:

• Does your company care about employee well-being? Why/ why not?
• What do you already do for it? 
• How often? 
• Why is it [hard/frustrating/complicated/etc…]? (depends on what a user answers to previous questions)
• Who in your company is responsible for wellbeing actions?
• Why is this important for your company?

Remember to listen and record the interview, or at least take notes. Ask follow-up questions as you go, but keep focus on: problem, existing behavior, and used paid solutions. Having done that, you can begin to synthesize what you’ve learn, draw up an MVP, and design follow-up experiments for validating product ideas.

ProductTalk wrote about the founder of ThoughtFlow, Kranthi Kiran, who used to say: “My earlier approach to building products was to start with an idea, build a product, and then market it.” However, this tactic of his was ultimately unsuccessful. While founding ThoughtFlow, instead of starting the development, he had spent 6 months interviewing people who use mind maps about their workflows and challenges. And this venture succeeded.

Where to Find Users To Interview

• Founder’s personal network of former colleagues and connections;
• Reddit forums;
• LinkedIn;
• Communities on Discord/Slack;
• Industry events.

Pitfalls of a First Customer Interview

• Don’t introduce your solution. Remember that you first need to learn about the problem and how customers solve it already. 
• Listen, don’t talk. This often overlaps with the first point. Once you start talking about the potential solution, you will not learn anything. 
• Don’t ask about features users might want. Your task is to learn about their behaviors, motivations, and struggles around the problem. It is your job and that of product designers to think of features.
• Yes/no questions. These can quickly stifle the conversation and lead to poor insight. 
• Asking two questions at the same time. This quickly creates confusion and frustration. Most often, a customer either chooses one of the questions to respond to or answers briefly to both. 

User Feedback during MVP Development

At YCombinator, they say – once you have even only a design of the feature, get user feedback. It does not have to be a full-on customer interview. You can simply show a mock-up, ask the user to play around with it, or give a more concrete task to complete. The key here is to ask the user:

• To talk out the process of interaction; 
• Comment on things as the user progresses.  

When you build a customer interview like that, you’ll get honest and immediate reactions to:

• The feel of the UX – fresh/overwhelming/crisp/etc.
• Information architecture – is it easy or intuitive to find the item on the navbar or navigate through pages?
• Informational load – does the user understand what is required of them, what kind of input they will need to provide, how much the user has to think about each action, and what is the next step?
• Responsiveness – after each action, is there enough feedback from the app?
• Overall experience – was it easy or difficult, will it be comfortable to perform this action repeatedly at work? 

Also, it will be useful to watch the user’s eye movement while interacting with a prototype: 

• Where they look first;
• What they try to click;
• Things they ignore. 

This is a good practice to get the idea of user expectations and existing mental models.

Below you can see screenshots of founders sharing about the moments when they knew they had made it. A few of them appeared during a customer interview.  

Finding Product-Market Fit Stage: A Customer Interview

First things first, finding Product-market Fit (PMF) essentially revolves around whether the user loves the product. So, to know if you’ve hit a PM, you have to ask your users:

1. How do you feel about using our product?
2. How would you feel if you couldn’t use our product anymore?

Other than that, this stage is largely driven by analytics. You already have an MVP and measurable user behavior, metrics, and KPIs. Therefore, the goal of a user interview is to dive deeper into motivations, the WHYs, and hidden patterns. For instance:

• Can you describe your experience when you first felt that the product was useful to you?
• Was the use of the product expected, or did something surprise you?
• On a typical day, how does the use of the product fit into your day?
• Have there been instances when you intended to use a product but did not? What happened?
• Is there anything that the product does not do your way? Do you have a workaround for it?
• Are there any other tools that you need to use in combination with our product?

Basically, those questions lead to deeper knowledge of users and their interactions with the product. It will help you reconcile the intended use with the actual and uncover missed opportunities to increase the product’s value.

Moreover, at this stage, you already have cohorts, and it can be beneficial to conduct interviews with representatives of each. You can also devise interview questions to probe price sensitivity and discover monetizable extras. 

Lastly, you’d want to try to interview users who churned.  Below is the screenshot of a Superhuman success moment driven solely by interviewing churned users.

User Feedback at the Growth Stage

Usually, many companies do not conduct interviews at this stage due to a strong PMF and powerful analytics setup. And, truly, analytics does perform most of the heavy lifting. In addition, there is a lot of A/B testing and fast incremental experimentation going on, which utilizes worked-out and usually rapid workflows. Analytics will show points of drop off, time-consuming tasks, unused functionality, and so on. 

But, conducting a user interview will help you:

• Not to miss a valuable user insight during rapid experimentation; 
• Hit the right spot by conditioning concrete scenarios during interviews;
• Make sure that experimentation while delivering short-term wins does not cause negative side effects in the long run;
• Keep tabs on the emotional connection between the user and the app;
• Save engineering effort and reduce development cost.

In connection with these limitations of A/B testing, Brian Byun, CEO & Founder of Hubble, shares the following solution: concept testing. This is basically the same A/B tests, but done in Figma by UX researchers or Product Designers. And, instead of going through the entire trouble of coding the change, testing it, shipping, and launching on selected traffic, you simply ask users’ opinions. In some cases, concept testing is much more useful, cheaper, and does not require interpretative guesswork. You just get straight answers.

Customer Interview for Concept Testing: Questions to Ask

So, at this stage, conducting a user interview is basically presenting users with new designs and workflows and asking their feedback. Questions revolve around:

• What do you expect to happen if you click [X] button? 
• In your opinion, how does this layout fit with your current use of the product?
• Looking at variant A, what is the first thing that draws your attention?
• After working through variant B, does your perception of the product change?

Additionally, you can ask:

1. Recall the last three uses of the product – what made you open the app?
2. What related tasks do you do more often than you use the product for?
3. Is there a part of the workflow connected to the tasks you perform with our product, but you still have to use spreadsheets or docs?
4. Do you wish to automate the step you do right before using our product? What about the one right after?
5. In times when you didn’t intend to use our product, but ended up using it, what was the reason? Deadline? Client request? Colleague’s question?

Conducting a User Interview for the Scaling Stage

The major reason to do user interviews here is new markets, geographies, users from new backgrounds and cultures. So, anything and everything that is connected with new uncertainties. Businesses, entering new countries with distinct cultural backgrounds, usually localize their product, adapt pricing, and so on. The need for user interviews is obvious. You want to make sure you do not offend users in new markets, do not confuse them, and comply with their cultural expectations and mental models. 

The questions are alike the very first stage: you want to talk about the problem your product solves and user behaviors around it in the target country/culture. However, you might want to present your solution at the end, trying to scope the impressions/reactions. So, add questions like:

• What feels unnatural or foreign in this interface?
• Is there anything that feels off due to translation or tone? What?
• What trust signals can you pinpoint in the app?
• What feelings do you experience when considering starting to use this app? Why?

Maturity Stage: A Customer Interview of Strategic Significance

With mature products, the core goal is to make sure that:

1. The product is protected from stagnation;
2. The next growth loop is identified.

Today, users’ workplaces and contexts change faster than ever before. 3M is the textbook example of utilizing user interviews for these purposes. They utilized a technique called ‘The Lead User”. Basically, they found users of their products whose needs were always ahead of the market. Then, interviewed them and kept extending their products to fit the needs of these lead users. However, most mature companies do not publish these techniques as they form their competitive advantage and the ability to stay ahead in the market.

Final Words

Conducting a user interview matters in its own way at every startup lifecycle stage. Whether you’ve just got an idea or have a mature business, user feedback often becomes an indispensable source of success. 

• You can make sure usage of your apps becomes habitual through learning about users’ existing behaviors.  
• You can avoid overengineering and focus on solving value-adding problems.
• During A/B experimentation, talking to users allows you to keep track of long-term goals over short-term wins.
• Many businesses have benefited tremendously, either by opening new markets, novel business opportunities, functionalities, or through the opposite – not wasting time on things users already have free and easy workarounds for.
• Feedback techniques, mentioned in the article, include Lean LaunchPad, Concept Testing, The Lead User, as well as Superhuman’s focus on churned users.

FAQ: User Feedback: Getting Honest Insights Via Customer Interviews