User research has become a very popular discipline recently.
Why? Mainly because many people view it as essential to operating as a “user centric” company and “user centric” companies build the best products.
However, even if someone believes user research is essential in order to make great products, that doesn’t mean they believe in all of the results that talking to users will produce.
A few of the reasons people disagree with user research results include:
- Lack of clarity in how insights will drive decisions
- Results not statistically significant
- Conflicting data with an individual’s own belief
Each one of these points can absolutely be valid concerns. With that said, user researchers need to recognize when these concerns are real and when their stakeholders are using them as a crutch that prevents them from acting on the results of the research.
I would like to dive into each one of these reasons to flesh out when these concerns are valid, when they are not, and how you as a researcher can alleviate these concerns.
Lack of clarity in how insights will drive decisions
Valid Concern: As a researcher, you will know this concern is valid when you haven’t put the time up front to understand what the goal of the research is and what decisions this research will drive.
If you do not have a goal or this research isn’t driving a decision, more often then not, that research is not needed.
Spend the time up front to understand what your stakeholder is looking for so you can map your findings to the questions your stakeholder had in the first place.
Invalid Concern: If you have put in the effort up front to understand why you are conducting this study and your stakeholder still does not know how this research will drive decisions after you have collected the results, it is most likely because what you found in your research led to more questions than answers.
Although this may appear as bad results, it’s actually the opposite.
If you can’t directly make a decision based on the research you did because you now have more questions to answer, that just means there is more research to be done in order to see the full picture and deliver actionable insights.
How to Fix: If your study gave you more questions than answers which led to results that weren’t actionable, lay out what you learned for your stakeholder and recommend next steps that will help them get what they need to make their decisions.
It may be disappointing to them that your research will take longer, however, once you do have actionable insights, they are typically more powerful than you could have ever imagined.
Results not statistically significant
Valid Concern: This concern is absolutely valid when you are conducting an A/B test. Although you can get directional data from adjusting the period at which you are looking at an A/B test or the confidence level of the data, statistical significance is needed when truly understanding if one product change is working as expected.
If you are partnering with a member on your analytics team, I guarantee they will tell you this.
By looking at the results too early or decreasing your confidence level to reach a “significant” result as defined by you and your stakeholder, this could lead you to a result that isn’t exactly truthful.
Invalid Concern: More often than not people use this concern as an excuse to not take qualitative data into account when making decisions.
Yes, we may have only talked to 8 people on how they use our product and why X feature is important, but understanding those people’s motivations, while not “significant”, is extremely valuable and is enough to hear patterns that could dramatically increase the value of your product to your users.
How to Fix: To help alleviate this concern, try pairing quantitative data with qualitative data.
Now that you know why your users are doing something, work with your analytics team to see if your product is tracking an action that is comparable to your user’s motivations.
This can help you discover deep insights from gathering qualitative data and validate that these insights are correct by looking at the actions of a larger sample size.
Conflicting data with an individual’s own belief
Valid Concern: When the results of two research studies conflict, you will often see this concern come up. The good news is at least the results of one study are helping to shape someone’s opinion.
The bad news is, this can be a barrier to getting that person adopt the results of another study.
Invalid Concern: There will always be someone who believes in their opinion whether it’s grounded in user research or not. These people can be very hard to sway and often make decisions based on their gut instead of what your users are telling you.
What can I say, when someone operates this way, it is nearly impossible to conduct a study that will be useful to the decisions they are making.
If you are this person, please…stop.
If not for your coworkers, stop making decisions like this for your users.
How to Fix: As stated above, this can be really hard to fix. However, the best way to approach this concern is to understand why someone is feeling this way.
What in their gut makes this feel like the right approach?
Why do they believe their gut is better than what our users are telling us?
The better you can understand their hesitancy towards using research in order to make their decisions, the more likely you will be able to find a way to test their assumptions.
Hopefully, this will help you understand some of the common concerns people have in allowing user research to help drive their decisions within your organization.
Better yet, I hope this will help you alleviate some of the concerns your stakeholders and others in your organization have to give your research more weight when making decisions.
If you can address the concerns above, you’ll be well on your way to helping your company become more “user centric” and build amazing products.