UX designers strive to create the most user-friendly products and services possible. But without speaking directly to users about what they want and need, UX designers can only guess what the best user experience will be. This is why UX research is such an essential part of the UX design process.
UX research involves asking users to identify what problems a product or service can help them solve and provides insights into the best ways to do so. There are a wide variety of methods a UX researcher can use, but in general, each of these UX research methods will fall under the umbrella of either quantitative research or qualitative research.
In this post, we’ll delve into the differences between quantitative vs. qualitative research and the benefits and drawbacks of using each one by discussing the following topics:
- What is quantitative research?
- What is qualitative research?
- Examples of quantitative research
- Examples of qualitative research
- Which method should you choose?
- Presenting quantitative versus qualitative research
What is quantitative research?
Quantitative research answers questions about quantity such as:
- How much?
- How often?
- How many?
For example, you might want to conduct a quantitative study to find out how often people use a product, how much time they spend with the product, and how many features of the product they use.
Quantitative methods involve collecting numerical data that can be analyzed using statistics. This means quantitative studies are often straightforward to implement and can be easily automated and conducted remotely, however they also require enough participants to ensure they yield statistically significant results.
Advantages
- Studies are quick to implement and cost effective
- Analysis is fast
- Results are statistically significant and considered objective
- Numeric results are often easy to understand and use to improve a product’s UX
Disadvantages
- A large number of participants (at least 20) are required to ensure statistical significance
- Someone of the team must have expertise in statistical analysis
- Results lack context, so they can reveal what people did but not their reasoning
What is qualitative research?
Qualitative research answers questions about quality such as how users feel about an experience, why they like or dislike a feature, and why they make certain decisions when completing tasks. Ultimately the goal of qualitative research is to delve into users’ thinking to find out “why.”
Qualitative methods involve collecting data in the form of written or spoken words that are then analyzed for prominent themes and patterns. As a result, both the data collection and analysis of qualitative studies can be time-consuming and labor-intensive. On the other hand, only a few participants are required to arrive at valuable results that can provide nuanced insights into users’ thoughts and feelings about a product.
Advantages
- Only a small number of participants (usually around 5) are required
- Provides detailed, first-hand insight into UX issues
- Allows for flexibility during data collection
- Results in compelling stories that help stakeholders foster empathy for users
- May identify unexpected information about the user’s experience with the product
Disadvantages
- Time consuming to plan, run, and analyze
- Tends to cost more than quantitative studies due to the time commitment required
- Analysis is subjective and therefore results may be influenced by researcher bias
Examples of quantitative research
Quantitative research includes any studies and tests that yield numerical data, such as:
- First-click tests
- A/B tests
- Heatmaps
- Analytics
- Surveys with yes/no and multiple choice questions
Many of these methods can be implemented remotely with a growing group of UX research tools. For example, first-click tests and surveys can be run through Optimal Workshop, Google Analytics can provide any required analytics data, and Hotjar can record user interactions with your product in order to generate heatmaps of how users navigate through it. You’ll also use software, such as SPSS (or plug formulas into a Google or Excel spreadsheet) to statistically analyse quantitative results.
Examples of qualitative research
Qualitative research is often carried out in person either in a lab where users come to participate in the study or in the field where researchers go to where users are. It can include the following methods:
- User interviews
- Focus groups
- Ethnographic studies
- Shadow sessions
- Surveys with open-ended questions
Qualitative studies usually require at least one researcher to be there to perform tasks such as asking users questions during user interviews or observing users during ethnographic studies. However, there are some tools that enable researchers to run qualitative studies remotely. For example, video research platform Lookback enables researchers to run either moderated or unmoderated remote interviews.
Furthermore, while analysing qualitative data is a very hands-on process, researchers can use platforms like Rev to transcribe things like interview recordings and programs like Dedoose, NVivo, and Reframer to organize, synthesize, and analyze the large amount of text data qualitative studies inevitably yield.
Which method should you choose?
Qualitative and quantitative research can be used at pretty much any point in the design process but each method is most useful at different times.
Quantitative research tends to be most valuable during the design phase of the UX process when it can quickly uncover if the UX solutions being designed are working the way the UX team intends. Quantitative studies are also helpful to validate the final design of a product before it goes live. At this point, quantitative research can confirm the design is ready to go to clients and stakeholders or point to the tweaks that need to be made to ensure the product is the best it can be.
Quantitative research, often with Google Analytics, is also utilized once a product goes live to evaluate the product’s ongoing usability, compare it to competitors, and track its return on investment.
On the other hand, qualitative research is most valuable during the discovery phase of the UX process when it can provide extensive insight into users’ thoughts and feelings about the product the UX team is getting ready to design or redesign. Qualitative studies can also be valuable during the design phase when the UX team wants to understand how users feel about key user experience features, especially if they’re unique or unusual. Findings from such studies can help the UX team determine if it’s pursuing the best possible solutions.
Combining quantitative and qualitative research
While it can make sense to focus on either quantitative or qualitative research in some circumstances, more often than not, it can be incredibly valuable to combine them to conduct a mixed methods study. That’s because quantitative and qualitative methods are complementary approaches. For example, a survey can include a series of quantitative multiple choice questions that help point to how much users like a product’s different features, as well as several qualitative questions that enable participants to explain the reasons for their answers to the multiple choice questions.
Mixed methods research is the best of both worlds, ensuring you can answer not only the questions of how much and how many but also why. This provides results that have both statistical weight and depth of insight, with the findings of each method strengthening the other.
Some examples of mixed method research include:
- Card sorting where users explain their reasoning as they organize the cards
- Tree testing where users describe their decisions out loud
- Surveys that include multiple choice and open-ended questions
Presenting quantitative versus qualitative research
Whether you performed quantitative, qualitative, or mixed method research, each UX research report should follow the same basic outline:
- An introduction
- Research goals
- Business value
- Methodology
- Key learnings
- Recommendations
Most of these sections involve explaining what you did and why you did it, but in the key learnings section, where you’ll present the findings from your research, the data and artefacts you present will depend on the research methods you used.
For instance, if you performed a quantitative study, this is where you’ll present any relevant statistics you found, such as the percentage of participants who preferred one UX design solution over another. If you performed a qualitative study, there are more ways to present your findings and you’ll have to decide which ones best represent your results. Options can include sharing the main themes or patterns you found in the qualitative data, illustrative quotes from participants, or artefacts like personas, journey maps, affinity diagrams, or storyboards.
To present the best key learnings section, it helps to combine qualitative and quantitative results whenever possible. This provides an opportunity for you to back up the statistical results uncovered by your quantitative research with quotes, personas, affinity maps, or other findings from your qualitative research. Not only does this create a convincing demonstration of the value of your research, it makes a stronger case for your findings and fosters greater empathy for your products’ users.
Conclusion
Both quantitative and qualitative UX research has strengths and weaknesses. As a result, UX researchers must think carefully about the goals of the research studies they’re conducting before deciding which method to utilize. No matter which one you choose, however, the results of UX research can make an invaluable contribution to the success of the user experience of any product.
