Quantitative vs. Qualitative Data: Using Both to Build a Better Digital Platform

You’ve just rolled out an important new feature on your platform, and it’s time to answer the all-important question: is it getting the results you want? If you’ve set up an analytics tool, you can look at performance indicators like registrations, logins, downloads, or shares. But that kind of quantitative data will only get you so far.

Let’s say that new feature isn’t having the impact you’d hoped for — maybe registrations are lacking or engagement is low. You have a problem you need to solve, but you don’t have any information about why it’s happening. And you may have an entirely different underlying problem you need to address.

Where can you find actionable information? Enter qualitative research.

By answering the why behind what’s happening, qualitative data provides context for problems that surface through quantitative analysis. It helps you uncover the root of the problem you have and can also reveal problems you didn’t even know existed.

In this article, we’ll cover how to use both types of research to inform your platform design.

First, the Numbers: Quantitative Research

When you’re evaluating the performance of a digital platform, a good place to start is the cold, hard numbers. Quantitative research provides numerical data that can indicate, at a glance, whether your platform is meeting your business objectives. It can also show the scale of any problems and help prioritize which ones to address.

One major benefit of quantitative data is benchmarking. Tracking your data over time reveals whether UI changes are producing the results you want — and can help you measure the ROI of your efforts. You can also compare your data to an industry benchmark or a competitor’s stats as a barometer for your own performance.

Here are some examples of quantitative research methods:

Web analytics

This data describes what people are doing with your platform: where they go, what they click on, what features they use. It’s good for finding problems and monitoring the performance of content or features.

A/B testing

Here, you’re using experiments to compare different UI designs. By creating two live versions of the same element, like a call-to-action button, you can see which one performs best. Learn more in our article on A/B testing.

Surveys and questionnaires

Surveys let you gather information about your users’ preferences, attitudes, and behaviors, and they can produce a combination of quantitative and qualitative data. For easy-to-capture numerical data, use techniques like ratings and multiple-choice questions.

Usability testing

By measuring user experience with hard data, you can test how easy (or not) a platform feature is to use. Let’s say you just released a reminder function, and you want to know if users can create a reminder in two minutes or less. You can run a test where you ask participants to set a reminder, and measure what percentage are able to complete the task within two minutes.

Now that you’ve got a sense of what users are doing on your platform, let’s look at ways to learn why and how they’re doing it.

Now for the Words: Qualitative Research

Qualitative research can help you investigate why something is happening, identify ways to fix problems, and even determine whether you should phase out a feature or redesign it. Using detailed, contextual descriptions of users’ experiences, you can dive deeper into exactly which elements are working well and which are problematic.

Quantitative and qualitative research both provide useful data, but they’re more powerful when used together.

Unlike with quantitative research, you don’t need a ton of data points to get usable info. For example, if you see five customers in a row walk into the corner of a display in a retail store’s entrance, you can safely assume that most visitors will do the same thing.

You may be avoiding qualitative data because it seems expensive. And some techniques, like focus groups, require a greater investment than others. But, because you don’t need an enormous amount of data, qualitative research can be very cost-effective. It might even save you money by helping you identify and fix problems faster.

Here are some examples of qualitative research methods:

User Interviews

There are a number of different ways to handle user interviews, depending on the type and specificity of info you’re looking for. Here are a few:

• Talk to a subset of your platform users and ask what they like, don’t like, and why. What could be improved?
• Listen to users narrate their experience as they move through your platform, to learn how they feel about particular elements or tasks.
• Give test subjects a post-task survey, to capture their experiences while they’re fresh.

Focus Groups

These are similar to user interviews, but they’re done in a group setting. The advantage of a group is that it can often generate more feedback, as people tend to open up when they hear the experiences of others. Just be sure you have a moderator who gives everyone a chance to speak.

Field Studies

What people say they do… is often not what they actually do. Watching platform users in their natural environment can reveal gaps in your understanding of the user experience. You can use direct observation, interviews, contextual inquiry, and usability tests to learn how people do things and why they do them in particular ways.

Diary Study

This method asks users to document their experiences over time, making it useful for understanding longer-term behaviors. You can learn things like what motivates people to use certain features, what they’re trying to accomplish, how they feel, and what their overall journey looks like.

User Surveys

As opposed to quantitative surveys, qualitative surveys use open-ended questions to learn what users think and feel in their own words. One common pitfall: avoid leading questions. Instead of asking, say, “How easy was it to find the info you needed?”, ask “Describe your experience looking for that information.”

Like Peanut Butter & Jelly

Quantitative and qualitative research both provide useful data, but they’re more powerful when used together. Remember that quantitative data can tell you when there’s a problem with your platform design, but you’ll need qualitative data to know how to fix it.

Chances are, you’ll use them at different times. Qualitative research can be done during the initial design phase, once you have a working product, or during a redesign. It’s especially valuable at the beginning of a design process because it can help you focus on what your users need and why. Quantitative research is generally done only when you have a working product (either at the beginning or end of a design cycle), so you can measure the results of a design or change.

Want to learn more about how data-driven design can improve your platform performance? We’d love to help. Contact us today to schedule a call.