You’ve just rolled out an important new feature on your platform, and it’s time to answer the all-important question: is it getting the results you want? If you’ve set up an analytics tool, you can look at performance indicators like registrations, logins, downloads, or shares. But that kind of quantitative data will only get you so far.
Let’s say that new feature isn’t having the impact you’d hoped for — maybe registrations are lacking or engagement is low. You have a problem you need to solve, but you don’t have any information about why it’s happening. And you may have an entirely different underlying problem you need to address.
Where can you find actionable information? Enter qualitative research.
By answering the why behind what’s happening, qualitative data provides context for problems that surface through quantitative analysis. It helps you uncover the root of the problem you have and can also reveal problems you didn’t even know existed.
In this article, we’ll cover how to use both types of research to inform your platform design.
First, the Numbers: Quantitative Research
When you’re evaluating the performance of a digital platform, a good place to start is the cold, hard numbers. Quantitative research provides numerical data that can indicate, at a glance, whether your platform is meeting your business objectives. It can also show the scale of any problems and help prioritize which ones to address.
One major benefit of quantitative data is benchmarking. Tracking your data over time reveals whether UI changes are producing the results you want — and can help you measure the ROI of your efforts. You can also compare your data to an industry benchmark or a competitor’s stats as a barometer for your own performance.
Here are some examples of quantitative research methods:
Web analytics
This data describes what people are doing with your platform: where they go, what they click on, what features they use. It’s good for finding problems and monitoring the performance of content or features.
A/B testing
Here, you’re using experiments to compare different UI designs. By creating two live versions of the same element, like a call-to-action button, you can see which one performs best. Learn more in our article on A/B testing.
Surveys and questionnaires
Surveys let you gather information about your users’ preferences, attitudes, and behaviors, and they can produce a combination of quantitative and qualitative data. For easy-to-capture numerical data, use techniques like ratings and multiple-choice questions.
Usability testing
By measuring user experience with hard data, you can test how easy (or not) a platform feature is to use. Let’s say you just released a reminder function, and you want to know if users can create a reminder in two minutes or less. You can run a test where you ask participants to set a reminder, and measure what percentage are able to complete the task within two minutes.
Now that you’ve got a sense of what users are doing on your platform, let’s look at ways to learn why and how they’re doing it.
Now for the Words: Qualitative Research
Qualitative research can help you investigate why something is happening, identify ways to fix problems, and even determine whether you should phase out a feature or redesign it. Using detailed, contextual descriptions of users’ experiences, you can dive deeper into exactly which elements are working well and which are problematic.
Quantitative and qualitative research both provide useful data, but they’re more powerful when used together.
Unlike with quantitative research, you don’t need a ton of data points to get usable info. For example, if you see five customers in a row walk into the corner of a display in a retail store’s entrance, you can safely assume that most visitors will do the same thing.
You may be avoiding qualitative data because it seems expensive. And some techniques, like focus groups, require a greater investment than others. But, because you don’t need an enormous amount of data, qualitative research can be very cost-effective. It might even save you money by helping you identify and fix problems faster.
Here are some examples of qualitative research methods:
User Interviews
There are a number of different ways to handle user interviews, depending on the type and specificity of info you’re looking for. Here are a few:
- Talk to a subset of your platform users and ask what they like, don’t like, and why. What could be improved?
- Listen to users narrate their experience as they move through your platform, to learn how they feel about particular elements or tasks.
- Give test subjects a post-task survey, to capture their experiences while they’re fresh.
Focus Groups
These are similar to user interviews, but they’re done in a group setting. The advantage of a group is that it can often generate more feedback, as people tend to open up when they hear the experiences of others. Just be sure you have a moderator who gives everyone a chance to speak.
Field Studies
What people say they do… is often not what they actually do. Watching platform users in their natural environment can reveal gaps in your understanding of the user experience. You can use direct observation, interviews, contextual inquiry, and usability tests to learn how people do things and why they do them in particular ways.
Diary Study
This method asks users to document their experiences over time, making it useful for understanding longer-term behaviors. You can learn things like what motivates people to use certain features, what they’re trying to accomplish, how they feel, and what their overall journey looks like.
User Surveys
As opposed to quantitative surveys, qualitative surveys use open-ended questions to learn what users think and feel in their own words. One common pitfall: avoid leading questions. Instead of asking, say, “How easy was it to find the info you needed?”, ask “Describe your experience looking for that information.”
Like Peanut Butter & Jelly
Quantitative and qualitative research both provide useful data, but they’re more powerful when used together. Remember that quantitative data can tell you when there’s a problem with your platform design, but you’ll need qualitative data to know how to fix it.
Chances are, you’ll use them at different times. Qualitative research can be done during the initial design phase, once you have a working product, or during a redesign. It’s especially valuable at the beginning of a design process because it can help you focus on what your users need and why. Quantitative research is generally done only when you have a working product (either at the beginning or end of a design cycle), so you can measure the results of a design or change.
Want to learn more about how data-driven design can improve your platform performance? We’d love to help. Contact us today to schedule a call.
THE BRIEF
Wingspans’ primary audience is digital natives — young, tech-savvy users who expect fast, frictionless interactions and relevant content. Fail to deliver, and they’ll abandon you in a heartbeat.
The new platform needed to provide a scalable, flexible foundation for a range of content and tools being developed by the Wingspans team. We had to turn a collection of disparate pieces — story content, user data, school information, and more — into a cohesive digital framework that could grow and evolve. Above all, Wingspans needed a design-first approach, wrapping the educational aspects in an intuitive, engaging digital experience.
THE APPROACH
While storytelling formed the heart of the Wingspans platform, the site’s interactive features would be crucial for getting students to explore and engage with the content. Building on Lindsay’s familiarity with the educational market, we mapped out the content architecture, workflows, and functions for a host of interactive features to keep students engaged.
For the tech stack, we turned to a mix of microservices to provide a stable, flexible, and scalable architecture with lightning-fast performance. These included a Gatsby front end, Firebase database, AWS cloud storage, Algolia site search, Cosmic JS content management system, and more. We also worked to ensure the technology reflected Lindsay’s empathy-driven approach. For instance, we customized Algolia to deliver search results specifically tailored to a student’s profile and interests—in other words, an encyclopedia that understood its users and presented its information in a distinctly human way.
THE RESULTS
The platform’s most impactful feature is how easily students can find and bookmark career stories that resonate with who they are. With over 700 stories and 40 mini-documentaries available, each with an associated set of lessons, the site’s personalized search function and ultrafast content delivery are key. On the backend, the customized CMS and robust content architecture make it easy for the Wingspans team to align content with users’ profiles and browsing activity.
Bringing it all together, the Career Builder feature lets students select stories and content to create a customized career roadmap that they can share with parents, teachers, and counselors. A core element of the platform’s personalized user experience, the Career Builder brings Wingspans’ central premise to life: If you can see it, you can be it.
Oomph really fulfilled their commitment to building an immersive and radically personal platform that brought my vision to life.
— Lindsay Kuhn, Wingspans Founder and CEO
THE BRIEF
When Seraphic Group’s founder, Zach Bush, MD, saw patterns in people’s health linked directly to problems with the food supply, he became an advocate for regenerative farming. As a potential solution to deteriorating public health, global warming, and even poverty, regenerative farming offers benefits for local and global communities. But, getting farmers to switch to it from conventional techniques is a challenge.
Regenerative farming is good for the environment and the economy in the long run—but, short term, it’s more work and more expensive than chemical-heavy, conventional farming. Add in that the appropriate techniques depend on variables like geography, soil type, and climate, and it’s a difficult thing for people to figure out on their own.
Their platform idea, Atlus∗U, needed to not only educate farmers about regenerative agriculture, but also motivate them to try it, and stick with it, for the long haul.
THE APPROACH
Understanding the Educational Purpose
As we noted in an article on different types of online learning platforms, a platform’s educational purpose determines the tools and features that will best achieve its objectives. Atlus∗U spans two purpose categories, Student Stakes Learning and Broad Stakes Learning, which means that effective education is crucial for both the learners and their larger communities.
To that end, our design vision focused heavily on content comprehension, along with keeping users motivated and engaged. Our framework included educational content and tools, accountability systems, and community features. A key component was personal stories: sharing the experiences of farmers who had successfully converted their businesses to regenerative farming and could help and encourage others to do the same.
Above all, Seraphic wanted Atlus∗U to grow and evolve over time as a kind of living guide to regenerative farming. While most online learning platforms stop when the coursework ends (think of a CPR course, where you get a certificate and you’re done), for this platform, the end of the coursework was just the beginning of the journey.
THE RESULTS
In our design, the whole community drives the learning experience, not just the teachers and coursework. It’s easy for students to connect with others who are taking the same courses, while members-only forums provide a place for productive networking, questions, stories, and support. Some forums are attached to specific lessons, so that the dialogue isn’t just between teachers and students; all members, including alumni, can participate and share their learnings on a given topic.
Another component, the accountability partner system, was crucial for achieving Seraphic’s goal of driving lasting change. Research shows that publicly sharing a goal gives people a 65% chance of success, while reporting to a specific accountability partner boosts that chance to 95%.
Finally, our learning tools were designed to enhance both content comprehension and retention. Course videos were a key feature, designed not just for the course, but for reference over time. Students have the ability to bookmark videos and attach notes to specific sections, letting them revisit important info whenever they need it.
THE IMPACT
While online learning has been around for a long time, recent advancements in design and functionality make it possible for learning platforms to have a transformative impact on individuals and across society.
In the case of Atlus∗U, it’s not just the coursework that drives users’ learning; an entire community is mobilized to help you succeed. With a focus on collaborative, lifelong learning, our design brings together farmers from around the world to improve their business, grow healthier food, and protect our world.
Need help building an effective online learning platform? Let’s talk about your goals and how to achieve them.
The Challenge
Execute on a digital platform strategy for a global private equity firm to create a centralized employee destination to support onboarding, create interpersonal connections between offices, and drive employee satisfaction.
The key components would be an employee directory complete with photos, bios, roles and organizational structure; News, events, and other communications made easily available and organized per location as well as across all locations; The firm’s investment portfolio shared through a dashboard view with all pertinent information including the team involved.
These components, and the expected tactical assets that an intranet provides, would help the firm deepen connections with and among employees at the firm, accelerate onboarding, and increase knowledge sharing.
The Approach
Supporting Multiple Intentions: Browsing vs. Working
An effective employee engagement platform, or intranet, needs to support two distinct modes — task mode and explore mode. In task mode, employees have access to intuitive navigation, quick page loading, and dynamic search or filtering while performing daily tasks. They get what they need fast and proceed with their day.
At the same time, a platform must also encourage and enable employees to explore company knowledge, receive company-wide communications, and connect with others. For this firm, the bulk of content available in explore mode revolves around the firm’s culture, with a special focus on philanthropic initiatives and recognition of key successes.
Both modes benefit from intuitive searching and filtering capabilities for team members, news, events, FAQs, and portfolio content. News and events can be browsed in a personalized way — what is happening at my location — or a global way — what is happening across the company. For every interaction within the platform, the mode was considered and influential of nearly all design decisions.
From a technical standpoint, the private equity firm needed to support security by hosting the intranet on their own network. This and the need to completely customize the experience for close alignment with their brand meant that no off-the-shelf pre-built intranet solution would work. We went with Drupal 8 to make this intranet scalable, secure, and tailor-made to an optimal employee experience.
The Results
The platform deployment came at a time when it was most needed, playing a crucial role for the firm during a global pandemic that kept employees at home. What was originally designed as a platform to deepen employee connections between offices quickly became the firm’s hub for connecting employees within an office. As many businesses are, the firm is actively re-evaluating its approach to the traditional office model, and the early success of the new platform indicates that it is likely to play an even larger role in the future.
THE BRIEF
The RISD Museum publishes a document for every exhibition in the museum. Most of them are scholarly essays about the historical context around a body of work. Some of them are interviews with the artist or a peek into the process behind the art. Until very recently, they have not had a web component.
The time, energy, and investment in creating a print publication was becoming unsustainable. The limitations of the printed page in a media-driven culture are a large drawback as well. For the last printed exhibition publication, the Museum created a one-off web experience — but that was not scalable.
The Museum was ready for a modern publishing platform that could be a visually-driven experience, not one that would require coding knowledge. They needed an authoring tool that emphasized time-based media — audio and video — to immediately set it apart from printed publications of their past. They needed a visual framework that could scale and produce a publication with 4 objects or one with 400.
THE APPROACH
A Flexible Design System
Ziggurat was born of two parents — Oomph provided the design system architecture and the programmatic visual options while RISD provided creative inspiration. Each team influenced the other to make a very flexible system that would allow any story to work within its boundaries. Multimedia was part of the core experience — sound and video are integral to expressing some of these stories.
The process of talking, architecting, designing, then building, then using the tool, then tweaking the tool pushed and pulled both teams into interesting places. As architects, we started to get very excited by what we saw their team doing with the tool. The original design ideas that provided the inspiration got so much better once they became animated and interactive.
Design/content options include:
- Multiple responsive column patterns inside row containers
- Additionally, text fields have the ability to display as multiple columns
- “Hero” rows where an image is the primary design driver, and text/headline is secondary. Video heroes are possible
- Up to 10-colors to be used as row backgrounds or text colors
- Choose typefaces from Google Fonts for injection publication-wide or override on a page-by-page basis
- Rich text options for heading, pull-quotes, and text colors
- Video, audio, image, and gallery support inside any size container
- Video and audio player controls in a light or dark theme
- Autoplaying videos (where browsers allow) while muted
- Images optionally have the ability to Zoom in place (hover or touch the image to see the image scale by 200%) or open more
There are 8 chapters total in RAID the Icebox Now and four supporting pages. For those that know library systems and scholarly publications, notice the Citations and credits for each chapter. A few liberally use the footnote system. Each page in this publication is rich with content, both written and visual.
RAPID RESPONSE
An Unexpected Solution to a New Problem
The story does not end with the first successful online museum publication. In March of 2020, COVID-19 gripped the nation and colleges cut their semesters short or moved classes online. Students who would normally have an in-person end-of-year exhibition in the museum no longer had the opportunity.
Spurred on by the Museum, the university invested in upgrades to the Publication platform that could support 300+ new authors in the system (students) and specialized permissions to limit access only to their own content. A few new features were fast-tracked and an innovative ability for some authors to add custom javascript to Department landing pages opened the platform up for experimentation. The result was two online exhibitions that went into effect 6 weeks after the concepts were approved — one for 270+ graduate students and one for 450+ undergraduates.
THE BRIEF
Transform the Experience
The core Earthwatch experience happens outdoors in the form of an expedition — usually for about a week and far away from technology in locations like the Amazon Basin, Uganda, or the Great Barrier Reef. But before this in-person experience happens, an expedition volunteer encounters a dizzying array of digital touchpoints that can sow confusion and lead to distrust. Earthwatch needed “Experience Transformation.”
SURVEY THE LANDSCAPE
Starting with a deep strategy and research engagement, Oomph left no stone unturned in cataloging users and their journeys through a decade’s worth of websites and custom applications. We were able to conduct multiple interview sessions with engaged advocates of the organization. Through these interviews, the Earthwatch staff learned how to conduct more interviews themselves and listen to their constituents to internalize what they find wonderful about the experience as well as what they find daunting.
CREATE THE MAP
With a high-level service blueprint in place, Oomph then set out to transform the digital experiences most essential to the organization: the discovery and booking journey for individuals and the discovery, research, and inquiry journey for corporate sustainability programs.
The solution took shape as an overhaul and consolidation of Earthwatch’s public-facing websites.
THE RESULTS
The Journey Before the Journey
A fresh design approach that introduces new colors, beautiful illustrations, and captivating photography.
Expedition discovery, research, and booking was transformed into a modern e-commerce shopping experience.
Corporate social responsibility content architecture was overhauled with trust-building case studies and testimonials to drive an increase in inquiries.
IN THEIR WORDS
The Oomph team far surpassed our (already high!) expectations. As a nonprofit, we had a tight budget and knew it would be a massive undertaking to overhaul our 7-year-old site while simultaneously launching an organizational rebrand. Oomph helped to guide us through the entire process, providing the right level of objective, data-driven expertise to ensure we were implementing user experience and design best practices. They listened closely to our needs and helped to make the website highly visual and engaging while streamlining the user journey. Thanks to their meticulous project management and time tracking, we successfully launched the site on time and exactly on budget.
ALIX MORRIS MHS, MS, Director of Communications, Earthwatch
THE BRIEF
The American Veterinary Medical Association (AVMA) advocates on behalf of 91,000+ members — mostly doctors but some veterinary support staff as well. With roots as far back as 1863, their mission is to advance the science and practice of veterinary medicine and improve animal and human health. They are the most widely recognized member organization in the field.
Make the Brand Shine
The AVMA website is the main communications vehicle for the organization. But the framework was very out of date — the site was not mobile-friendly and some pages were downright broken. The brand was strong, but the delivery on screen was weak and the tools reflected poorly.
Our goals were to:
IMPROVE THE SITE MAP
Content bloat over the years created a site tree that was in bad need of pruning.
IMPROVE SEARCH
When a site has so much content to offer, search can be the quickest way to find relevant information for a motivated user. Our goals were to make search more powerful while maintaining clarity of use.
COMMUNICATE THE VALUE OF MEMBERSHIP
Resources and benefits that come with membership were not clearly illustrated and while members were renewing regularly, they were not interacting with the site as a resource as often as they could.
STRENGTHEN THE BRAND
If the site was easier to navigate and search, if it had a clear value proposition for existing and prospective members, and if the visual design were modern and device-friendly, the brand would be stronger.
THE APPROACH
Put Members First
Oomph embarked on an extensive research and discovery phase which included:
- A competitor Analysis of 5 groups in direct competition and 5 similar membership-driven organizations
- An online survey for the existing audience
- A content and SEO audits
- Several in-person workshops with stakeholder groups, including attendance at their annual convention to conduct on-the-spot surveys
- More phone interviews with volunteers, members, and additional stakeholders
With a deep bed of research and personal anecdotes, we began to architect the new site. Communication was high as well, with numerous marketing, communications, and IT team check-ins along the way:
- An extensive card sort exercise for information architecture improvements — 200+ cards sorted by 6 groups from throughout the organization
- A new information architecture and audience testing
- A content modeling and content wireframe exercises
- A brand color accessibility audit
- Over a dozen wireframes
- Three style tiles (mood boards) with revisions and refinements
- Wireframe user testing
- A set of deep-dive technical audits
- Several full design mockups with flexible component architecture
Several rounds of style tiles explored a new set of typefaces to support a modern refresh of the brand. Our ideas included darkening colored typography to meet WCAG thresholds, adding more colored tints for design variability, and designing a set of components that could be used to create marketing pages using Drupal’s Layout Builder system.
THE RESULTS
The design update brought the main brand vehicle fully into the modern web. Large headlines and images, chunks of color, and a clearer hierarchy of information makes each pages’ purpose shine. A mega-menu system breaks complex navigation into digestible parts, with icons and color to help differentiate important sections. The important yearly convention pages got a facelift as well, with their own sub-navigation system.
FINAL THOUGHTS
Supporting Animals & Humans Alike
Membership to the AVMA for a working veterinary doctor is an important way to keep in touch with the wider community while also learning about the latest policy changes, health updates, and events. The general public can more easily find information about common pet health problems, topical issues around animal well-being during natural disasters, and food and toy recalls. The goal of supporting members first while more broadly providing value to prospective members and non-members alike has coalesced into this updated digital property.
We look forward to supporting animal health and human safety as we continue to support and improve the site over the next year.
THE BRIEF
“Out of Home” (OHH) advertising is widely understood as billboards, transit stops, and street furniture. But OHH advertising is also place-based mobile advertising and digital messages that can react to the time of day and weather. These emerging avenues were some of the customer education messages that OUTFRONT Media, one of the top three outdoor advertising companies in the country, needed a new website to convey.
OHH was new to us, but we enjoy diving deep into new industries and absorbing information from all directions. While getting to know OUTFRONT Media during our initial explorations and discovery meetings, Oomph and Straightline Media were surprised to learn:
THE DISCOVERY
Expanding Outfront’s reach from Agencies to Mom-and-Pops
OUR RESEARCH & UNDERSTANDING
“The United States of Audiences” is the demographic landscape made up of microcosms centered around interests and activities. OUTFRONT understands all these niche groups and can hyper target them to get businesses those audiences.
Through information architecture testing, an understanding of their business goals, and internalization of this new brand message, we crafted a main navigation that spoke directly to OUTFRONT’s target audiences. These pages communicate they “get” them in two ways — OUTFRONT understands these audiences’ needs and
CUSTOMER ACTIONS
To buy media placements from OUTFRONT, a business needs to:
- Understand the value of OHH advertising
- Explore the media options in their location
- Investigate the costs of these different options
- Call a sales associate in their area to discuss custom packages
Finding a market lets a potential customer define themselves even further by gathering their location
they can put the right message in front of the right people to get clients new business.
Large brands already understand how important it is to have OHH in their advertising mix. An emerging market for companies like OUTFRONT is actually smaller businesses — local chains and even mom and pop businesses. To court these new customers, OUTFRONT has to create a customer portal that supports do-it-yourself media management, and the website messaging has to educate a small business about the process and how to buy.
data. From here, they can easily access the media that is available specifically to their geographic area. After a sale is made, a customer needs continuing support. The website should:
- Make it easy for a customer to find production specs for various media
- Provide links to a portal that give them analytics and reporting statistics
- Give them a clear path to extending their current campaign or starting a new one
THE APPROACH
Defining Audience Destinations
OUTFRONT narrowed its focus to the audiences that it needs to win and continue to support — new advertisers, new and existing agency relationships, and property owners who lease their real estate to OUTFRONT for displays (these could be individual real estate owners, large municipalities, or large destinations like airports, stadiums, and entire city transit systems). The journey for these new customers is the main navigation, and these journies funnel through the Market Finder to become more personalized.
Returning customers will find inspiration, case studies, galleries, and support resources in the footer navigation. One of the reasons for this bifurcation was the amount of content that OUTFRONT Media needs to manage. Their previous navigation tried to be everything to everyone, and therefore, was confusing. Labels like “Who We Are / What We Do / Where We Are…” contained too much “we” and not enough “you”. A modern customer doesn’t have time to figure out the navigation — it should speak to them directly in language that they understand.
Giving Visitors Clear Directions
he previous market finder used “inside baseball” language — terms that people at OUTFRONT understand, but that the general populace does not. Markets were labeled by city and state with vague geographic terms like “non-metro”. It was not friendly to the DIY customer. We helped guide technical conversations around how a better flow might work and captured the journey in wireframes and external working examples.
The Market finder is an integral part of the customer journey, so we placed it in page content to support the education process. As a customer understands how OHH and OUTFRONT works, they are prompted to enter their location and get more direct information. As part of wireframes and design, the language was an important aspect to try out and sharpen. Vague button text like “Get Started” was replaced with stronger and more descriptive actions like “Start Building Your Campaign.”
THE RESULTS
Focusing Attention on Results
OHH is a powerful mechanism to reach people even as consumers become blind to advertising in traditional media — a billboard or subway poster can’t be ad-blocked. One of the most powerful ways to tell this story is through data. OUTFRONT has a number of “Insights” but potential customers are less likely to seek this information out themselves. Instead, we made sure to include teasers along their journies that highlight the effectiveness of sample campaigns. These Insights provide powerful results with brands that visitors are familiar with.
The entire journey from discovery to wireframing, testing, and design was an exciting and challenging process that pushed both teams towards an excellent outcome. We hope that the clarity of the navigation, message, and visitor journey continues to get new eyeballs and customers.