Imagine you own a clothing store and want to take a leap into the world of e-commerce by launching an online platform. You’ve never built a website before, but with help from some friends and a few YouTube tutorials, you develop a prototype site to sell clothes on. Before launching, you want to ensure your website is user-friendly so that people can easily navigate it and make purchases. 

In order to evaluate the interface of your website, you conduct user testing, recruit participants, and ask them to find and purchase a specific item to test the functionality of the prototype. By observing them while they navigate the site, you can find any pain points they encounter while trying to make the purchase.¹ You observe one participant who is having trouble filtering the clothes by size. This insight allows you to make changes to your website to improve the user experience by making the filtering function easier to use. Now that you’ve enhanced your interface based on iterative user testing, you’re ready to launch your website!

User testing is a crucial tool for identifying and resolving issues related to the usability of a website or app. Although companies will first engage in internal testing, getting employees and developers to test the product, these people are familiar with the product and can have a biased perspective. Alternatively, user testing is performed with actual end users who represent the target audience, providing a fresh perspective on how people will actually interact with the product when it is launched.

There are five common methods of user testing:

• Usability Testing: Observing participants as they interact with an electronic interface to evaluate its efficiency and usability. The observation doesn’t always mean standing behind the participant (which can be daunting and skew results); it can also be analyzing their user journey (how they click from page to page) or heat maps (seeing where their cursor spends the most time) from the backend.¹ 
• Surveys: Collecting quantitative feedback from users to understand customer satisfaction. These are usually given or sent to actual end users after they have interacted with a new feature of your product to see how it is performing.¹ 
• A/B Testing: A form of user testing where participants are split into two different groups that each interact with a slightly different interface, helping you decide which version is better.²
• Focus Groups: Leading a discussion with a small group of real end users to gain qualitative insight into your product.
• Beta Testing: Similar to usability testing, but conducted at the final stages before your product or new feature goes live, usually focused on identifying bugs and resolving small functional issues.²

By conducting user testing through one or a combination of these methods, designers gain a better understanding of how people will actually interact with their product, enabling them to make adjustments to deliver the best product possible.

Usability Testing: A method of testing the functionality of a product, such as a website or an app, by observing how real end users interact with it. It focuses on how easily and effectively the product can be used. Usability testing is one component of user testing that places a greater focus on whether the product meets the needs of its target audience.

User Experience (UX): The overall experience—including feeling, perception, and satisfaction—that a customer gets while interacting with a product. It can refer to both physical and digital attributes of the product, how accessible it is, and how it performs (e.g., its speed or reliability). Prioritizing UX helps businesses create products that effectively meet user needs.⁴

UX Research: The process of studying users' behaviors, needs, and motivations to inform the design and development of a product. This research is focused on evaluating aspects of the user experience and relies on qualitative methods like interviews as well as quantitative metrics such as eye-tracking, heat maps, and user journey mapping.

User Interface: The space where people interact with technology, often a computer or phone screen. It refers to the part of the machine where inputs are received, and outputs are delivered (e.g., when you click a link on a website and a pop-up appears), placing focus on the visual and interactive elements of the product. The goal for companies is to create products that have an engaging and easy-to-use interface, decreasing the number of inputs required for a desired output. That’s why you often see a “Go Straight to Checkout” page, allowing customers to skip the cart and instantly make their purchases.

User Persona: A research-based profile created by a business that reflects the demographics, characteristics, and behaviors of people within their target audience. Creating various user personas during the design process allows companies to build the product knowing how their target audience will engage with it and ensuring it appeals to each persona. 

Prototype: A preliminary model of a product that is used for user testing before it is launched. There are two types of prototypes: low-fidelity and high-fidelity. Low-fidelity prototypes are simplified versions of the product, often one of the first iterations, that are tested to see if their core functionality works. High-fidelity prototypes are more advanced versions that are near-finished products. These prototypes are likely to be used in beta testing right before the product is launched to resolve any remaining minor issues.¹ 

Pain Points: Any area within the user journey where a user encounters difficulties or frustrations. One of the goals of user testing is to eliminate pain points.²

In the early twentieth century, before the term user testing was coined, ergonomics and “human factors” were discussed in industry and the military, with a focus on creating products and systems to improve the efficiency and productivity of people in their work environment. They explored how people interacted with the products required to succeed at their jobs and used these insights to adapt designs. For example, the military wanted to make weapons that would be easy and safe to learn and use, gathering intel on various metrics to answer questions such as “How quickly will a new crew member learn how to use the artillery cannon?” or “How does a design improvement decrease soldier fatigue?”5

This was the start of factoring in the mechanics of human interaction when developing a new product. The first usability lab was founded by Bell Labs with help from pioneers like John E. Karlin, who believed that the human experience should be the top priority in design. Instead of focusing on training people to be able to use complex technology, Karlin believed technology needed to conform to human behavior. 

Still, at this time, usability was a discipline only explored by a few people in select industries, mostly within academia and phone companies like Bell Labs.⁶ It wasn’t until computer prices dropped in the 1980s that usability boomed: now, most employees had their own personal computers, requiring the design to be adjusted to ensure the average user could competently operate them.5 Apple was one of the first companies to focus on user-centered design, building computers meant for the average person—soccer moms or even kids.⁷

With usability at the front of every company’s mind by the 1990s, usability engineer Don Norman coined the term “user experience” when he joined Apple.⁸ Norman emphasized the need to test products with real users and develop new iterations in response to their feedback.⁹ However, developers at this time were resistant to user testing, as it required precious time and resources. It wasn’t until Jakob Nielsen, named the “king of usability” by Internet Magazine, developed a short checklist of usability considerations (later known as the 10 usability heuristics) and proposed that only five users needed to be tested to find 80% of an interface’s usability flaws, that developers were convinced.¹⁰ User testing no longer seemed like a daunting or resource-intensive task and could be adopted by various industries and companies, no matter their size.

With the rise of the internet, mobile devices, and e-commerce in the 2000s, user testing became an essential step in product development. This period also marked the ability for companies to employ remote usability testing methods instead of in-lab tests, making it a more accessible tool. Dozens of remote UX research tools like UserTesting, Loop11, and WebEx entered the market.¹¹ Companies began hiring UX researchers and designers, putting usability at the center of every product. Today, advances in AI and automation have expanded the ways that user testing can be conducted, allowing for the analysis of user data in real-time.¹²

John E. Karlin

A psychologist and electrical engineer who worked for Bell Labs and is often referred to as the father of usability.¹³ Before joining Bell Labs, Karlin worked in military communications research at Harvard.¹⁴ Karlin is responsible for many of the old dial phone design features that we are (or used to be) familiar with, such as the shape of the keypad and the arrangement of the numbers. Karlin put the user first, figuring out how the arrangement would help people remember numbers and whether it was easiest to push round or square buttons.¹⁵ 

Don Norman

An author, professor, and corporate advisor often referred to as the “father of user experience” and one of the first to center humans at the heart of design.¹⁶ He coined the term “user experience” during his time at Apple and was the first UX Architect at the company. He went on to establish the University of California San Diego Design Lab, which applies design thinking and behavioral science principles to solve complex problems in areas such as healthcare, education, and urban planning.¹⁷

Jakob Nielsen

A Danish consultant and usability pioneer known as the “king of usability” who introduced heuristic evaluation to user testing.¹⁸ He has authored multiple best-selling books about usability and holds 79 patents related to making the internet easier for people to navigate, most of which he filed for during his time at Sun Microsystems.¹⁸ One such patent is for adaptive font sizes, making it possible for systems to remember a user’s font preferences and adjust the text on a website to match it.¹⁹

User testing plays a critical role in shaping products that truly meet user needs by providing actionable insights throughout the development process. From enhancing satisfaction to uncovering hidden opportunities and reducing costs, user testing helps businesses create intuitive, user-friendly experiences that drive long-term success.

Improved User Satisfaction

Usability is all about putting humans at the center of design, so if that’s the case, it makes sense to test products on real end users. Instead of guessing what aspects of an interface will resonate with people—and which will frustrate them beyond belief—user testing allows companies to make informed decisions based on user needs and experiences. 

While product development teams may think they know best, user testing demonstrates whether your product or feature is intuitive to your target audience. Developing user personas to match the different kinds of people included in your target audience allows you to recruit participants for user testing that ideally match up with actual end users.² 

When someone feels that your product or feature reflects their needs, they feel understood, which increases customer satisfaction and loyalty.²⁰ For example, imagine a coffee shop launching a mobile app where customers can order ahead. Through user testing, the company has identified that one of the most important features for customers is being able to easily customize their drinks (you can thank Starbucks for starting this trend!). As a result, they emphasize the customization button and make it easy to adjust your order. Since we’re usually half-asleep before we drink our morning coffee, if a company makes it easy for us to get our caffeine fix, we’re going to keep going back!

Uncover Hidden Opportunities

User testing not only helps identify bugs and pain points but can also uncover untapped opportunities. This is especially true for user testing methods like user interviews, where companies gather qualitative information about how users would like to engage with their product or service.²¹ The findings may be different from what the company was originally searching for, but if the point is to make a product customers want, then user testing illuminates a path to success.

For example, Slack was born thanks to user testing. Initially, a gaming company called Tiny Speck was trying to create a multiplayer game called Glitch. Though the game wasn’t successful, they learned from speaking with users that the internal communication aspect really resonated with them. This helped the founders to pivot and design an internal chat system called Slack for teams to use during game development. Quickly, Slack expanded into other fields and is now one of the most popular tools for company communication and collaboration.²² 

Reduced Development Costs and Time

Although it does require some time and effort to conduct user tests, utilizing them throughout the design iteration process can actually save companies a lot of time and effort. User tests help uncover pain points early on, allowing teams to fix issues throughout the development cycle instead of only at the end after the product has already gone to market.² The low cost of remote testing tools and the small number of participants required (as little as five, according to Jakob Nielsen) allows for companies to conduct user testing multiple times throughout development, resulting in a finished product that will likely have a higher adoption rate from consumers.²⁰

It is widely accepted that user testing is a critical tool for success. However, how exactly that testing is conducted can raise challenges. Businesses must balance the need for adaptability with quality, find ways to minimize bias and address ethical concerns related to informed consent.

Balancing Speed versus Accuracy

When it comes to user testing, researchers have to find the right balance between speed and accuracy. In today’s digital age, where consumer expectations evolve quickly and competition is fierce, businesses need to iterate and adapt at an unprecedented pace. Businesses want to release their product quickly, before a competitor beats them to it, and continuously improve as user preferences can quickly shift. This need pushes businesses to keep user testing relatively simple and rely on small sample sizes.

However, prioritizing speed can negatively impact quality.²³ It may cause the insights gathered from user testing to only scratch the surface and not be an accurate reflection of how the product will be received by consumers. More in-depth insights require more time- and resource-intensive forms of user testing, such as user insights, which can cause delays in launches or adjustments.

Whether a company chooses to focus on speed or accuracy might depend on the type of product or service it provides. For example, if a company is launching an online banking system, the sensitive and private nature of the information stored means that accuracy is of utmost importance. The company must ensure there are no major bugs that would cause any of the data to be leaked.²³ Alternatively, a game company that is creating new levels may care more about being consistent with their releases because the consequences of a bug are much less serious. These businesses may adopt more of a “launch-and-learn” approach, not worrying about delivering a perfect product on the first go.²⁴

Potential for Bias

Although user testing involves real end-user participants, that doesn’t necessarily mean that their insights reflect the real-world experience. In any experiment, various biases can affect the results. People who agree to sign up for user testing may be more likely to already like the product or brand, or the response bias can cause people to share insights they think the researcher is looking for. Participants may emphasize what they like about the product instead of offering constructive criticism for how it could be improved. The Hawthorne effect—improved performance due to awareness of being observed—can also come into play. If you ask a participant to find a specific item of clothing on your website, they are likely to put in more effort to complete the task during the test, whereas in their everyday life, they may give up if the task proves to be difficult. 

There are ways to mitigate bias in user testing, such as by avoiding leading questions.²⁵ Instead of asking a participant a question like, “What do you like about the website?” you can try a more open-ended approach, such as, “Tell me about your experience navigating the website.”

Ethical Risks

One way to minimize biases like the response bias and the Hawthorne effect in user testing is to conduct tests on participants who are not aware they are participating in a test. However, this brings up a new set of issues, namely ethical ones. Ethical research requires experimenters to gain informed consent from participants so that they are aware their behavior is being monitored and data is being collected, but this can impact how they behave.

In 2012, Facebook was criticized for changing the information that showed up on users’ news feeds to see if they were influenced by the news they consumed without informing users of the experiment. Their user test revealed that people who saw more positive posts contributed more positive posts, and those who saw negative posts were more negative in their own posts, suggesting people’s emotions are impacted by what they consume online.²⁶ When the paper came out, revealing that over 600,000 Facebook users were unknowingly part of a user test, it sparked outrage online. Facebook responded by claiming that users give blanket consent when signing up for the service. Although this test seemed unethical, if Facebook had told users they were conducting an experiment on how newsfeed posts impacted their emotions, it’s possible people would have thought more critically about the media they consume and post.

We often do provide consent to data collection when we visit a website, but lack of clarity on how our data is being used can cause us to feel manipulated. Many people are unaware that what they do online is constantly being tracked by businesses for user testing. The 2012 Facebook incident reveals some of the ethical risks involved in user testing that doesn’t explicitly gain informed consent, highlighting the importance of transparency when conducting tests, especially those that impact people’s thoughts, emotions, and behaviors.

Airbnb’s Booking Experience

When Airbnb was created, the purpose of the site was for hosts to rent out air mattresses. Today, it is a global phenomenon and a massive success. A lot of Airbnb's growth and success can be attributed to user testing.

When founders Joe Gebbia and Brian Chesky first launched Airbnb, they struggled to attract users or investors to their website. If they wanted to avoid failure, they needed to act quickly. After advice from a mentor, they decided to travel to New York and meet with some of the hosts who had signed up for their website. Their initial plan was to help hosts take professional photos of their homes to create more attractive listings, but what Gebbia and Chesky gained was much more valuable than they could have expected.

During these meetings, hosts showed Gebbia and Chesky the process of listing their homes and communicating with guests. The founders quickly realized that the process was far too complicated, causing many users to give up. They believed they had created a simple interface, but user testing highlighted many flaws. They made some quick adjustments to the website and saw immediate results. Seven days later, Airbnb’s weekly average revenue had doubled.²⁷ 

Gebbia and Chesky had accidentally stumbled into user testing, but this experience laid the foundation for rigorous testing practices that have been maintained throughout Airbnb’s evolution. Gebbia and Chesky focused on user experience, reaching out to hosts and guests to understand their needs and any pain points they encountered while using the platform. To this day, Airbnb regularly collects feedback through surveys and user reviews and implements changes to ensure that visitors can easily navigate the website.²⁸ 

Launching the McDonald’s Mobile App

You may think a company as big and successful as McDonald’s—which serves 62 million customers daily—intuitively understands its customers and may not need to conduct user testing when it launches a new product, but you’d be wrong!

McDonald’s was late to the smartphone app game, so when they began developing one in 2015, they wanted to ensure they got it right. They hired SimpleUsability, a UK-based market research company, to conduct user testing. SimpleUsability conducted 20 usability tests to gain insights into every aspect of a customer’s journey on the app, as well as 225 user interviews. User testing revealed that consumers found the call-to-action buttons hard to find and experienced difficulty when trying to customize their order. These insights helped McDonald’s make adjustments to the app before launching to better address customer preferences.²⁹

Using the Power of Behavioral Science to Elevate User Experience (UX) Design

With the plethora of choices afforded by today’s digital landscape, for a product to succeed, it has to have an engaging user experience. Behavioral insights can tell us a lot about how people like to interact with a product and what will enhance the customer experience. In this article, Head of UX & Design at ITML Matthaios Mantzios explores how UX can be designed in a way that will play to people’s emotional response—how they will feel when exploring the interface—in an ethical manner. 

The Mistake That Almost Half of Product Managers Make

User testing provides valuable data that can be analyzed to inform design changes. However, to be effective, the data must be interpreted correctly. As our writer Turney McKee states in this article, many people will not make a change if it is deemed to be statistically “insignificant” in terms of improving user experience. McKee explores why managers should not be so quick to dismiss user testing data even if it doesn’t quite meet the confidence threshold.

1. Omniconvert. (2024, October 21). What is user testing? Omniconvert. https://www.omniconvert.com/what-is/user-testing/
2. Baker, K. (2022, June 10). User testing: The ultimate guide to improving your website or product. HubSpot. https://blog.hubspot.com/service/user-testing
3. User Interviews. (2023, September 7). User research and UX quotes to inspire you and your team. User Interviews. https://www.userinterviews.com/blog/user-research-and-ux-quotes-to-inspire-you-your-team
4. Interaction Design Foundation - IxDF. (2016, June 1). What is User Experience (UX) Design?. Interaction Design Foundation - IxDF. https://www.interaction-design.org/literature/topics/ux-design
5. Soegaard, M. (2012, May 23). The history of usability: From simplicity to complexity. Smashing Magazine. https://www.smashingmagazine.com/2012/05/the-history-of-usability-from-simplicity-to-complexity/
6. Nielsen, J. (2008, April 20). 25 years in usability. Nielsen Norman Group. https://www.nngroup.com/articles/25-years-in-usability/
7. Wobbrock, J. O. (2024, January 19). Forty years ago, the Mac triggered a revolution in user experience. Smithsonian Magazine. https://www.smithsonianmag.com/innovation/forty-years-ago-the-mac-triggered-a-revolution-in-user-experience-180983623/
8. Lyonnais, S. (2017, August 28). Where did the term “user experience” come from? Adobe Blog. https://blog.adobe.com/en/publish/2017/08/28/where-did-the-term-user-experience-come-from
9. Sirk, C. (2024, January 23). The godfather of UX: Don Norman & user-centered design. CRM.org. https://crm.org/articles/the-godfather-of-ux-don-norman-user-centered-design
10. Dickinson, A. (2020, June 10). It’s not five users. Medium. https://medium.com/@anna.y.dickinson/its-not-five-users-c11da404f299
11. Stokes, T. (2023, September 24). The past, present, and future of usability. Drillbit Labs. https://depth.drillbitlabs.com/p/the-past-present-and-future-of-usability#footnote-2-137006865
12. Baufest. (2024, November 11). The revolution of artificial intelligence in UX testing. Baufest. https://baufest.com/en/revolution-artificial-intelligence-ux-testing/
13. Lea, G. J. (n.d.). John Karlin: Father of usability. Glenn J. Lea. Retrieved January 27, 2025, from https://glennjlea.com/usability/john-karlin-father-of-usability/
14. IEEE History Center. (n.d.). John E. Karlin. Engineering and Technology History Wiki. Retrieved January 27, 2025, from https://ethw.org/John_E._Karlin
15. Zweifel, D. (2013, March 18). Plain Talk: Remembering the man behind the keypad. Capital Times. https://captimes.com/news/opinion/column/dave_zweifel/plain-talk-remembering-the-man-behind-the-keypad/article_74c88a4e-73f6-5318-94b8-f29e44270b73.html
16. Norman, D. (n.d.). Don Norman. Britannica. https://www.britannica.com/contributor/Don-Norman/12354118
17. Interaction Design Foundation. (n.d.). Don Norman. Interaction Design Foundation. https://www.interaction-design.org/literature/topics/don-norman
18. UX Tigers. (n.d.). People. UX Tigers. https://www.uxtigers.com/about/people
19. Justia Patents. (n.d.). Jakob Nielsen patents. Justia Patents. https://patents.justia.com/inventor/jakob-nielsen?page=4
20. UserPilot. (n.d.). What is user testing?. UserPilot. https://userpilot.com/blog/what-is-user-testing/
21. Moran, K. (2019, December 1). Usability testing 101. Nielsen Norman Group. https://www.nngroup.com/articles/usability-testing-101/
22. Mathews, A. (2024, October 8). From game over to $1 billion: How Slack transformed workplace communication in just 8 months. AIM Research. https://aimresearch.co/market-industry/from-game-over-to-1-billion-how-slack-transformed-workplace-communication-in-just-8-months
23. Reichert, A. (2024, August 20). Speed vs. quality in software testing: Can you have both? TechTarget. https://www.techtarget.com/searchsoftwarequality/tip/Speed-vs-quality-in-software-testing-Can-you-have-both
24. Fox, G. (2017, October 25). To user test or not to user test: That is the question. Prototypr. https://blog.prototypr.io/to-user-test-or-not-to-user-test-that-is-the-question-c89a7d49579f
25. UserTesting. (2023, May 24). How to avoid bias in usability testing. UserTesting. https://www.usertesting.com/blog/how-to-avoid-bias-in-usability-testing
26. Goel, V. (2014, June 29). Facebook tinkers with users’ emotions in news feed experiment, stirring outcry. The New York Times. https://www.nytimes.com/2014/06/30/technology/facebook-tinkers-with-users-emotions-in-news-feed-experiment-stirring-outcry.html
27. Carroll, I. (2019, January 4). Think user testing is a luxury? Ask Airbnb. Telerik. https://www.telerik.com/blogs/think-user-testing-is-a-luxury-ask-airbnb
28. Ahmedkhan, S. (n.d.). Unlocking Airbnb’s success: Understanding user needs. Medium. https://medium.com/@shahanazahmekhan/unlocking-airbnbs-success-understanding-user-needs-d86939378bf2
29. Reno. (2020, October 5). Usability testing examples. Analysia.https://www.analysia.com/usability-testing-examples/