As with User Experience and Usability, there are some substantial differences between User Testing and Usability Testing. We are talking about different methods, targets and even phase of use in the life cycle of a product. Yet, they are two concepts that are often confused. Let's get straight to the point: what are the differences?
We have said that the differences concern the meaning, the objective, the phases of use and the method. Let's see them all.
Doing User Testing means understanding the needs of the user and validate the idea (which can be an app, a service, a product, etc.). It helps to understand how people satisfy that need today and if and how they would use an alternative to do so. With user testing you can also understand what the user persona or buyer persona is.
To understand if your users or potential customers are able to use your service or product, you can perform Usability Testing. For example, on an eCommerce you could test that users don't get stuck in the process and fail to continue.
So: User Testing has to do with functionality, while Usability Testing has to do with user behavior.
As you can tell from the meaning of the two tests and the image above, User Testing is about understanding whether or not people need the solution you're thinking about (or already working on). It also allows you to begin understanding who the target user of your product or service is. Let's summarize the objectives:
Usability Testing, on the other hand, is about understanding whether the solution is usable in practice or not. For example, in a restaurant booking app (think of TheFork), if users can make a reservation and understand that it has been sent and confirmed. With this test, you can discover the issues with the product and it is extremely useful to improve its functionalities. Again, let’s summarize the objectives:
At this point, the use phase is also quite intuitive. Since User Testing is used to understand if people need that solution, this step is clearly at the beginning of the product life cycle. To simplify, we can say that User Testing is undertaken as soon as you have an idea for a product.
To understand if the solution is suitable, indeed, usable, it is necessary to have a draft. The Usability Testing therefore is being carried out since a prototype or a design is made.
For both User Testing and Usability Testing, user involvement is essential.
In the case of User Testing, it is common to use focus groups and interviews, since we need to try to give an answer to questions like:
How do you currently solve this problem?
Have you ever thought of a better way to solve it?
How do you feel about this solution?
Would you pay to use this solution?
The goal of the focus group is to gauge the impressions of a testing group to find out what users really want from the solution. Through an interview you can investigate users' habits and how they solve or would be happy to solve a problem.
Things get more interesting with Usability Testing, since you already have a draft of the solution in hand and there comes an exciting time to find out if users can use it. This test focuses on how the user actually interacts with the product and whether the experience is simple, immediate and functional to the user's goals.
In this case, the questions to be answered are different, for example:
Can you register for this service in one minute?
How would you log in?
Usability Testing can be qualitative, quantitative, remote or in-person.
This test is used to collect insights into how users use the product, and is more common than quantitative testing. It helps to identify potential Usability frictions and areas of improvement of the digital service offered.
To carry it out, groups of testers who are on target with the digital service offered are hired and asked to use it (according to the objectives of the test). During the use, videos are recorded using the Thinking Aloud methodology and qualitative surveys are administered. A UX Specialist is involved in the test and defines the test script, writes the survey questions and the reports. The result of this test is a report with the usability frictions (observed in the users) and the suggestions (of the UX Specialist) to improve the service.
This type of testing identifies Usability frictions with the Thinking Aloud methodology and the analysis of quantitative metrics. It focuses on obtaining metrics that describe the user experience, i.e. effectiveness, efficiency and quality, and aims to establish benchmarking parameters. The process is similar to qualitative testing with the addition, obviously, of metrics measurement.
Even in this variant, the goal is to analyze users' interactions with the offered product and deepen what are the potential usability frictions to identify improvements to be made. As described above, testers record videos as they first interact with the product so that the UX Specialist can analyze usability frictions and propose improvements. The UX Specialist can be in-person, thus physically close to the user as they test the product, or remotely, analyzing the videos on a delayed basis.
The big difference is in the organization of these tests. An in-person test is extremely complex to organize because it must be able to fit in the agendas of UX Specialists, testers and even the marketing department working on the product. Travel must be taken into account and the timeframe is terribly long. In addition, it is not easily integrated in the perspective of an Agile development, in which the product is re-worked and re-submitted to the testers.
So: Are User Testing and Usability Testing the same thing? No.
The difference is that the User Testing helps to understand if the future users need that product or service, while the Usability Testing is used to discover if the users succeed to use the product or service on which you are working.
While the first one is useful at an early stage of the project to validate the idea, the second one comes into play at a more advanced stage, when there is already at least a draft or prototype of the project. As a result, the testing method also changes.
Said that, often the two terms are interchanged or used with the same meaning, that of Usability Testing, so don't worry if you get confused!
UNGUESS provides both services (User Testing and Usability Testing) and all the methods described in this article. In our Services page you won't find all the available tests, but only a selection of the most requested or "container" services of many other more specific services.
Curious to see what results you can get in the real world? Download the case study here below to see how UNGUESS helped Costa Crociere eCommerce increase its conversion rate by 25%.