How and why to effectively carry out UX research with a useful testing methodology to analyse apps and software before release to the public.
How and why to effectively carry out UX research with a useful testing methodology to analyse apps and software before release to the public.
Together with Luigi Greco, founder of Conflux, I wrote a white paper that explains how and why to effectively do UX research in crowdtesting. You can also download the content in white paper format at the bottom of the article.
Learn what makes crowdtesting a valid analysis tool and how it can be integrated as part of a structured UX research project
Crowdtesting is a useful testing methodology for analysing apps and software before they are released to the public, to track down bugs, and to verify the quality of digital touchpoints in an omnichannel perspective, also identifying any problems related to system usability and hence to the user experience.
The practicality, "remote" approach, and rapid speed of crowdtesting are very attractive elements for companies. In order to be effective and produce correct results, however, this testing methodology must be governed by expert user experience researchers.
In fact, using this methodology involves a series of factors that must be managed with suitable tools, for example, those for synthesising results that are aimed at reducing background noise. A substantial and complex amount of data is often collected in this type of project, thus mandating the creation of processes, methods, and tools to extract insights from the collected data and coordinate the work team with efficient collaboration tools.
Find out more about our UX Research services.
Crowdtesting is a testing methodology and an emerging trend that, in some cases, can be useful to analyse the software before it is released to the public (apps, websites, e-commerce, etc.), mainly to track bugs and quickly test the usability of the system.
Crowdtesting differs from traditional methods in that the analysis is performed remotely by a large number of unbiased testers with fresh eyes who come from different places, therefore not by consultants and professionals who are experts in user experience and testing methodologies, but users, understood as a representative sample of the panel reproduced in the laboratory.
The software or app is challenged through tests on multiple devices and browsers, with the aim of identifying errors or any difficulties of use, thus tracking down the largest and most obvious bugs.
The characteristic of this type of test is that it is engaged on web-based platforms and performed by non-expert users lacking specific skills regarding the analysis to be carried out.
The convenience and speed of crowdtesting are very attractive elements for companies, which are often behind schedule in their project plans. Using non-expert testers for the specific task can expose companies to the risk of receiving non-exhaustive, out-of-scope evaluations which then need to be evaluated by a team of experts and professional researchers. The same experts have the task, from a methodological point of view, of detecting cognitive biases that could lead to errors in evaluation: biases are mental shortcuts and misrepresentations of reality that are characteristic of each person and affect their decision-making processes.
Cognitive biases indicate a tendency to create one's own subjective reality, not necessarily corresponding to the evidence, developed based on the interpretation of the information available. They often lead us to make errors of judgement or to lack objectivity in our judgements; they are preferences and inclinations that lead us to make decisions quickly, instinctively, but not always correctly. They are also often referred to as "systematic" errors of the brain, which lead us to make incorrect conclusions about the world around us.
"Cognitive biases [..] often lead us to make errors of judgement or to lack objectivity in our judgements. They are preferences and inclinations that lead us to make decisions quickly, instinctively, but not always correctly".
In this sense, the ability of experts and professional researchers to recognise and manage cognitive biases makes a difference in the quality of the obtainable results.
Knowledge of the cognitive biases that guide users' minds is a very powerful tool, as it helps us understand how they perceive and process information about services or products.
Applying this knowledge to the development or analysis of a new digital product is essential for understanding the world around us and the people who interact with it.
In order to design processes and experiences, but also to analyse them, it is essential to know how evaluation errors, mental shortcuts and the lack of objectivity in people's judgements function in order to anticipate them and thus act in advance, recognising and transforming weaknesses into strengths and in this way optimising the overall user experience.
Expert researchers in the field of user experience are able to analyse complex systems in a structured and organic way, having a clear idea of the biases implemented by users and developing tests based on this knowledge, also by using crowdtesting.
Based on the foregoing, it is therefore clear that the use of methodologies that follow scientific processes can lead to:
As demonstrated by McKinsey, defining services with a high-quality user experience is strategic for companies, as the user experience can generate improved performance in terms of turnover – even double compared to those who do not invest in UX. This assertion is supported by the fact that market analyses reveal that only 22% of companies are satisfied with the conversion rates generated by their digital touchpoints, going so far as to define bugs in software as real conversion killers.
It therefore goes without saying that the quality development of digital products and services, as well as the meticulousness of testing processes – also through crowdtesting – become critical factors for the success of companies.
The data also show that 84% of users who try faulty apps with a complex browsing experience do not use them, which is why investment in optimisation, in terms of user experience, is becoming increasingly crucial.
The crowdtesting methodology has the considerable advantage of execution time and must, in order to be used effectively, be included in a broader methodological framework of analysis of the product/service being implemented through the careful study of teams of experts or specialised companies, whose core business is the improvement of the user experience, thus relying on an approach and expertise that only professionals in the sector can offer.
As CEO & co-founder of UNGUESS, I know how the use of this methodology presents a series of factors that must be managed with the proper tools:
Immediate and timely communication with the group of testers created: precisely because it is defined on a national and international scale and with a very large number of subjects, effective and efficient control and coordination tools are fundamental, such as web-based "remote collaboration" platforms, direct communication channels via a dedicated mobile app, etc., for both cultural and geographical reasons, and for the number of testers involved;
Constant control and monitoring: thorough and continuous supervision of the work of the selected group of testers is necessary, both through direct communication channels and by building a suitable organisational structure. Like "crowd managers", experts in the management of large groups of testers;
Ability to synthesise the results and reduce background noise: it may happen that the testers focus on a particular area of the site that has less impact than the task that determines the conversion and it is necessary, therefore, to create processes, methods and tools to extract insights from the data collected by focusing testers on the things that are truly important to analyse;
Managing the confidentiality and privacy of testers: the more testers there are, the greater the need to have tools that ensure that the testers examining the system do not reveal sensitive data (such as a multilevel NDA, private-access digital testing rooms, etc.). Furthermore, the testers' personal data must be managed in an aggregate manner that is also transparent.
Continuous tester engagement and proper training: companies must have the necessary tools to train testers on the processes in which they will be involved in order to guarantee the quality of the data collected and eliminate any biases. Furthermore, efficient, purely remote tools must be built in order to keep testers' engagement high during each test phase, such as recruiting processes on multiple channels, gamification elements, and continuous community support.
The very features of the methodology under analysis, the weighed advantages and disadvantages, suggest an in-depth approach in order to be able to give value to the engagement of heterogeneous groups of people. A dedicated internal team or a specialised company provide the appropriate tools to identify possible defects with respect to the most complex parts of the software, having a specific knowledge of what they are analysing.
Methods such as usability tests in the laboratory or task analysis are fundamental steps in testing a digital product: in the latter case, the efficiency and effectiveness with which users complete an assigned task are evaluated.
As Luigi Greco, CEO & co-founder of Conflux, tells us, task analysis is integrated with eye-tracking studies (the tracking of a user's ocular behaviour before a web page) and interviews in order to perform optimal UX research. Companies often spend millions on projects driven by a technological approach and not built based on their customers' real needs. This approach is fundamental to building successful digital products/services. Indeed, the biggest risk of digital companies is precisely that of making products that nobody wants!
If you are thinking about a new digital project, or you are not satisfied with your site or app, you can take advantage of the UNGUESS technology platform as well as UX research consultancy support such as that proposed by Conflux. This type of value proposition allows you to follow all the phases of the project, from analysis to testing, up to the creation of the UX requirements, to give rise to a successful product/service built around your users, with the Human-Centered Design (HDC) approach.
All the information is included in the free downloadable white paper below. Have fun reading!
Discover how AI empowers user research while real user feedback captures human context. Combine AI and crowdtesting for comprehensive understanding
Bending Spoons has enlisted the support of UNGUESS to maintain a very high level of quality by freeing up internal development team resources.
Atomic UX research is a new way to manage and use research information in areas such as usability testing and more.
Discover how user-centered Design helps users to interact efficiently with different digital platforms such as websites, apps, and e-commerce sites