--- title: "How to test a chatbot before the release: methods and tools" description: What you should keep in mind before releasing a chatbot and how to test it and train it before it gets into your real users' hands. image: https://blog.unguess.io/hubfs/Chatbot2%201.png --- [![Vai alla homepage di UNGUESS](https://blog.unguess.io/hubfs/Imported%20images/unguess-logo.svg)](https://unguess.io/it) - PRODUCTS BY INDUSTRY - [Retail & Ecommerce](https://unguess.io/services/industry/retail-ecommerce/) - [Fast-Moving Consumer Goods](https://unguess.io/services/industry/fast-moving-consumer-goods/) - [Banking, Insurance & Financial Services](https://unguess.io/services/industry/banking-insurance-financial-services/) - [Travel & Hospitality](https://unguess.io/services/industry/travel-hospitality/) - [Utilities](https://unguess.io/services/industry/utilities/) - [Healthcare](https://unguess.io/services/industry/healthcare/) - [Media & Entertainment](https://unguess.io/services/industry/media-entertainment/) - [Automotive](https://unguess.io/services/industry/automotive/) - PRODUCTS BY USE CASE - [Ai Training](https://unguess.io/services/ai-training-testing/) - [User Experience](https://unguess.io/services/use-case/user-experience/) - [Software Quality](https://unguess.io/services/use-case/software-quality/) - [Cyber Security](https://unguess.io/services/use-case/cyber-security/) - [Accessibility](https://unguess.io/services/accessibility/) - [HOW IT WORKS](https://unguess.io/how-it-works/) - [OUR CROWD](https://unguess.io/our-crowd/) - COMPANY - [About us](https://unguess.io/about-us) - [Life at UNGUESS](https://unguess.io/life-at-unguess/) - [Partners](https://unguess.io/unguess-partners/) - [SHOWCASES](https://unguess.io/showcases/) - [Case Study](https://unguess.io/showcases/all-showcases/?jsf=jet-engine:shocase-listing-grid&tax=resource-category:76/#readmore) - [Unguess Challenges](https://unguess.io/showcases/all-showcases/?jsf=jet-engine:shocase-listing-grid&tax=resource-category:74/#readmore) - [White Paper](https://unguess.io/showcases/all-showcases/?jsf=jet-engine:shocase-listing-grid&tax=resource-category:75/#readmore) - [BLOG](https://blog.unguess.io) - - - - - [PRODUCTS](https://unguess.io/services/) - BY INDUSTRY - [Retail & Ecommerce](https://unguess.io/services/industry/retail-ecommerce/) - [Fast-Moving Consumer Goods](https://unguess.io/services/industry/fast-moving-consumer-goods/) - [Banking, Insurance & Financial Services](https://unguess.io/services/industry/banking-insurance-financial-services/) - [Travel & Hospitality](https://unguess.io/services/industry/travel-hospitality/) - [Utilities](https://unguess.io/services/industry/utilities/) - [Healthcare](https://unguess.io/services/industry/healthcare/) - [Media & Entertainment](https://unguess.io/services/industry/media-entertainment/) - [Automotive](https://unguess.io/services/industry/automotive/) - BY USE CASE - [Ai Training](https://unguess.io/services/ai-training-testing/) - [User Experience](https://unguess.io/services/use-case/user-experience/) - [Software Quality](https://unguess.io/services/use-case/software-quality/) - [Cyber Security](https://unguess.io/services/use-case/cyber-security/) - [Accessibility](https://unguess.io/services/accessibility/) - [VIEW ALL PRODUCTS](https://unguess.io/services/) - [HOW IT WORKS](https://unguess.io/how-it-works/) - [OUR CROWD](https://unguess.io/our-crowd/) - COMPANY - [About us](https://unguess.io/about-us/) - [Life at UNGUESS](https://unguess.io/life-at-unguess/) - [Partners](https://unguess.io/partners/) - [SHOWCASES](https://unguess.io/showcases/) - [Case Study](https://unguess.io/showcases/case-study/) - [Unguess Challenges](https://unguess.io/showcases/unguess-challenges/) - [White Paper](https://unguess.io/showcases/white-paper/) - [Webinar](https://unguess.io/showcases/webinar/) - [BLOG](https://blog.unguess.io) - - - - [CONTACT US](https://unguess.io/it/inizia-ora/) [SIGN UP](https://app.unguess.io/) Chatbot # How to test a chatbot before the release: methods and tools What you should keep in mind before releasing a chatbot and how to test it and train it before it gets into your real users' hands. [Angela Meduri](https://blog.unguess.io/author/angela-meduri) Feb 28, 2021 --- `"2/3 of leading global financial service firms have implemented the use of chatbots for their apps since the onset of the COVID-19 pandemic."` ([Forrester](https://go.forrester.com/blogs/covid-19-stokes-the-chatbot-hype-in-financial-services/), 2020)   Chatbots are changing the customer service we have been used to. On the company's side, chatbots can save up to 80% of routine questions, helping businesses save on customer service costs ([IBM](https://www.ibm.com/blogs/watson/2017/10/how-chatbots-reduce-customer-service-costs-by-30-percent/), 2017). The chatbot market is also expected to grow to a whopping $102.29 billion by 2026 ([Research and Markets](https://www.researchandmarkets.com/reports/4622740/chatbot-market-growth-trends-covid-19-impact), 2021). What's important is that the chatbot arrives in your users/clients’ hands already tested and trained. In this way, it won't cause damages to the company's reputation and add even more workload on "human" customer service, as well as causing negative reviews of your service. But how do you test a chatbot? Let’s see the main strategies and the tools that allow you to make a good impression with your customers/users. ## Test the chatbot: what to keep in mind **before** the release Let’s start with some aspects to keep into consideration before releasing a chatbot: - **User’s intent** First, it is essential to be able to distinguish between intent and entity. **The intent is the user’s goal**, what he has in mind while writing his question in the chatbot, what he really wants to find. **The entity** is the attribute, which is **how the user describes his problem.** They are what the user is looking for and how he is looking for it. Understanding this difference allows you to correctly predict the chatbot training. ![Tangowork-Explanations-for-Graphic-Export.001](https://blog.unguess.io/hs-fs/hubfs/Imported_Blog_Media/Tangowork-Explanations-for-Graphic-Export_001-2.png?width=1500&name=Tangowork-Explanations-for-Graphic-Export_001-2.png) - - **The conversation flow** The chatbot must be able not only to have a memory of the conversation and the concatenation of questions, but it must as well be flexible enough to be able to jump from one question to another without a connection. In addition, it is important that the chatbot knows how to end a conversation. According to a study by Jain, Kumar, et al (2018), users expect an interaction similar to the one they would have with a human, including an introduction and a conclusion. A Chatbot, however, seems to be less prepared to end a conversation. Chaves and Gerosa (2018) pointed out that the chatbot needs to know when the conversation ends, for example with explicit statements from the user (*thanks,* or *bye*). - **Error management ** It is unavoidable, sooner or later the bot won't know how to answer. For this reason, it is crucial to have a strategy to cover those cases.  ## How to test a chatbot before the release There are three methods to test the chatbot before the release: **general test, product/service specific test and limits test.** Completing these tests before the release would be idea because these tests show the status of the key points and allow you to understand which are the underlying issues. These tests should not stop after the release: **planning new tests (even automated)** will be equally important to ensure that the new versions do not affect the pre-existing features.   ## General test Doing a general test allows you to **understand the progress status and the correct functioning of the chatbot**. If the chatbot does not respond correctly to the general test, it is useless to continue with further tests before fixing the existing problems. In this phase, **general questions are asked, for example greetings** (so not even a question but a previous phase), and test if the chatbot welcomes the user. In fact, the role of a good bot is not only to answer the questions, but also to maintain a pleasant conversation flow. **If the chatbot fails this step, the user will probably stop trying** (if it gets released) or, in the worst case, leave the website. If your chatbot has already been released without running a general test, try checking the analytic metrics and see if users have not left after the first two exchanges.   ## Product/service specific test After the greetings, you get into the conversation. The second step already goes into the product/service specifics on which the chatbot responds. While in the general test generic questions are asked or even only greetings, here we move into the specific product/service vocabulary. If your product happens to be a bank's website the questions will surely include transfers, balance and bank statements, for example “how long does an international transfer takes?” or “I need a bank statement”.   ## Test of the chatbot limits If the first two tests go well, we can say that your chatbot answers correctly to the most common and significant expressions. Now we must test **what happens when the user writes something irrelevant to the product** or service to see if the chatbot can handle the conversations or it fails.   ## How well do these tests work? Your team could face two problems while planning or running these tests: 1. **Developer confirmations bias**, also called confirmation bias. “Confirmation bias is the tendency to search for, interpret, favor, and recall information in a way that confirms or supports one's prior beliefs or values. People display this bias when they select information that supports their views, ignoring contrary information, or when they interpret ambiguous evidence as supporting their existing attitudes" (source: [Wikipedia definition of Confirmation bias](https://en.wikipedia.org/wiki/Confirmation_bias)). 1. **The lack of time to address these activities**. We are all overwhelmed by the amount of tasks we need to carry out, and for sure the developer is not an exception. The chatbot testing activity is long and there isn’t always enough time to run it properly. However, this can cause the release of a bot that is not ready for it final users.   The solution for the chatbot pre-release test If the internal team is in charge of the process,[testing the chatbot](https://unguess.io/services/chatbot-voice-training-testing/) can take a very long time. The team would need to add the testing tasks to developing and fixing the product. And that's not the only issue. When the developer and the tester are the same person, bias tend to become a problem. Also, `developers` don't always match the real user (especially if the real user lives in another country or belongs to a less tech-savvy generation). **Luckily, there is an extremely useful tool that help developers overcome these problems: crowdtesting.** Once your marketing team has defined your target/ buyer persona and has given us an overview on the chatbot goal, we'll do the rest. UNGUESS's Quality Leader and Customer Success Manager will be in charge of selecting the testers who correspond to your real users and proceed with the [**chatbot testing and training**](https://unguess.io/services/chatbot-voice-training-testing/). Within a couple of days, you will have your chatbot tested by your own users before its market release.   [Chatbot](https://blog.unguess.io/tag/chatbot) [Share via Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fblog.unguess.io%2Fen%2Fhow-to-test-chatbot-before-release) [Share via Twitter](https://twitter.com/intent/tweet?url=https%3A%2F%2Fblog.unguess.io%2Fen%2Fhow-to-test-chatbot-before-release&text=How+to+test+a+chatbot+before+the+release%3A+methods+and+tools) [Share via Email](mailto:?subject=How+to+test+a+chatbot+before+the+release%3A+methods+and+tools&body=https%3A%2F%2Fblog.unguess.io%2Fen%2Fhow-to-test-chatbot-before-release) [Share via LinkedIn](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Fblog.unguess.io%2Fen%2Fhow-to-test-chatbot-before-release&title=How+to+test+a+chatbot+before+the+release%3A+methods+and+tools&summary=What+you+should+keep+in+mind+before+releasing+a+chatbot+and+how+to+test+it+and+train+it+before+it+gets+into+your+real+users%27+hands.) ## Similar posts UX & UI ### [The tests your e-commerce needs before Black Friday](https://blog.unguess.io/test-ecommerce-before-blackfriday?hsLang=en) How to keep your customers from abandoning their shopping carts? Here are the key tests to take before Black Friday and Christmas. Angela Meduri Oct 3, 2022 UX & UI ### [The test your e-commerce needs before Black Friday](https://blog.unguess.io/the-test-your-e-commerce-needs-before-black-friday?hsLang=en) How to prevent your customers from abandoning the shopping cart? Here are the key tests to carry out before Black Friday and Christmas. Angela Meduri Oct 30, 2024 UX & UI ### [eCommerce UX: The checklist to understand what to improve](https://blog.unguess.io/ecommerce-ux-the-checklist-to-understand-what-to-improve?hsLang=en) Retailers are aware of the centrality of eCommerce UX, but they often don't know where or how to improve it. Here are the answers. Newsroom Jun 6, 2024 Chatbot ### [Coronavirus: UNGUESS' Contribution to Companies and Testers](https://blog.unguess.io/en/coronavirus-appqualitys-contribution-to-companies-and-testers?hsLang=en) We've thought a lot about what can we do to help during this really hard time. That's why we launched a campaign to support both companies and testers Angela Meduri Mar 10, 2020 ![unguess-logo-1](https://blog.unguess.io/hubfs/unguess-logo-1.svg) **© 2023 UNGUESS S.r.l.** UNGUESS is a registered trademark of UNGUESS S.r.l.   - [![U2Y - Verified Carbon Footprint](https://blog.unguess.io/hs-fs/hubfs/U2Y%20-%20Verified%20Carbon%20Footprint.png?width=50&height=50&name=U2Y%20-%20Verified%20Carbon%20Footprint.png)](https://app.u2y.io/brands/314) - [![UNGUESS is a leader in Crowd Testing Tools on G2](https://images.g2crowd.com/uploads/report_medal/image/1004327/medal.svg)](https://www.g2.com/products/unguess/reviews?utm_source=rewards-badge) - [![UNGUESS is a leader in Test Management on G2](https://images.g2crowd.com/uploads/report_medal/image/1004327/medal.svg)](https://www.g2.com/products/unguess/reviews?utm_source=rewards-badge) - [![UNGUESS is a leader in Crowd Testing Tools on G2](https://images.g2crowd.com/uploads/report_medal/image/1004379/medal.svg)](https://www.g2.com/products/unguess/reviews?utm_source=rewards-badge) - [![UNGUESS is a leader in Europe Test Management on G2](https://images.g2crowd.com/uploads/report_medal/image/1004391/medal.svg)](https://www.g2.com/products/unguess/reviews?utm_source=rewards-badge) - [![UNGUESS is a leader in EMEA Test Management on G2](https://images.g2crowd.com/uploads/report_medal/image/1004447/medal.svg)](https://www.g2.com/products/unguess/reviews?utm_source=rewards-badge) #### SERVICES - [AI Training](https://unguess.io/services/ai-training-testing/) - [User Experience](https://unguess.io/services/use-case/user-experience/) - [Software Quality](https://unguess.io/services/use-case/software-quality/) - [Cyber Security](https://unguess.io/services/use-case/cyber-security/) - [Accessibility](https://unguess.io/services/accessibility/) #### SHOWCASE - [Blog](https://blog.unguess.io) - [Our Crowd](https://unguess.io/our-crowd/) - [Tryber.me](https://tryber.me/) - [Integrations](https://unguess.io/integrations/) - [Partners](https://unguess.io/partners/) #### COMPANY [Work with us](https://unguess.io/life-at-unguess/#job-positions) [Get in touch](https://unguess.io/get-started/) **General inquires:** [info@unguess.io](mailto:info@unguess.io)   [Privacy Policy](https://unguess.io/privacy-policy/)  | [ESG Policy](https://unguess.io/esg-policy/)  | [Personal Data Processing Notice: Customers and Suppliers](https://unguess.io/personal-data-processing-notice-customers-and-suppliers/) | [Model 231](https://unguess.io/model-231/) | [Ethical Code](https://unguess.io/ethical-code/) | [Cookies Settings](https://www.iubenda.com/privacy-policy/833252/full-legal)  |  [Terms & Conditions](https://unguess.io/terms-and-conditions/) UNGUESS S.r.l. – VAT 01603290196   © 2022 Kalungi, Inc. - All Rights Reserved. [Powered by Atlas - a B2B SaaS HubSpot theme](https://www.kalungi.com/atlas-hubspot-theme-for-b2b-saas-software)