Given recent advances in technology, such as Artificial Intelligence (AI) and AI-enabled chatbots, a personalized digital interface that simulates conversations across a variety of audiences can be carried out in all known languages. At KNCV Tuberculosis Foundation we have been at the forefront of introducing innovative digital health solutions within the scope of our mission. Earlier this year, looking at the possibility that AI technology bridges knowledge gaps on tuberculosis (TB), we conducted a pilot project in partnership with Dimagi:
Person-centered TB information for people from migrant populations: using (AI) to overcoming the language barriers and stigma communication on TB.
The objective: to explore the use and generate learnings on the use of AI-enabled chatbots to improve knowledge of TB among the general public in the Netherlands, with an emphasis on migrants with different language backgrounds.
Since its inception 120 years ago, KNCV has led efforts to increase awareness of TB in the Netherlands. Although nowadays TB has become an increasingly rare disease in the Netherlands, it continues to affect people who reside, but were not born, in the Netherlands. According to the National Institute of Public Health and the Environment (NIPHE/RIVM), around 80% of all people with TB in 2022 were born in countries outside the Netherlands coming from 78 different countries. These numbers reflect the need to focus our efforts on people from migrant populations, who speak different languages and have diverse cultural backgrounds.
With this in mind, we piloted a project in which we recruited volunteers from the general public that were interested to participate, TB professionals from the Municipal Health Services (TB nurses and TB medical assistants) and from KNCV’s network. To test the AI chatbot accuracy in different languages, we also recruited volunteers with languages other than Dutch and English. These volunteers were encouraged to participate by interacting with the chatbot either via web browser or WhatsApp, and completing a pre- and post-interaction surveys.
Training the chatbot
In order to “train” the chatbot on TB prevention and care in the Netherlands, we included several resource documents produced either by RIPHE/RIVM or KNCV. The following documents were included:
- Vaccination against TB: Information about BCG vaccination.
- After-care instructions following BCG vaccination: how to care for your child after the vaccination.
- TB infection: Information about the treatment.
- Skin test and blood test TB.
- TB transmission, coughing hygiene and rules of daily behaviour.
- TB source and contact tracing.
- Chest X-ray to test for TB: information about a chest X-ray to test for TB.
- TB screening for immigrants: information about TB screening for immigrants
- Travelling and TB: tips to avoid TB
- Frequently Asked Questions about TB (KNCV produced)
Evaluating the pilot
Although we had a relatively low number of participants, the overall language skills of the chatbot are considered good in Dutch and English in terms of spelling, grammar and accurate communication. It was identified that for less spoken languages the chatbot language skills were perceived insufficient by the participants. This resonates with the correlation between the language used and the overall accuracy of the answers. Additionally, we received positive feedback on the accuracy of general TB information and specific topics that were part of the source materials feeding the chatbot.
We consider the pilot successful as overall the chatbot was rated a 7.9 out of 10. Participants highlighted the ease of use in both the web browser and the WhatsApp version of this tool. Besides this insights, one of the most promising aspects of AI in this context is its ability to engage with individuals in culturally relevant ways. However, there are still several adjustments needed before this chatbot can be used in a programmatic setting.
An opportunity for improving knowledge of TB in a potentially cost-effective and scalable manner
TB remains one of the leading causes of death worldwide, disproportionately affecting populations in vulnerable situations, including migrants who may lack access to consistent healthcare or accurate information. As AI continues to evolve, its role in supporting our mission to ending TB is expected to expand. The lessons learned from this project are intended to reach beyond its use in the Netherlands and will be the model for the utilization in the Low and Middle Income Countries (LMICs) that KNCV is supporting around the globe.
You can access the following resources from this project here:
- AI chatbot pilot project full report
- AI chatbot pilot project report annex
- AI chatbot pilot project fact sheet