Extending Azure Health Bot with Azure OpenAI Service

[Guest Bert Hoorne]

Extending Azure Health Bot with Azure OpenAI Service

 

 

 

 

Over the past few months, we have witnessed a tremendous shift in the way Artificial Intelligence (AI) is used across every industry. The adoption of Large Language Models (LLM) is gaining traction at an immense speed.

 

 

 

Azure Health Bot empowers healthcare organizations to build and deploy AI-powered, compliant, conversational healthcare experiences at scale. Already today, healthcare organizations are using Azure Health Bot to provide end users with self-serve functionality and easy to access information. The service combines built-in medical intelligence with natural language understanding tuned for clinical terminology in addition to easy customization and extension capabilities. But as with every chatbot, there are limitations to its ability to understand more complex questions or questions beyond its scope.

 

 

 

We are excited to share the preview of a new Azure Health Bot template that allows our customers to experiment with the integration of Azure OpenAI Service into their Health Bot instances for fallback answers. This feature does not aim to facilitate the bot to answer unknown queries in the medical space, rather, it enables organizations to access the Azure OpenAI Service API and decide how to use the model to improve their bot built through the Azure Health Bot service.

 

 

 

At this time, we are offering the model in private preview for internal testing and evaluation purposes only.

 

 

Building your Health Bot instance in a responsible way

 

 

 

 

We evaluated how to leverage Large Language Models (LLM) to empower our customers to achieve morewhile at the same time, making sure this was done in a responsible way.

 

 

 

Microsoft is committed to developing AI responsibly and in a way that benefits society as a whole. As a company we follow responsible AI principles for the development and use of AI

 

 

 

With this first step of empowering our customers to integrate Azure OpenAI Service into their Azure Health Bot instance, we are creating a responsible way of using LLM models. Azure Health Bot is already capable of providing patient triage and healthcare related information from clinically validated sources, but in some scenarios the bot does not understand what the end user is trying to ask. In such cases of unrecognized utterances, where the bot does not know what to do, we created a side-by-side integration of Azure OpenAI Service in the Azure Health Bot to help provide fallback answers.

 

 

 

To use the Azure OpenAI Service integration feature in a responsible way, we recommend the following approaches:

 

• Always use the pre-built intents for triage and medical content answers and not disable them, unless you are replacing them with your own intents and flows
  
• By default, every answer from Azure OpenAI Service will be marked as an AI generated answer, we recommend that you always inform end users that these answers are coming from an AI tool and should be verified before relying on the information.
  
• Create your own intents, scenarios, and flows for use cases where you want to be in full control of the responses, rather than relying on Azure OpenAI Service.
  
• Enable continuous clinical assessment by subject matter experts to evaluate the answers. This will give you the insights and the opportunity to improve the responses.
  
• Ground your AI model from trusted documents and implement a mechanism to have the bot draw responses from these documents, when answering end-user questions. It is the customer responsibility that it only draws from the trusted documents and does not make up answers.
  
• Monitor and analyze your unrecognized utterances. This will provide you with insights on what your users are asking and give you the information to create relevant scenarios and a more predictable flow. It will also help to mitigate the risk for known limitations of these models, such as potential hallucinations. It is your responsibility to measure and mitigate potential hallucinations to ensure that the bot does not respond outside of the provided trusted documents. Monitoring will also help mitigate harm arising from end users attempting to bypass the safety system (e.g., breaking the metaprompt).
  
• Act upon user feedback from the Azure OpenAI Service responses. The current implementation contains a simple feedback mechanism with a Thumbs-up and Thumbs-down icon. This feedback information can be stored in your backend or logged into the service. As a customer you should analyze this information to validate if users find the responses valuable or if you need to tweak the responses for better results
  

 

 

 

Importing the Azure OpenAI Service Template into your Azure Health Bot instance.

 

 

 

 

We are enabling Azure OpenAI Service through the Azure Health Bot Scenario Template Catalog. The catalog allows customers to import validated healthcare scenarios into their Azure Health Bot instance. You can think of these templates as starter kits to decrease time to market.

 

 

 

 

 

 

The Azure OpenAI Service template allows customers to connect Azure Health Bot with their own Azure OpenAI endpoint. This is done through a secure channel and makes sure all requests and data is kept within the tenant of the customer. Customers who want to access Azure OpenAI Service can request this here.

 

 

 

After a successful import of the Azure OpenAI Service template into see the Azure Health Bot instance, the customers get a new scenario that is ready to use and automatically configured to catch unrecognized utterances through Azure OpenAI Service.

 

 

 

 

 

 

To provide a tailored experience for the end users, we are using the built-in user context capability from the Azure Health Bot. We use this feature to send context aware information to Azure OpenAI Service which increases the quality of AI generated responses. The user can, at any time, ask the bot to be forgotten. All information of the end-user will then be deleted.

 

 

 

The Azure Health Bot is also able to answer different healthcare related questions. With the help of conversational language understanding, the service is able to catch medical complaints, medication and information requests, condition information, symptoms , etc. We recommend keeping this feature enabled when using the Azure OpenAI integration.

 

 

 

 

 

 

In the event of an unrecognized utterance, Azure Health Bot will now forward these to Azure OpenAI Service. To provide the most accurate and relevant answer, the Azure Health Bot scenario already implements a small but effective prompt engineering statement:

 

• “you can only answer on medical related content. You need to use information from CDC Works 24/7 or U.S. Food and Drug Administration. don't provide an url but a name of where you looked for the content” .
  

 

This prompt tells the model to look for answers only from these validated sources and include the sources in the responses. For production use cases we recommend that the customer implement the bot such that it only draws from the trusted documents thereby reducing hallucinations as much as possible.

 

 

 

 

 

 

 

 

The Azure OpenAI Service catalogue template will be available in the coming months in preview. We can’t wait to see how customers will experiment with and use this feature.

 

 

 

Learn More:

 

Azure Health Bot

 

Azure Health Bot documentation

 

Continue reading...