Build a Doctor Appointment Bot with Azure Bot Service, Language Understanding, and Twilio SMS
Time to read: 14 minutes
Telehealth services can improve access to healthcare for people living in remote areas. One common issue in remote areas is unreliable internet service. It’s a challenge for patients in remote areas to make appointments via web applications when internet service is poor or not available at all.
SMS is a reliable and low-cost alternative to reach people living in remote areas. In this article, you will learn how to build a doctor appointment booking system which allows users to book via SMS.
System Overview
The architecture of the SMS booking system is illustrated below.
Twilio is the communication channel between Microsoft Azure Bot Service and the end user. The core of the system is an Azure Bot built on the Microsoft Bot Framework. When the bot receives a message, it asks Language Understanding (LUIS) to analyze the message. LUIS responds with the intent of the message, which the bot uses to respond with a helpful answer.
Prerequisites
You'll need these technologies to follow along:
- An OS that supports .NET (Windows/macOS/Linux)
- .NET (Core) SDK version 3.1 or later
- PowerShell Core or Windows PowerShell (you can use any shell, but you'll need to make minor modifications)
- Azure subscription (get a free subscription)
- A free Twilio account
- Azure CLI 2.2.0 or later
- A .NET IDE (Recommended: Visual Studio 2019 or later, Visual Studio Code with the C# plugin, or JetBrains Rider).
You can find the completed source code in this GitHub repo.
Create the Language Understanding Service app
Azure Configuration
You need to create a resource group to store the Azure resources you will create later.
Open a PowerShell and sign in to Azure using the Azure CLI:
Run the following command to create a new resource group:
Replace [AZURE_LOCATION]
with the name of your preferred Azure location.
To interact with specific types of Azure resources, the resource provider for those resource types needs to be enabled. The most common resource providers are enabled by default, but you'll be using Language Understanding (LUIS) which is part of the Cognitive Services resource provider that is not enabled by default. You can register the Cognitive Services resource provider in Azure Portal, or using the Azure CLI as shown below.
First, check the status of the Cognitive Services resource provider using this command:
If the RegistrationState is UnRegistered, then run the following command to register it:
This command may take a minute to complete.
Create the LUIS app and train the model
Language Understanding (LUIS) is a cloud-based conversational AI service, part of Azure’sCognitive Services. It can process natural language text to predict overall meaning, and pull out relevant, detailed information.
To learn how to use LUIS, you need to know a few core concepts:
- Intent: The purpose or goal expressed in a user’s utterance.
- Entity: An item or an element that is relevant to the user’s intent
- Utterance: Unit of speech. Every sentence from a user is an utterance.
LUIS will act as the brain of the bot by helping the bot understand the incoming messages.
Create the LUIS App
Log in to the LUIS portal with your Azure account.
If you are a new user to LUIS, you will be prompted to Choose an authoring resource.
Click Create a new authoring resource if you haven’t created one before.
Another modal will appear to Create a new authoring resource.
Choose "rg-bot" in the Azure resource group dropdown, and enter “appointment-booking” in the Azure resource name field. Then pick your preferred region in the Location dropdown, choose "F0" in Pricing Tier, and click Done.
You will be redirected back to previous Choose an authoring resource modal. Now, you can click on the Done button, then you will be redirected to the dashboard page.
On the dashboard page, click Create new app. The Create new app modal will appear.
Enter “Appointment-booking” in the Name field and click Done.
After the app is created, a tutorial modal will be shown. Click outside the modal to go to the newly created app details page.
You'll need to collect some key information for later use. Click on the MANAGE tab and then on the Settings link. Take note of the App ID. Now, click on the Azure Resources link on the left menu and then click on the Authoring Resource tab.
Take note of the Primary key (LUIS API key) and the Endpoint URL.
Now, the appointment-booking LUIS app is created successfully. The next step is to create the heart of the app: the LUIS model.
Import the Model
There are two ways to create the model. You can navigate to the Build tab of the app, and then create entities and intents manually. Or you can import a predefined model file into the app. To save time, you can download this LUIS model JSON file and import it.
After downloading the JSON model file, navigate to the Versions page via MANAGE > Versions. Click on the Import button, and choose “Import as JSON” to open the import popup as shown below.
Click on the Choose file button, select your JSON file, and click on the Done button. The new model will be imported into the app.
Navigate to the Build > App Assets page and pick “vAppointmentBookingBot.LUISModel.json” from the versions dropdown which you can find in the breadcrumb navigation at the top-left of the page.
Now, you will see the newly created intents and entities.
Train, Test, and Publish the LUIS Model
After the intents and entities are imported, the Train button in the top navigation bar is enabled.
Click the Train button to start the training process. The Train button will be disabled and the Test button will be enabled after training is completed.
To test the new model, click on the Test button. A Test flyout panel will appear on the right. You can type an utterance into the test panel to try it out.
In the screenshot below, “i want to see doctor kathy” is given a score 0.973 out of 1 for the BookAppointment intent. In the Inspect window, it also identifies the Doctor entity as “kathy” correctly.
Since the test result looks pretty good, you can publish the LUIS app now. Click on the Publish button on the navigation bar, select the Production Slot, and click Done.
After publishing is completed, a successful message notification is shown as shown above. That means the LUIS app is ready to be used!
Build the bot
Create the bot using a Bot Framework template
In this tutorial, you’re going to use Bot Framework v4 SDK Templates to create the bot project. Open a shell and install the Bot Framework templates using the .NET CLI with these commands:
You'll only be using the CoreBot template in this tutorial, but feel free to explore the EchoBot and EmptyBot template.
Now, you can use the newly installed template to generate a new bot project. Run the following command to create the bot project:
After creating the project with the previous command, the project is created into the AppointmentBot/CoreBot folder and the root namespace is set to "CoreBot". This is inconsistent with how .NET templates usually work, but it can easily be rectified. The following PowerShell script will move the contents into the AppointmentBot folder, rename the project, and change all the namespaces to "AppointmentBot". Run the following script using PowerShell:
Open the project using your preferred .NET editor. The project structure will look like below.
The generated project comes with a flight booking bot sample. Remove those related model and dialog files as listed below.
- [projectRoot]\CognitiveModels\FlightBooking.cs
- [projectRoot]\CognitiveModels\FlightBooking.json
- [projectRoot]\CognitiveModels\FlightBookingEx.cs
- [projectRoot]\Dialogs\BookingDialog.cs
- [projectRoot]\Dialogs\MainDialog.cs
- [projectRoot]\BookingDetails.cs
- [projectRoot]\FlightBookingRecognizer.cs
To save time, you can run the script below to remove the above files. Run the script from the project root folder:
You will also need to remove lines 41 to 51 in the Startup.cs file. Those are the references to the deleted files.
After the cleanup, the Startup
class will look like below:
Appointment Booking Cognitive Model
Now that the project has been cleaned up, you can start implementing your own logic. Next, you'll be creating the model, which LUIS will return to us with analysis data.
Next, you'll create these files under the CognitiveModels folder:
- DoctorBooking.cs: This file will contain the
DoctorBooking
class, which represents the data returned by LUIS. - DoctorBookingEx.cs: This file will extend
DoctorBooking
using a partial class to simplify accessing the entities of the LUIS results
Create the CognitiveModels/Doctorbooking.cs and add the following code:
I generated this class using the Bot Framework CLI and provided it for your convenience, but you can also generate this yourself.
Create the CognitiveModels/DoctorBookingEx.cs file and add the following code:
Connect the bot to the LUIS App
To integrate the bot service with the LUIS app, you need to add the LUIS App ID, API key, and API Endpoint URL into the project configuration.
Replace the contents of appsettings.json with the JSON below:
Replace [YOUR_LUIS_APP_ID]
with your LUIS App ID, and [LUIS_ENDPOINT_URL]
with the LUIS Endpoint URL you took note of earlier.
Please note that you should not store sensitive information including API keys or tokens in your source-code. That's why you'll configure the LuisApiKey
using the Secret Manager tool.
Enable the Secret Manager tool for your project by running the following command at the project root directory:
Run the following command to configure the LuisApiKey
using the Secret Manager:
Replace [YOUR_LUIS_API_KEY]
with the LUIS App Primary Key you took note off earlier.
The bot application will retrieve the settings you just configured to establish the connection to your LUIS app in the AppointmentBookingRecognizer
class below. Create a new file AppointmentBookingRecognizer.cs and add the following contents:
A recognizer is used to recognize user input and return intents and entities within a DialogContext
. In the AppointmentBookingRecognizer
class, a connection is established to the LUIS API endpoint. It also implements the RecognizeAsync
method, which is called by dialogs to extract intents and entities from a user's utterance.
Control the Conversation Flow using Dialogs
You need to use Dialogs to manage conversation between the user and the bot.
Dialogs are a central concept in the SDK, providing ways to manage a long-running conversation with the user. A Dialog can be composed with other dialogs.
Bot framework provides a rich set of dialogs to make it easier to create a conversation flow. In this example, you will create an AppointmentBookingDialog
class to manage the main conversation. It is composed of a few dialogs, including a waterfall dialog and prompt dialogs.
The waterfall dialog is used to define the sequence of steps. As illustrated in the diagram below, the bot interacts with the user via a linear process.
Create a new file AppointmentDetails.cs into project root and add the following code:
The AppointmentDetails
class is the model class for the dialog. Next, create the AppointmentBookingDialog.cs file into the Dialogs folder. AppointmentBookingDialog
class will implement the process above. Add the following code to the file:
Main Dialog
The MainDialog
class manages the main process flow. Create the MainDialog.cs file in the Dialogs folder and add the following code:
This diagram gives you an overview of what the MainDialog
class does.
It's quite a lot of code, but the important part of the MainDialog
class is below:
In a nutshell, when a message activity is received, the bot runs the MainDialog
.
The MainDialog
prompts the user using the IntroStepAsync
method, then calls the ActStepAsync
method.
In the ActStepAsync
method, the bot calls the LUIS app to get the luisResult
object which will include the user's intent and entities. The user's intent and entities are used to determine the next step, either performing validation or invoking other dialogs.
At the end, the bot calls the FinalStepAsync
method to complete or cancel the process.
Twilio adapter and controller
By default, the Azure Bot service will connect to the web chat channel which is handled by the default AdapterWithErrorHandler
adapter. The default adapter is injected into the default BotController
class, and the controller exposes an endpoint /api/messages.
To connect the bot to Twilio, you will create a new TwilioAdapterWithErrorHandler
class extended from the TwilioAdapter class. Run the following command to install the Microsoft.Bot.Builder.Adapters.Twilio NuGet package:
After the NuGet installation is completed, create the TwilioAdapterWithErrorHandler.cs file in the project root directory, and add the following code:
To handle the HTTP webhook requests from Twilio, you'll need to add a TwilloController
. Create a new file TwilioController.cs in the Controllers folder, and add the following code:
The endpoint for TwilioController
is /api/twilio. After adding the new endpoint, the bot can handle messages via both web channel and Twilio SMS channel.
Finally, you need to register the dialogs and LUIS recognizer in the Startup
class. Insert the following lines at the end of the ConfigureServices
method in the Startup.cs file:
Test Locally
You'll need to install the Bot Framework Emulator to test the bot locally. To Install the Bot Framework Emulator:
- Navigate to GitHub releases page of the Bot Framework Emulator project
- Click on the setup file for your OS to download it.
- After the download is completed, click the file to start the installation. Follow the installation wizard, and use the default options to complete the installation.
Next, start the bot project using the .NET CLI:
Now, start the Bot Emulator, click on the Open Bot button, and enter the bot’s URL in your local environment, which by default, is http://localhost:3978/api/messages. Then click on the Connect button.
A chat window will be opened. You can type the message and start testing.
After you are happy with test results, the bot can be deployed to Azure.
Deploy the bot to Azure
To deploy the .NET bot to Azure, you need to use Azure CLI to create the following resources:
- A resource group: You have already created a
rg-bot
resource group earlier. - A managed identity: Although this resource won't be used for the Twilio bot, it is required by the Bot Framework deployment templates
- An App Service plan and an App Service
- A Bot Service
To create a new App Service plan, run the following command:
Here's what the parameters do:
- -g or --resource-group: The resource group the resource should be placed in, in this case into the "rg-bot" resource group you created earlier.
- -n or --name: The name of the App Service plan, which is asp-bot. "asp" is short for App Service plan.
- -l or --location: The Azure location the resource should reside in. Replace
[AZURE_LOCATION]
with the location closest to you or your users, like you did when creating the resource group earlier. - --sku: The size (CPU/RAM/etc.) of the App Service plan by SKU, which in this case is F1 (free).
To make sure the .NET project will be deployed correctly, you'll need to generate a deployment file. The deployment file can be generated with the command below:
Please note that --code-dir
and --proj-file-path
need to match together to resolve the path to the project file.
Create the managed identity using the following command:
After the command finished, a new "identity-appointment-bot" managed identity has been added in Azure, which will be used to create the new App Service and Bot Service in the next step.
The App Service and Bot Service can be generated using the existing App Service plan and the Azure Resource Manager (ARM) template which is part of the "CoreBot" .NET template.
You be using the DeploymentTemplates/template-with-preexisting-rg.json ARM template, but it requires a lot of parameters, which is why you should use a parameter file. Create a new ParameterFiles folder in the project root and create a new file RegisterAppParams.json with the following contents:
Some parameters have already been configured with the names of the previously created resources, but you still need to update a few with your own specific settings:
- appId: The value of
clientId
in the response of thecreate identity
command. You can also query theclientId
like this:az identity show -g rg-bot -n identity-appointment-bot --query clientId
- tenantId: The value of
tenantId
in the response of thecreate identity
command. You can also query thetenantId
like this:az account show --query tenantId
- appServicePlanLocation: The Azure region you used when creating your App Service plan.
- botId: The name for you Bot Service. This name has to be globally unique. Replace
[UNIQUE_SUFFIX]
with anything that would make the name unique, like "firstname-lastname1234". If it doesn't accept the name, change it up and try again. - newWebAppName: The name for your App Service. This name has to be globally unique because it will be used as a subdomain to azurewebsites.net. Replace
[UNIQUE_SUFFIX]
with anything that would make the name unique, like "firstname-lastname1234". If it doesn't accept the name, change it up and try again.
After the parameter file is updated, run the following command to generate the App Service and Bot Service:
Azure will take a minute to deploy this infrastructure. After the App Service is generated, run the following command below to get the App Service hostname:
Replace [UNIQUE_SUFFIX]
with the suffix you used in the parameters file. Take note of this hostname as you will need it later.
Now that the App Service infrastructure has been provisioned, you can deploy your local .NET bot project. Run the following command which will create a ZIP file and deploy the ZIP file to App Service:
It can take about 30 seconds for the deployment to complete. When it is done, you will see the success response like below:
Setting up Twilio for SMS communication
You've tested the bot locally using the web chat, but the goal of this tutorial is to use SMS to communicate. To receive and send SMS messages, you'll need a Twilio Phone Number.
- Go and buy a new phone number from Twilio. The cost of the phone number will be applied to your free promotional credit if you're using a trial account.
Make sure to take note of your new Twilio phone number. You'll need it later on! - If you are using a trial Twilio account, you can only send text messages to Verified Caller IDs. Verify your phone number or the phone number you want to SMS if it isn't on the list of Verified Caller IDs.
- Lastly, you'll need to find your Twilio Account SID and Auth Token. Navigate to your Twilio account page and take note of your Twilio Account SID and Auth Token located at the bottom left of the page.
When your Twilio Phone Number receives a text message, Twilio should forward it to your .NET bot hosted on Azure App Service. To configure that, navigate to Phone numbers > Manage > Active numbers, and click on your Twilio Phone Number to access the Configure page.
Under the Messaging section, set the dropdown under CONFIGURE WITH OTHER HANDLERS to “Webhook”, and in the adjacent text field, enter "https://", then paste in the App Service hostname you took note of earlier, and then enter "/api/twilio". The URL should look like https://your-hostname.azurewebsites.net/api/twilio.
Click the Save button on the bottom left. Take note of this webhook URL, you will need it again soon.
Lastly, you need to add some configuration to your App Service. Run the following command to configure the app settings:
Before running the command, replace the placeholders.
- Replace
[YOUR_LUIS_API_KEY]
with the LUIS Primary Key you took note of earlier. - Replace
[YOUR_TWILIO_PHONE_NUMBER]
with your Twilio Phone Number you bought earlier. Enter the phone number using the E.164 which looks like+11234567890
. - Replace
[YOUR_TWILIO_ACCOUNT_SID]
with your Twilio Account SID which you took note of earlier. - Replace
[YOUR_BOT_TWILIO_ENDPOINT]
with the webhook URL you took note of earlier. It should look like https://your-hostname.azurewebsites.net/api/twilio.
You can restart the App Service to make sure the app settings are loaded by the bot. Run the following command to restart the App Service:
End-to-End Test
Finally, you have built and assembled all the moving parts. Let’s test it!
Send a text message to your Twilio Phone Number and you should see a response from your bot. Here you can see I sent the below SMS messages to my Twilio Phone Number, and it works!
Next Steps
There are some important bot features that aren’t covered in this article. You may like to explore further when developing a production grade bot.
- Authentication: When a bot needs to access resources on behalf of a user, you must authenticate the user identity. The user authentication can be handled by Identity providers such as Azure AD with OAuth 2.0. Your bot will use the token generated by Azure to access those resources. The details on how to add Azure AD authentication to a bot can be found at Microsoft's documentation.
- Bot Framework Composer: It is an open-source visual designer and authoring tool to create a bot with Azure Bot Service. You can use it to build dialogs in the UI and visualize the flow to business users. It also allows you to train LUIS models within the tool, thus saving the need to switch between different environments. If your project requires involving non-technical people to bot development, it is definitely a good tool to consider.
Conclusion
In this article, you walked through the steps to build an SMS booking system using Twilio, Azure Bot Framework, and LUIS. You could extend this by adding more channels, expanding the LUIS model to support real-life scenarios, and incorporating other features like image recognition or multiple language support.
Yan Sun is a full stack developer. He loves coding, always learning, writing and sharing. Yan can be reached at sunny [at] gmail.com.
Related Posts
Related Resources
Twilio Docs
From APIs to SDKs to sample apps
API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
Resource Center
The latest ebooks, industry reports, and webinars
Learn from customer engagement experts to improve your own communication.
Ahoy
Twilio's developer community hub
Best practices, code samples, and inspiration to build communications and digital engagement experiences.