D
denverbrittain
Intelligent apps with Azure OpenAI are easier than ever to configure and deploy, especially when using a PaaS platform like Azure App Service for hosting.
While this demo is a full tutorial for deploying an App Service Web App, Database, Managed Identity, and Azure OpenAI, most of the code and instructions for each section can be repurposed for individually unique use-cases. If you already know how to perform some of these actions (there's ample documentation on most of this in Microsoft Learn Documentation), feel free to skip ahead and focus on the sections most useful to your needs. Each section builds on the previous, but the code and general principles can be applied to a wide range of applications.
The application source code for this article is available on GitHub and was originally forked from the Azure Samples version of the Spring PetClinic application. This version of petclinic is largely the same, but with the addition of an Intelligent ChatBot powered by Azure OpenAI. All resources in this demo are securely connected using Azure Managed Identity for secure passwordless connections.
The Portal
Creating Azure Resources
Starting with the portal actions, while these can all be performed over CLI or Bicep, the portal is the easiest way to show the actions required to set up your application.
App Service
We'll start by creating our App Service Plan and Web App in the portal. In the Azure Portal, select App Service and choose to create a Web App.
On the App Service Create page, we need to choose Java Version 11 and Java SE (Embedded Web Server). The choice of Java SE allows us to run any JVM based language (ex: Kotlin, Scala, etc.) with an embedded web server on the App Service Platform, in this case, a SpringBoot app written in standard Java. You'll also need to name your Web App and choose an App Service Plan, I've chosen P0V3, but B1 should also work for this app.
Next, we'll need to deploy a database.
MySQL Database
In the Azure Portal, select Azure Database for MySQL flexible servers, and choose the quick create option for the MySQL option (no WordPress needed. On the create page for the Flexible server, ensure that you have a standard (B1) compute SKU with at least 20GiB storage, Dev/Test workload is sufficient for a sample application like this one. After choosing a name, region, admin username and password, create your MySQL Server.
Now we have a database, on to Azure OpenAI!
Azure OpenAI
In the Azure Portal, choose Azure OpenAI service and click create. After choosing your options for naming, resource group, etc. (this demo uses an S0 sku), create your OpenAI service. In the Azure OpenAI Studio, be sure to select and deploy the gpt-35-turbo model, this is what we will use for chat completions in our SpringBoot app.
Managed Identities
To connect all our resources together, we will use Managed Identity. This ensures our connections are handled by Microsoft Entra in a secure, passwordless setup. There are no secrets to manage here. With our Managed Identities properly configured, your Azure resources will automatically have the permissions to connect to each other securely without any need for secrets management.
Starting with our App Service Web App, let's first enable a system assigned identity for this service under Settings > Identity.
After enabling our system assigned identity for our Web App, we'll need to create some user assigned managed identity resources. One for our MySQL database and one for our Azure OpenAI resource. We'll also use the Azure OpenAI identity for GitHub Actions later, but you can always create a Managed Identity resource per resource access if desired, or even use a single user managed identity for all resource access associated with this application. The passwordless end-result is the same either way.
Starting with our identity for Azure OpenAI, go to Managed Identities in the portal and click create. After naming your identity, go to Azure role assignments and click Add role assignment. Here, I've enabled this identity to be a few things, a Cognitive Services OpenAI user on my Azure OpenAI resource, a Cognitive Services Contributor on my resource group for this app, and a Website Contributor to the Staging slot of my App Service Web App.
Next, let's create our identity for the MySQL database. In this case, I left this one a bit open-ended. Following the same procedure as the last Managed Identity, but in this case making our MySQL related identity a Contributor to our resource-group as a whole. This allows us to effectively have one Managed Identity that can "contribute" to anything in the selected resource group. In this case, we're only interested in the MySQL server.
Securely Connecting Our Resources
Now that we've created all our managed identities, let's add them to our Web App. After browsing to our App Service Web App in the portal, select Settings > Identity > User assigned. Here we'll add the two identities we just created.
We're almost done, but for this next connection, we need to use the Azure CLI and Service Connector. In your terminal, run the following command to securely connect your MySQL database to your Web App using Service Connector.
Be sure to store the username produced here, this will be important later. Now all our resources are securely connected! On to the source code to put all these resources to good use.
The Source Code
Dependencies
Our first stop is the pom.xml file. Be sure to include the following dependencies in your pom.xml file. Remember, the full source code for this application is available on GitHub.
After adding the required dependencies, let's look at the Chatbot source code itself.
Application Source Code for Intelligent Chatbot using SpringBoot & Azure OpenAI
Starting off with directory structure, the Chatbot in its entirety lives here:
This directory contains 3 files, ChatController.java, WebSocketConfig.java, and WebSocketEventListener.java. Let's examine the ChatController individually and take a quick look at our WebSocket related files.
ChatController.java
This is some boilerplate code for a pretty standard Chatbot using Web Sockets running in a SpringBoot app. The goal of this basic chat controller is that it can be reused, modified, or otherwise repurposed for a wide range of uses.
While I won't go line-by-line, I'll highlight a few major parts of the code. The first being the dependencies associated with Azure OpenAI and Managed Identity, these are all essential.
Next let's look at the ChatController class. The sendMessageAI method contains the majority of our Chatbot configuration. The following lines contain the Managed Identity connection to Azure OpenAI.
The remaining content of the method handles the AI portion of the chatbot, in this case, instructing our Azure OpenAI model on what role to play during our chat and handling the receiving and answering of user provided chat messages. All of this is done over the Web Socket spec defined in WebSocketConfig.java and WebSocketEventListener.java.
The remaining methods in this file define the control flow for messages sent by the user (sendMessage), and users added to the chat (addUser).
The Azure OpenAI connection and usage scenario may seem simple to set up (and it is), but the results are incredibly powerful and the possibilities for integrations with business apps are limitless. Now that we've seen the OpenAI setup, let's look at our database connection.
Database Connection with Managed Identity
Back when we made our managed identities and used Service Connector to connect our database and web app, you should have received a MySQL username that looks something like: aad_mysqlflexible_abcd. Now we'll take this username and add it to our SpringBoot configuration file, in this case, the one for MySQL called application-mysql.properties. Edit the file to include your database URL and MYSQL username like the example below.
Setting spring.sql.init.mode=always lets us execute the data.sql file in our resources/ directory automatically on startup.
In just a few steps, we've connected a MySQL database to our App Service web app using managed identity and service connector for secure passwordless database connections. Next up, we'll look at some of our deployment options.
Deployments
Deployments - CLI
Let's look at deploying our app, starting with the Azure CLI and Maven. The first step with any Maven-based build and release for a Java application is to build our app. We can start by running a Maven build using the following command:
I'm skipping tests for the purpose of demonstration, but you may want to avoid this with a production app.
Next up, deploy your application JAR file with the Azure CLI like this:
Checking your deployed application, you should see something like this:
Now browsing to the chat tab should give you the option to chat with an AI assistant powered by Azure OpenAI!
If that seems easy, it's because it is. We can add our own data, modify the behavior of our chatbot, and even AI framework we're using for the app, but we'll save that for a later post.
Deployments - GitHub Actions with Managed Identity
CLI deployments are a great way to get started, but CI/CD is typically the preferred deployment method for web apps. Using GitHub Actions with Azure App Service, CI/CD is straightforward and easy to configure. In this case, having our Managed Identities already configured will make it even easier.
Starting off, we'll need to browse to our web app in the Azure portal and select Deployment > Deployment Center. Here we can configure our app to deploy from a number of different sources, but we'll select GitHub for now. Picking our Organization, Repository, and Branch, we're then allowed to create a new workflow file or add a new one. Let's just add a new one, which we can preview later. This workflow just performs a standard Maven build and deployment to our app on merges into main.
We'll address the notice at the top of the page in the next section, but for now let's look at the authentication settings. Here we can use an existing user-assigned identity (Managed Identity), or we can create a new one. If you're on a Free, Basic, or Standard SKU, select your preferred identity (or create a new one) and click save. Otherwise, click Discard and continue to the next section to set up a staging slot for your web app.
Using Deployment Slots (Premium SKU Required)
To set up deployment slots for your app, you must be on a Premium SKU. Browse to your web app > Deployment > Deployment Slots and click Add.
Here you can name your staging slot and clone the settings of your production slot. After adding the slot, click on it to switch over to your new staging slot. Now on the staging slot, we'll configure GitHub Actions deployments just as we did before. Under Deployment > Deployment Center we'll fill out the form naming GitHub as our source, specifying Organization, Repository, Branch, selecting to add a new workflow and configuring our user-assigned identity for auth. After clicking save, a deployment will be kicked off and the app will be deployed to your staging slot.
Slot Swap into Production
While the staging slot is great for ensuring our app is fully functional before we deploy to production, eventually, we'll need to officially swap our slots and deploy our production app. To do this, go back to Deployment > Deployment slots and select Swap. Here you'll need to set your source and the staging slot and target as the production slot. After clicking start swap and waiting for the changes to complete, your application from staging will be fully deployed and running in production.
Now any time a change is made, the staging slot will be updated first and, after a manual review, can be swapped into production at any time. This also works in reverse, if a production app is having issues, swapping the previous version from the staging slot can effectively revert a change that had unintended consequences.
Conclusion
Configuring and deploying secure OpenAI powered applications to Azure App Service has never been easier! With the flexibility of the PaaS platform, the passwordless connections, and the power of Azure OpenAI, the concepts demonstrated here can be applied to a wide range of applications. Reach out with any questions or comments, this article will contain links in the future to other versions of the same app using LangChain4j and SpringAI.
Continue reading...
While this demo is a full tutorial for deploying an App Service Web App, Database, Managed Identity, and Azure OpenAI, most of the code and instructions for each section can be repurposed for individually unique use-cases. If you already know how to perform some of these actions (there's ample documentation on most of this in Microsoft Learn Documentation), feel free to skip ahead and focus on the sections most useful to your needs. Each section builds on the previous, but the code and general principles can be applied to a wide range of applications.
The application source code for this article is available on GitHub and was originally forked from the Azure Samples version of the Spring PetClinic application. This version of petclinic is largely the same, but with the addition of an Intelligent ChatBot powered by Azure OpenAI. All resources in this demo are securely connected using Azure Managed Identity for secure passwordless connections.
The Portal
Creating Azure Resources
Starting with the portal actions, while these can all be performed over CLI or Bicep, the portal is the easiest way to show the actions required to set up your application.
App Service
We'll start by creating our App Service Plan and Web App in the portal. In the Azure Portal, select App Service and choose to create a Web App.
On the App Service Create page, we need to choose Java Version 11 and Java SE (Embedded Web Server). The choice of Java SE allows us to run any JVM based language (ex: Kotlin, Scala, etc.) with an embedded web server on the App Service Platform, in this case, a SpringBoot app written in standard Java. You'll also need to name your Web App and choose an App Service Plan, I've chosen P0V3, but B1 should also work for this app.
Next, we'll need to deploy a database.
MySQL Database
In the Azure Portal, select Azure Database for MySQL flexible servers, and choose the quick create option for the MySQL option (no WordPress needed. On the create page for the Flexible server, ensure that you have a standard (B1) compute SKU with at least 20GiB storage, Dev/Test workload is sufficient for a sample application like this one. After choosing a name, region, admin username and password, create your MySQL Server.
Now we have a database, on to Azure OpenAI!
Azure OpenAI
In the Azure Portal, choose Azure OpenAI service and click create. After choosing your options for naming, resource group, etc. (this demo uses an S0 sku), create your OpenAI service. In the Azure OpenAI Studio, be sure to select and deploy the gpt-35-turbo model, this is what we will use for chat completions in our SpringBoot app.
Managed Identities
To connect all our resources together, we will use Managed Identity. This ensures our connections are handled by Microsoft Entra in a secure, passwordless setup. There are no secrets to manage here. With our Managed Identities properly configured, your Azure resources will automatically have the permissions to connect to each other securely without any need for secrets management.
Starting with our App Service Web App, let's first enable a system assigned identity for this service under Settings > Identity.
After enabling our system assigned identity for our Web App, we'll need to create some user assigned managed identity resources. One for our MySQL database and one for our Azure OpenAI resource. We'll also use the Azure OpenAI identity for GitHub Actions later, but you can always create a Managed Identity resource per resource access if desired, or even use a single user managed identity for all resource access associated with this application. The passwordless end-result is the same either way.
Starting with our identity for Azure OpenAI, go to Managed Identities in the portal and click create. After naming your identity, go to Azure role assignments and click Add role assignment. Here, I've enabled this identity to be a few things, a Cognitive Services OpenAI user on my Azure OpenAI resource, a Cognitive Services Contributor on my resource group for this app, and a Website Contributor to the Staging slot of my App Service Web App.
Next, let's create our identity for the MySQL database. In this case, I left this one a bit open-ended. Following the same procedure as the last Managed Identity, but in this case making our MySQL related identity a Contributor to our resource-group as a whole. This allows us to effectively have one Managed Identity that can "contribute" to anything in the selected resource group. In this case, we're only interested in the MySQL server.
Securely Connecting Our Resources
Now that we've created all our managed identities, let's add them to our Web App. After browsing to our App Service Web App in the portal, select Settings > Identity > User assigned. Here we'll add the two identities we just created.
We're almost done, but for this next connection, we need to use the Azure CLI and Service Connector. In your terminal, run the following command to securely connect your MySQL database to your Web App using Service Connector.
az webapp connection create mysql-flexible --resource-group ${RESOURCE_GROUP} --name ${WEBAPP_NAME} --target-resource-group ${RESOURCE_GROUP} --server ${MYSQL_SERVER_NAME} --database petclinic --user-identity client-id=${MYSQL_MANAGED_IDENTITY_CLIENT_ID} subs-id=${SUBSCRIPTION_ID} mysql-identity-id=${MYSQL_MANAGED_IDENTITY_OBJECT_ID} --client-type javaBe sure to store the username produced here, this will be important later. Now all our resources are securely connected! On to the source code to put all these resources to good use.
The Source Code
Dependencies
Our first stop is the pom.xml file. Be sure to include the following dependencies in your pom.xml file. Remember, the full source code for this application is available on GitHub.
Code:
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-ai-openai</artifactId>
<version>1.0.0-beta.8</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.12.2</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.30</version>
</dependency>
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity-extensions</artifactId>
<version>1.1.5</version>
</dependency>
</dependencies>After adding the required dependencies, let's look at the Chatbot source code itself.
Application Source Code for Intelligent Chatbot using SpringBoot & Azure OpenAI
Starting off with directory structure, the Chatbot in its entirety lives here:
src\main\java\org\springframework\samples\petclinic\chatThis directory contains 3 files, ChatController.java, WebSocketConfig.java, and WebSocketEventListener.java. Let's examine the ChatController individually and take a quick look at our WebSocket related files.
ChatController.java
This is some boilerplate code for a pretty standard Chatbot using Web Sockets running in a SpringBoot app. The goal of this basic chat controller is that it can be reused, modified, or otherwise repurposed for a wide range of uses.
While I won't go line-by-line, I'll highlight a few major parts of the code. The first being the dependencies associated with Azure OpenAI and Managed Identity, these are all essential.
Code:
import com.azure.ai.openai.OpenAIClient;
import com.azure.ai.openai.OpenAIClientBuilder;
import com.azure.ai.openai.models.ChatChoice;
import com.azure.ai.openai.models.ChatCompletions;
import com.azure.ai.openai.models.ChatCompletionsOptions;
import com.azure.ai.openai.models.ChatRequestMessage;
import com.azure.ai.openai.models.ChatRequestAssistantMessage;
import com.azure.ai.openai.models.ChatRequestSystemMessage;
import com.azure.ai.openai.models.ChatRequestUserMessage;
import com.azure.ai.openai.models.ChatResponseMessage;
import com.azure.identity.DefaultAzureCredentialBuilder;
import com.azure.core.credential.TokenCredential;Next let's look at the ChatController class. The sendMessageAI method contains the majority of our Chatbot configuration. The following lines contain the Managed Identity connection to Azure OpenAI.
Code:
TokenCredential defaultCredential = new DefaultAzureCredentialBuilder().build();
OpenAIClient client = new OpenAIClientBuilder().credential(defaultCredential).endpoint("https://java-demo.openai.azure.com/").buildClient();The remaining content of the method handles the AI portion of the chatbot, in this case, instructing our Azure OpenAI model on what role to play during our chat and handling the receiving and answering of user provided chat messages. All of this is done over the Web Socket spec defined in WebSocketConfig.java and WebSocketEventListener.java.
The remaining methods in this file define the control flow for messages sent by the user (sendMessage), and users added to the chat (addUser).
The Azure OpenAI connection and usage scenario may seem simple to set up (and it is), but the results are incredibly powerful and the possibilities for integrations with business apps are limitless. Now that we've seen the OpenAI setup, let's look at our database connection.
Database Connection with Managed Identity
Back when we made our managed identities and used Service Connector to connect our database and web app, you should have received a MySQL username that looks something like: aad_mysqlflexible_abcd. Now we'll take this username and add it to our SpringBoot configuration file, in this case, the one for MySQL called application-mysql.properties. Edit the file to include your database URL and MYSQL username like the example below.
Code:
# database init, supports mysql too
database=mysql
spring.datasource.url=${MYSQL_URL:jdbc:mysql://java-jdemo-mysql-db.mysql.database.azure.com/petclinic}
spring.datasource.username=${MYSQL_USER:aad_mysqlflexible_e8bir}
# SQL is written to be idempotent so this is safe
spring.sql.init.mode=alwaysSetting spring.sql.init.mode=always lets us execute the data.sql file in our resources/ directory automatically on startup.
In just a few steps, we've connected a MySQL database to our App Service web app using managed identity and service connector for secure passwordless database connections. Next up, we'll look at some of our deployment options.
Deployments
Deployments - CLI
Let's look at deploying our app, starting with the Azure CLI and Maven. The first step with any Maven-based build and release for a Java application is to build our app. We can start by running a Maven build using the following command:
mvn package -DskipTestsI'm skipping tests for the purpose of demonstration, but you may want to avoid this with a production app.
Next up, deploy your application JAR file with the Azure CLI like this:
az webapp deploy --resource-group ${RESOURCE_GROUP} --name ${WEBAPP_NAME} --src-path target/${APPNAME_AND_VERSION}.jar Checking your deployed application, you should see something like this:
Now browsing to the chat tab should give you the option to chat with an AI assistant powered by Azure OpenAI!
If that seems easy, it's because it is. We can add our own data, modify the behavior of our chatbot, and even AI framework we're using for the app, but we'll save that for a later post.
Deployments - GitHub Actions with Managed Identity
CLI deployments are a great way to get started, but CI/CD is typically the preferred deployment method for web apps. Using GitHub Actions with Azure App Service, CI/CD is straightforward and easy to configure. In this case, having our Managed Identities already configured will make it even easier.
Starting off, we'll need to browse to our web app in the Azure portal and select Deployment > Deployment Center. Here we can configure our app to deploy from a number of different sources, but we'll select GitHub for now. Picking our Organization, Repository, and Branch, we're then allowed to create a new workflow file or add a new one. Let's just add a new one, which we can preview later. This workflow just performs a standard Maven build and deployment to our app on merges into main.
We'll address the notice at the top of the page in the next section, but for now let's look at the authentication settings. Here we can use an existing user-assigned identity (Managed Identity), or we can create a new one. If you're on a Free, Basic, or Standard SKU, select your preferred identity (or create a new one) and click save. Otherwise, click Discard and continue to the next section to set up a staging slot for your web app. Using Deployment Slots (Premium SKU Required)
To set up deployment slots for your app, you must be on a Premium SKU. Browse to your web app > Deployment > Deployment Slots and click Add.
Here you can name your staging slot and clone the settings of your production slot. After adding the slot, click on it to switch over to your new staging slot. Now on the staging slot, we'll configure GitHub Actions deployments just as we did before. Under Deployment > Deployment Center we'll fill out the form naming GitHub as our source, specifying Organization, Repository, Branch, selecting to add a new workflow and configuring our user-assigned identity for auth. After clicking save, a deployment will be kicked off and the app will be deployed to your staging slot.
Slot Swap into Production
While the staging slot is great for ensuring our app is fully functional before we deploy to production, eventually, we'll need to officially swap our slots and deploy our production app. To do this, go back to Deployment > Deployment slots and select Swap. Here you'll need to set your source and the staging slot and target as the production slot. After clicking start swap and waiting for the changes to complete, your application from staging will be fully deployed and running in production.
Now any time a change is made, the staging slot will be updated first and, after a manual review, can be swapped into production at any time. This also works in reverse, if a production app is having issues, swapping the previous version from the staging slot can effectively revert a change that had unintended consequences.
Conclusion
Configuring and deploying secure OpenAI powered applications to Azure App Service has never been easier! With the flexibility of the PaaS platform, the passwordless connections, and the power of Azure OpenAI, the concepts demonstrated here can be applied to a wide range of applications. Reach out with any questions or comments, this article will contain links in the future to other versions of the same app using LangChain4j and SpringAI.
Continue reading...