C
Chris_Noring
Ok, so you might have read somewhere that API keys is not secure, and you might even have heard about this managed identity thing. But what is it, and why is it better than API keys? Let's try to answer that question and show a practical example of how to use managed identities in Azure.
References
- APIM + Generative AI intro post]
- Move away from API keys]
- APIM + managed identity
- Azure Open AI Managed identity
- Dev Center
What is managed identity?
It's a feature in Azure that allows applications to authenticate to cloud services securely without needing to manage credentials like API keys.
Managed identity in Azure works by providing an *automatically managed identity* for applications to use when connecting to resources that support Azure Active Directory (Azure AD) authentication.
> Ok, so I don't need to create this identity myself, sounds good so far.
Types of Managed Identities
Here's a breakdown of how it works. There are two types of managed identities in Azure:
System-Assigned Managed Identity
- Creation: Automatically created when enabled on an Azure resource like a virtual machine or an app service.
- Lifecycle: Tied to the lifecycle of the resource. When the resource is deleted, the managed identity is also deleted.
- Usage: Only the resource it is created for can use this identity to request tokens from Azure AD¹.
> Ok, so tied to a specific resource, got it. What about the other type?
User-Assigned Managed Identity:
- Creation: Created as a standalone Azure resource.
- Lifecycle: Managed separately from the resources that use it.
- Usage: Can be assigned to multiple Azure resources, allowing them to share the same identity.
> Ok, so this one is more flexible and can be shared across resources and there are different types, but how does it actually work, take me through the process.
Process - How It Works
Here's the process from authentication to accessing resources:
1. Authentication:
- When an application needs to access a resource, it requests a token from Azure AD using its managed identity.
- Azure AD authenticates the managed identity and issues a token.
> Ok great, I get a token, but how does it actually use it to access the resource?
2. Accessing Resources:
- The application uses the token to access the desired resource (e.g., Azure Key Vault, Azure Storage).
- The resource validates the token and grants access based on the permissions assigned to the managed identity.
> Ok, so essentially, it's like a middleman that gets me the token to access the resource securely.
Practical example: Azure Open AI Service + APIM
Ok, let's see how we can use managed identities in a real-world scenario. We'll use the Azure Open AI Service as an example. So you started using the Azure Open AI Service to generate text for your application, fantastic. You're currently using an API key to authenticate your application with the service and your tech lead mentioned that it's not the most secure way to do it - use managed identities instead.
> Ok, how hard can it be? Let's see if you can plan out the steps.
Oh, you also remember that you need additional features like error management, load balancing, keep track of usage, and more so you also need to consider that in your plan.
You're giving it some thoughts and decide you need Azure API Management to handle the additional features and Azure Open AI Service. But how do we enable managed identities in this setup?
-0- Create the cloud resources
You need to create an Azure API Management instance and an Azure Open AI Service. You can do this in the Azure portal or using Bicep or ARM templates.
We'll go through a real example in a bit, but for now, let's assume you have these resources created.
-1- Azure API Management (APIM)
Enabling managed identity for APIM is done in the following way:
Enable Managed Identity:
- Go to the Azure portal.
- Navigate to the Azure API Management instance.
- Select Security > Managed identity > Set status to On.
Add managed identity policy to the API
You also need to add policy to the API for your managed identity to be used. You can do this by adding the following policy to your API
- Go to your API in the Azure API Management instance.
- Add the below to your API in Design mode:
Code:
<authentication-managed-identity resource="https://cognitiveservices.azure.com" output-token-variable-name="managed-id-access-token" ignore-error="false" />
<set-header name="Authorization" exists-action="override">
<value>@("Bearer " + (string)context.Variables["managed-id-access-token"])</value>
</set-header>The above policy will get the token from the managed identity and set it as the Authorization header for the API call to the Azure Open AI Service.
-2- Azure Open AI Service: Enabling managed identity for the Azure Open AI
There are two steps to enable managed identity for the Azure Open AI Service:
Enable Managed Identity:
- Go to the Azure portal.
- Navigate to the Azure Open AI Service instance.
- Enable the managed identity for the service by selecting Resource Management > Identity > System-assigned. Once in there you can enable the system-assigned managed identity.
Assign Permissions:
Because APIM will be calling the Azure Open AI Service, you need to assign permission for APIM to call the Azure Open AI Service.
In Azure Portal for the resource:
- Select Access Control (IAM) > + Add > Add role assignment > Search for "Cognitive Services User"
- Select **Next**, check managed identity and select "+ Select members". Here you select your APIM instance and click **Select**.
Great, now you have the managed identity enabled for both APIM and Azure Open AI Service, and you have assigned the necessary permissions for APIM to call the Azure Open AI Service.
-3- Try it out
For the practical parts of this, let's use an existing Azure Sample [Azure sample for Azure Open AI and APIM](TODO)
Open a terminal:
- git clone <GitHub - Azure-Samples/genai-gateway-apim: sample repo for APIM + Gen AI>
- Rn `azd auth login`, this should log you in to your azure account.
- Run `azd up`, this should deploy the resources to your azure account.
- Navigate to `src` folder in the cloned repo.
- Run `npm install`, this will install the necessary packages.
- Run `npm start`
- open browser and navigate to http://localhost:3000
- Input your text and select the arrow to send the message, you should see the generated text.
Infrastructure as code
So far, we've shown you how to set up the managed identity for APIM and Azure Open AI Service manually via Azure Portal, but you can also do this using Infrastructure as Code (IaC) tools like Bicep or ARM templates. Let's show how to do this using Bicep.
Bicep file for APIM
Let's show how to enable managed identity for APIM using Bicep, while also creating it:
Code:
resource apimService 'Microsoft.ApiManagement/service@2023-09-01-preview' = {
name: name
location: location
tags: union(tags, { 'azd-service-name': name })
sku: {
name: sku
capacity: (sku == 'Consumption') ? 0 : ((sku == 'Developer') ? 1 : skuCount)
}
properties: {
publisherEmail: publisherEmail
publisherName: publisherName
// Custom properties are not supported for Consumption SKU
customProperties: sku == 'Consumption' ? {} : {
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_GCM_SHA256': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA256': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA256': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TripleDes168': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls10': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls11': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Ssl30': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls10': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls11': 'false'
'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Ssl30': 'false'
}
}
identity: {
type: 'SystemAssigned'
}
}From the viewpoint of the managed identity, you can see that we've added the `identity` property to the APIM resource with the type set to `SystemAssigned`.
Check out this Bicep file for APIM for more details.
Bicep file for Azure Open AI Service
For Azure Open AI Service, you can enable the managed identity using the following Bicep code:
Code:
resource openAI 'Microsoft.CognitiveServices/accounts@2021-04-30' = {
name: 'your-openai-resource-name'
location: 'your-location'
sku: {
name: 'S0'
}
kind: 'OpenAI'
properties: {
// Add other necessary properties here
}
identity: {
type: 'SystemAssigned'
}
properties: {
publicNetworkAccess: 'Disabled'
networkAcls: {
defaultAction: 'Deny'
}
disableLocalAuth: true
}
}As you can see, we've added the `identity` property to the Azure Open AI Service resource with the type set to `SystemAssigned`, same as with APIM.
Another thing to call out for security:
- the `networkAcls` property, which is set to `Deny` by default. This is a security feature that restricts access to the Azure Open AI Service to only the specified resources.
- Also, `disableLocalAuth` is set to `true` to disable local authentication.
Then we assign the necessary permissions for APIM to call the Azure Open AI Service:
Code:
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
name: guid(openAI.id, 'cognitive-services-openai-user-role')
properties: {
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'c1c469a3-0a2d-4bba-b0e1-0eaf1d3d728b') // Role ID for Cognitive Services OpenAI User
principalId: openAI.identity.principalId
principalType: 'ServicePrincipal'
scope: openAI.id
}
}Note how the `scope` is set to the `openAI.id`, which is the Azure Open AI Service resource ID. That means that only the Azure Open AI Service can be called by the APIM instance.
Scope discussion
When assigning the "Cognitive Services OpenAI User" role for your APIM managed identity, the scope can be set at either the Cognitive Services account level or the resource group level, depending on your needs:
- Cognitive Services Account Level: This is more restrictive and ensures that the managed identity only has access to the specific Azure OpenAI resource. This is useful if you want to limit access to just one resource.
- Resource Group Level: This is broader and allows the managed identity to access all resources within the specified resource group. This is useful if you have multiple Azure OpenAI resources within the same resource group and want to manage access more easily.
However, if you want to allow other resources to call the Azure Open AI Service, you can specify the scope as the subscription or resource group level. Here's how you add it to the role assignment, on resource group level:
Code:
resource roleAssignment 'Microsoft.Authorization/roleAssignments@2020-04-01-preview' = {
name: guid(apimIdentity.id, resourceGroup().id, 'cognitive-services-openai-user-role')
properties: {
roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'c1c469a3-0a2d-4bba-b0e1-0eaf1d3d728b') // Role ID for Cognitive Services OpenAI User
principalId: apimIdentity.properties.principalId
principalType: 'ServicePrincipal'
scope: resourceGroup().id
}
}Calling the APIM endpoint
So far, you've seen how we can secure both Azure OpenAI and Azure API Management by having them use managed identity, great.
I bet you're wondering how we call APIM, let's explain that next.
Subscriptions in Azure API Management are a way to control access to APIs. When you publish APIs through APIM, you can secure them using subscription keys. Here’s a quick overview:
- Subscriptions: These are containers for a pair of subscription keys (primary and secondary). Developers need a valid subscription key to call the APIs.
- Subscription IDs: Each subscription has a unique identifier called a Subscription ID.
How does Subscription relate to the APIM resource though?
Scope of Subscriptions: Subscriptions can be associated with different scopes within an APIM instance:
- Product Scope: Subscriptions can be linked to a specific product, which is a collection of one or more APIs. Developers subscribe to the product to access all APIs within it.
- API Scope: Subscriptions can also be associated with individual APIs, allowing more granular control over access.
Code:
resource api1 'Microsoft.ApiManagement/service/apis@2020-06-01-preview' = {
parent: apimService
name: apiName
properties: {
displayName: apiName
apiType: 'http'
path: apiSuffix
format: 'openapi+json-link'
value: 'https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2024-03-01-preview/inference.json'
subscriptionKeyParameterNames: {
header: 'api-key'
}
}
resource apimDiagnostics 'diagnostics@2023-05-01-preview' = {
name: 'applicationinsights' // Use a supported diagnostic identifier
properties: {
loggerId: '/subscriptions/${subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.ApiManagement/service/${apimService.name}/loggers/${apimLogger.name}'
metrics: true
}
}
}
// Creating a product for the API. Products are used to group APIs and apply policies to them
resource product 'Microsoft.ApiManagement/service/products@2020-06-01-preview' = {
parent: apimService
name: productName
properties: {
displayName: productName
description: productDescription
state: 'published'
subscriptionRequired: true
}
}
// Create PRODUCT-API association the API with the product
resource productApi1 'Microsoft.ApiManagement/service/products/apis@2020-06-01-preview' = {
parent: product
name: api1.name
}
// Creating a user for the API Management service
resource user 'Microsoft.ApiManagement/service/users@2020-06-01-preview' = {
parent: apimService
name: 'userName'
properties: {
firstName: 'User'
lastName: 'Name'
email: 'user@example.com'
state: 'active'
}
}
// Creating a subscription for the API Management service
// NOTE: the subscription is associated with the user and the product, AND the subscription ID is what will be used in the request to authenticate the calling client
resource subscription 'Microsoft.ApiManagement/service/subscriptions@2020-06-01-preview' = {
parent: apimService
name: 'subscriptionAIProduct'
properties: {
displayName: 'Subscribing to AI services'
state: 'active'
ownerId: user.id
scope: product.id
}
}The following happens:
- An API is declared
- A product is defined
- An association between product and API is made
- A user is created
- Finally a subscription is created with the user and the scope is on product.
We can use this created subscription to call it in code, here's how:
Code:
let body = {
"model":"gpt-35-turbo","messages":[
{
"role":"system","content":"You're a helpful assistant"
},
{
"role":"user","content":prompt
}
]};
return fetch(URL_CHAT, {
method: "POST",
headers: {
"api-key": process.env.SUBSCRIPTION_KEY,
"Content-Type": "application/json"
},
body: JSON.stringify(body)
})In the preceding code, you can see how you call the APIM API with your subscription ID, but wait a key, can we do better?
Is this secure enough or?
Well, there are a couple of things we can do:
- Follow some general best practice guidelines on key rotation, store in Azure KeyVault and even restrict which API range can call and more.
- The other thing we can, which is outside the scope of this article is to use Entra and OAuth , read more about that here.
Summary - Benefits
Let's summarize our learning in this article. Managed identities offer several benefits over traditional API keys:
- No Credential Management: Eliminates the need to store and manage credentials in your code.
- Enhanced Security: Reduces the risk of credential leakage since credentials are not exposed.
- Simplified Access Control: Managed identities can be assigned specific roles and permissions, providing granular access control.
To use managed identity, we need to enable it on the Azure resources and configure the necessary permissions.
Now you can hopefully sleep better at night knowing that your applications are more secure with managed identities.
Continue reading...