Deploy Secure Azure AI Studio with a Managed Virtual Network

  • Thread starter Thread starter paolosalvatori
  • Start date Start date
P

paolosalvatori

This article and the companion sample demonstrates how to set up an Azure AI Studio environment with managed identity and Azure RBAC to connected Azure AI Services and dependent resources and with the managed virtual network isolation mode set to Allow Internet Outbound. For more information, see How to configure a managed network for Azure AI Studio hubs. For more information, see:






Azure Resources​


You can use the Bicep templates in this GitHub repository to deploy the following Azure resources:



managed-virtual-network.png





ResourceTypeDescription
Azure Application InsightsMicrosoft.Insights/componentsAn Azure Application Insights instance associated with the Azure AI Studio workspace
Azure Monitor Log AnalyticsMicrosoft.OperationalInsights/workspacesAn Azure Log Analytics workspace used to collect diagnostics logs and metrics from Azure resources
Azure Key VaultMicrosoft.KeyVault/vaultsAn Azure Key Vault instance associated with the Azure AI Studio workspace
Azure Storage AccountMicrosoft.Storage/storageAccountsAn Azure Storage instance associated with the Azure AI Studio workspace
Azure Container RegistryMicrosoft.ContainerRegistry/registriesAn Azure Container Registry instance associated with the Azure AI Studio workspace
Azure AI Hub / ProjectMicrosoft.MachineLearningServices/workspacesAn Azure AI Studio Hub and Project (Azure ML Workspace of kind 'hub' and 'project')
Azure AI ServicesMicrosoft.CognitiveServices/accountsAn Azure AI Services as the model-as-a-service endpoint provider including GPT-4o and ADA Text Embeddings model deployments
Azure Virtual NetworkMicrosoft.Network/virtualNetworksA bring-your-own (BYO) virtual network hosting a jumpbox virtual machine to manage Azure AI Studio
Azure Bastion HostMicrosoft.Network/virtualNetworksA Bastion Host defined in the BYO virtual network that provides RDP connectivity to the jumpbox virtual machine
Azure NAT GatewayMicrosoft.Network/natGatewaysAn Azure NAT Gateway that provides outbound connectivity to the jumpbox virtual machine
Azure Private EndpointsMicrosoft.Network/privateEndpointsAzure Private Endpoints defined in the BYO virtual network for Azure Container Registry, Azure Key Vault, Azure Storage Account, and Azure AI Hub Workspace
Azure Private DNS ZonesMicrosoft.Network/privateDnsZonesAzure Private DNS Zones are used for the DNS resolution of the Azure Private Endpoints



You can select a different version of the GPT model by specifying the openAiDeployments parameter in the main.bicepparam parameters file. For details on the models available in various Azure regions, please refer to the Azure OpenAI Service models documentation.


The default deployment includes an Azure Container Registry resource. However, if you wish not to deploy an Azure Container Registry, you can simply set the acrEnabled parameter to false.



Network isolation architecture and isolation modes​


When you enable managed virtual network isolation, a managed virtual network is created for the hub workspace. Any managed compute resources you create for the hub, for example the virtual machines of online endpoint managed deployment, will automatically use this managed virtual network. The managed virtual network can also utilize Azure Private Endpoints for Azure resources that your hub depends on, such as Azure Storage, Azure Key Vault, and Azure Container Registry. There are three different configuration modes for outbound traffic from the managed virtual network:



Outbound modeDescriptionScenarios
Allow internet outboundAllow all internet outbound traffic from the managed virtual network.You want unrestricted access to machine learning resources on the internet, such as python packages or pretrained models.
Allow only approved outboundOutbound traffic is allowed by specifying service tags.You want to minimize the risk of data exfiltration, but you need to prepare all required machine learning artifacts in your private environment.
* You want to configure outbound access to an approved list of services, service tags, or FQDNs.
DisabledInbound and outbound traffic isn't restricted.You want public inbound and outbound from the hub.



The Bicep templates in the companion sample demonstrate how to deploy an Azure AI Studio environment with the hub workspace's managed network isolation mode configured to Allow Internet Outbound.

The Azure Private Endpoints and Private DNS Zones in the hub workspace managed virtual network are automatically created for you, while the Bicep templates create the Azure Private Endpoints and relative Private DNS Zones in the client virtual network.




Managed Virtual Network​


When you provision the hub workspace of your Azure AI Studio with an isolation mode equal to the Allow Internet Outbound isolation mode, the managed virtual network and the Azure Private Endpoints to the dependent resources will not be created if public network access of Azure Key Vault, Azure Container Registry, and Azure Storage Account dependent resources is enabled.

The creation of the managed virtual network is deferred until a compute resource is created or provisioning is manually started. When allowing automatic creation, it can take around 30 minutes to create the first compute resource as it is also provisioning the network. For more information, see Manually provision workspace managed VNet.


If you initially create Azure Key Vault, Azure Container Registry, and Azure Storage Account dependent resources with public network enabled and then decide to disable it later, the managed virtual network will not be automatically provisioned if it is not already provisioned, and the private endpoints to the dependent resources will not be created.

In this case, if you want o create the private endpoints to the dependent resources, you need to reprovision the hub manage virtual network in one of the following ways:


  • Redeploy the hub workspace using Bicep or Terraform templates. If the isolation mode is set to Allow Internet Outbound and the dependent resources referenced by the hub workspace have public network access disabled, this operation will trigger the creation of the managed virtual network, if it does not already exist, and the private endpoints to the dependent resources.
  • Execute the following Azure CLI command az ml workspace provision-network to reprovision the managed virtual network. The private endpoints will be created with the managed virtual network if the public network access of the dependent resources is disabled.

az ml workspace provision-network --name my_hub_workspace_name --resource-group
At this time, it's not possible to directly access the managed virtual network via the Azure CLI or the Azure Portal. You can see the managed virtual network indirectly by looking at the private endpoints, if any, under the hub workspace. You can proceed as follows:



  1. Go to the Azure Portal and select your Azure AI hub.
  2. Click on Settings and then Networking.
  3. Open the Workspace managed outbound access tab.
  4. Expand the section titled Required outbound rules.



Here, you will find the private endpoints that are connected to the resources within the hub managed virtual network. Ensure that these private endpoints are active.



hub-workspace.png

You can also see the private endpoints hosted by the manage virtual network of your hub workspace inside the Networking settings of individual dependent resources, for example Key Vault:


  1. Go to the Azure Portal and select your Azure Key Vault.
  2. Click on Settings and then Networking.
  3. Open the Private endpoint connections tab.


Here, you will find the private endpoint created by the Bicep templates in the client virtual network along with the private endpoint created in the hub managed virtual network of the hub.



key-vault.png

Also note that when you create a hub workspace with the Allow Internet Outbound isolation mode, the creation of the managed network is not immediate to save costs. The managed virtual network needs to be manually triggered via the az ml workspace provision-network command, or it will be triggered when you create a compute resource or private endpoints to dependent resources.

At this time, the creation of an online endpoint does not automatically trigger the creation of a managed virtual network. An error occurs if you try to create an online deployment under the workspace which enabled workspace managed VNet but the managed VNet is not provisioned yet. Workspace managed VNet should be provisioned before you create an online deployment. Follow instructions to manually provision the workspace managed VNet. Once completed, you may start creating online deployments. For more information, see Network isolation with managed online endpoint and Secure your managed online endpoints with network isolation.



Limitations​


The current limitations of managed virtual network are:

  • Azure AI Studio currently doesn't support bringing your own virtual network, it only supports managed virtual network isolation.
  • Once you enable managed virtual network isolation of your Azure AI, you can't disable it.
  • Managed virtual network uses private endpoint connections to access your private resources. You can't have a private endpoint and a service endpoint at the same time for your Azure resources, such as a storage account. We recommend using private endpoints in all scenarios.
  • The managed virtual network is deleted when the Azure AI is deleted.
  • Data exfiltration protection is automatically enabled for the only approved outbound mode. If you add other outbound rules, such as to FQDNs, Microsoft can't guarantee that you're protected from data exfiltration to those outbound destinations.
  • Using FQDN outbound rules increases the cost of the managed virtual network because FQDN rules use Azure Firewall. For more information, see Pricing.
  • FQDN outbound rules only support ports 80 and 443.
  • When using a compute instance with a managed network, use the az ml compute connect-ssh command to connect to the compute using SSH.



Pricing​


According to the documentation, the hub managed virtual network feature is free. However, you will be charged for the following resources used by the managed virtual network:

  • Azure Private Link - Private endpoints used to secure communications between the managed virtual network and Azure resources rely on Azure Private Link. For more information on pricing, see Azure Private Link pricing.
  • FQDN outbound rules - FQDN outbound rules are implemented using Azure Firewall. If you use outbound FQDN rules, charges for Azure Firewall are included in your billing. Azure Firewall SKU is standard. Azure Firewall is provisioned per hub.

NOTE

The firewall isn't created until you add an outbound FQDN rule. If you don't use FQDN rules, you will not be charged for Azure Firewall. For more information on pricing, see Azure Firewall pricing.



Secure Access to the Jumpbox Virtual Machine​


The jumpbox virtual machine is deployed with Windows 11 operating system and the Microsoft.Azure.ActiveDirectory VM extension, a specialized extension for integrating Azure virtual machines (VMs) with Microsoft Entra ID. This integration provides several key benefits, particularly in enhancing security and simplifying access management. Here's an overview of what the Microsoft.Azure.ActiveDirectory VM extension offers:

Microsoft.Azure.ActiveDirectory VM extension is specialized for integrating Azure virtual machines (VMs) with Microsoft Entra ID. This integration provides several key benefits, particularly in enhancing security and simplifying access management. Here's an overview of the features and benefits of this VM extension:



  • Enables users to sign in to a Windows or Linux virtual machine using their Microsoft Entra ID credentials.
  • Facilitates single sign-on (SSO) experiences, reducing the need for managing separate local VM accounts.
  • Supports multi-factor authentication, increasing security by requiring additional verification steps during login.
  • Integrates with Azure RBAC, allowing administrators to assign specific roles to users, thereby controlling the level of access and permissions on the virtual machine.
  • Allows administrators to apply conditional access policies to the VM, enhancing security by enforcing controls such as trusted device requirements, location-based access, and more.
  • Eliminates the need to manage local administrator accounts, simplifying VM management and reducing overhead.



For more information, see Sign in to a Windows virtual machine in Azure by using Microsoft Entra ID including passwordless.

Make sure to enforce multi-factor authentication on your user account in your Microsoft Entra ID Tenant, as shown in the following screenshot:



mfa.png



Then, specify at least an authentication method in addition to the password for the user account, for example the phone number, as shown in the following screenshot:



authentication-methods.png



To log in to the jumpbox virtual machine using a Microsoft Entra ID tenant user, you need to assign one of the following Azure roles to determine who can access the VM. To assign these roles, you must have the Virtual Machine Data Access Administrator role, or any role that includes the Microsoft.Authorization/roleAssignments/write action, such as the Role Based Access Control Administrator role. If you choose a role other than the Virtual Machine Data Access Administrator, it is recommended to add a condition to limit the permission to create role assignments.






To allow a user to sign in to the jumpbox virtual machine over RDP, you must assign the Virtual Machine Administrator Login or Virtual Machine User Login role to the user at the subscription, resource group, or virtual machine level. The virtualMachine.bicep module assigns the Virtual Machine Administrator Login to the user identified by the userObjectId parameter.

To log in to the jumpbox virtual machine via Azure Bastion Host using a Microsoft Entra ID tenant user with multi-factor authentication, you can use the az network bastion rdp command as follows:



Code:
az network bastion rdp \
    --name <bastion-host-name> \
    --resource-group <resource-group-name> \
    --target-resource-id <virtual-machine-resource-id> \
    --auth-type AAD

After logging in to the virtual machine, if you open the Edge browser and navigate to the Azure Portal or Azure AI Studio, the browser profile will automatically be configured to the tenant user account used for the VM login.



Bicep Parameters​


Specify a value for the required parameters in the main.bicepparam parameters file before deploying the Bicep modules. Here is the markdown table extrapolating the name, type, and description of the parameters from the provided Bicep code:



NameTypeDescription
prefixstringSpecifies the name prefix for all the Azure resources.
suffixstringSpecifies the name suffix for all the Azure resources.
locationstringSpecifies the location for all the Azure resources.
hubNamestringSpecifies the name Azure AI Hub workspace.
hubFriendlyNamestringSpecifies the friendly name of the Azure AI Hub workspace.
hubDescriptionstringSpecifies the description for the Azure AI Hub workspace displayed in Azure AI Studio.
hubIsolationModestringSpecifies the isolation mode for the managed network of the Azure AI Hub workspace.
hubPublicNetworkAccessstringSpecifies the public network access for the Azure AI Hub workspace.
connectionAuthTypestringSpecifies the authentication method for the OpenAI Service connection.
systemDatastoresAuthModestringDetermines whether to use credentials for the system datastores of the workspace workspaceblobstore and workspacefilestore.
projectNamestringSpecifies the name for the Azure AI Studio Hub Project workspace.
projectFriendlyNamestringSpecifies the friendly name for the Azure AI Studio Hub Project workspace.
projectPublicNetworkAccessstringSpecifies the public network access for the Azure AI Project workspace.
logAnalyticsNamestringSpecifies the name of the Azure Log Analytics resource.
logAnalyticsSkustringSpecifies the service tier of the workspace: Free, Standalone, PerNode, Per-GB.
logAnalyticsRetentionInDaysintSpecifies the workspace data retention in days.
applicationInsightsNamestringSpecifies the name of the Azure Application Insights resource.
aiServicesNamestringSpecifies the name of the Azure AI Services resource.
aiServicesSkuobjectSpecifies the resource model definition representing SKU.
aiServicesIdentityobjectSpecifies the identity of the Azure AI Services resource.
aiServicesCustomSubDomainNamestringSpecifies an optional subdomain name used for token-based authentication.
aiServicesDisableLocalAuthboolSpecifies whether to disable the local authentication via API key.
aiServicesPublicNetworkAccessstringSpecifies whether or not public endpoint access is allowed for this account.
openAiDeploymentsarraySpecifies the OpenAI deployments to create.
keyVaultNamestringSpecifies the name of the Azure Key Vault resource.
keyVaultNetworkAclsDefaultActionstringSpecifies the default action of allow or deny when no other rules match for the Azure Key Vault resource.
keyVaultEnabledForDeploymentboolSpecifies whether the Azure Key Vault resource is enabled for deployments.
keyVaultEnabledForDiskEncryptionboolSpecifies whether the Azure Key Vault resource is enabled for disk encryption.
keyVaultEnabledForTemplateDeploymentboolSpecifies whether the Azure Key Vault resource is enabled for template deployment.
keyVaultEnableSoftDeleteboolSpecifies whether soft delete is enabled for this Azure Key Vault resource.
keyVaultEnablePurgeProtectionboolSpecifies whether purge protection is enabled for this Azure Key Vault resource.
keyVaultEnableRbacAuthorizationboolSpecifies whether to enable the RBAC authorization for the Azure Key Vault resource.
keyVaultSoftDeleteRetentionInDaysintSpecifies the soft delete retention in days.
acrEnabledboolSpecifies whether to create the Azure Container Registry.
acrNamestringSpecifies the name of the Azure Container Registry resource.
acrAdminUserEnabledboolEnable admin user that have push/pull permission to the registry.
acrPublicNetworkAccessstringSpecifies whether to allow public network access. Defaults to Enabled.
acrSkustringSpecifies the tier of your Azure Container Registry.
acrAnonymousPullEnabledboolSpecifies whether or not registry-wide pull is enabled from unauthenticated clients.
acrDataEndpointEnabledboolSpecifies whether or not a single data endpoint is enabled per region for serving data.
acrNetworkRuleSetobjectSpecifies the network rule set for the container registry.
acrNetworkRuleBypassOptionsstringSpecifies whether to allow trusted Azure services to access a network-restricted registry.
acrZoneRedundancystringSpecifies whether or not zone redundancy is enabled for this container registry.
storageAccountNamestringSpecifies the name of the Azure Storage Account resource.
storageAccountAccessTierstringSpecifies the access tier of the Azure Storage Account resource. The default value is Hot.
storageAccountAllowBlobPublicAccessboolSpecifies whether the Azure Storage Account resource allows public access to blobs. The default value is false.
storageAccountAllowSharedKeyAccessboolSpecifies whether the Azure Storage Account resource allows shared key access. The default value is true.
storageAccountAllowCrossTenantReplicationboolSpecifies whether the Azure Storage Account resource allows cross-tenant replication. The default value is false.
storageAccountMinimumTlsVersionstringSpecifies the minimum TLS version to be permitted on requests to the Azure Storage account. The default value is TLS1_2.
storageAccountANetworkAclsDefaultActionstringThe default action of allow or deny when no other rules match.
storageAccountSupportsHttpsTrafficOnlyboolSpecifies whether the Azure Storage Account resource should only support HTTPS traffic.
virtualNetworkResourceGroupNamestringSpecifies the name of the resource group hosting the virtual network and private endpoints.
virtualNetworkNamestringSpecifies the name of the virtual network.
virtualNetworkAddressPrefixesstringSpecifies the address prefixes of the virtual network.
vmSubnetNamestringSpecifies the name of the subnet which contains the virtual machine.
vmSubnetAddressPrefixstringSpecifies the address prefix of the subnet which contains the virtual machine.
vmSubnetNsgNamestringSpecifies the name of the network security group associated with the subnet hosting the virtual machine.
bastionSubnetAddressPrefixstringSpecifies the Bastion subnet IP prefix. This prefix must be within the virtual network IP prefix address space.
bastionSubnetNsgNamestringSpecifies the name of the network security group associated with the subnet hosting Azure Bastion.
bastionHostEnabledboolSpecifies whether Azure Bastion should be created.
bastionHostNamestringSpecifies the name of the Azure Bastion resource.
bastionHostDisableCopyPasteboolEnable/Disable Copy/Paste feature of the Bastion Host resource.
bastionHostEnableFileCopyboolEnable/Disable File Copy feature of the Bastion Host resource.
bastionHostEnableIpConnectboolEnable/Disable IP Connect feature of the Bastion Host resource.
bastionHostEnableShareableLinkboolEnable/Disable Shareable Link of the Bastion Host resource.
bastionHostEnableTunnelingboolEnable/Disable Tunneling feature of the Bastion Host resource.
bastionPublicIpAddressNamestringSpecifies the name of the Azure Public IP Address used by the Azure Bastion Host.
bastionHostSkuNamestringSpecifies the name of the Azure Bastion Host SKU.
natGatewayNamestringSpecifies the name of the Azure NAT Gateway.
natGatewayZonesarraySpecifies a list of availability zones denoting the zone in which the NAT Gateway should be deployed.
natGatewayPublicIpsintSpecifies the number of Public IPs to create for the Azure NAT Gateway.
natGatewayIdleTimeoutMinsintSpecifies the idle timeout in minutes for the Azure NAT Gateway.
blobStorageAccountPrivateEndpointNamestringSpecifies the name of the private link to the blob storage account.
fileStorageAccountPrivateEndpointNamestringSpecifies the name of the private link to the file storage account.
keyVaultPrivateEndpointNamestringSpecifies the name of the private link to the Key Vault.
acrPrivateEndpointNamestringSpecifies the name of the private link to the Azure Container Registry.
hubWorkspacePrivateEndpointNamestringSpecifies the name of the private link to the Azure Hub Workspace.
vmNamestringSpecifies the name of the virtual machine.
vmSizestringSpecifies the size of the virtual machine.
imagePublisherstringSpecifies the image publisher of the disk image used to create the virtual machine.
imageOfferstringSpecifies the offer of the platform image or marketplace image used to create the virtual machine.
imageSkustringSpecifies the image version for the virtual machine.
authenticationTypestringSpecifies the type of authentication when accessing the virtual machine. SSH key is recommended.
vmAdminUsernamestringSpecifies the name of the administrator account of the virtual machine.
vmAdminPasswordOrKeystringSpecifies the SSH Key or password for the virtual machine. SSH key is recommended.
diskStorageAccountTypestringSpecifies the storage account type for OS and data disk.
numDataDisksintSpecifies the number of data disks of the virtual machine.
osDiskSizeintSpecifies the size in GB of the OS disk of the VM.
dataDiskSizeintSpecifies the size in GB of the data disk of the virtual machine.
dataDiskCachingstringSpecifies the caching requirements for the data disks.
enableMicrosoftEntraIdAuthboolSpecifies whether to enable Microsoft Entra ID authentication on the virtual machine.
enableAcceleratedNetworkingboolSpecifies whether to enable accelerated networking on the virtual machine.
tagsobjectSpecifies the resource tags for all the resources.
userObjectIdstringSpecifies the object ID of a Microsoft Entra ID user.



We suggest reading sensitive configuration data such as passwords or SSH keys from a pre-existing Azure Key Vault resource. For more information, see Create parameters files for Bicep deployment



Getting Started​


To set up the infrastructure for the secure Azure AI Studio, you will need to install the necessary prerequisites and follow the steps below.



Prerequisites​


Before you begin, ensure you have the following:



  • An active Azure subscription
  • Azure CLI installed on your local machine. Follow the installation guide if needed.
  • Appropriate permissions to create resources in your Azure account
  • Basic knowledge of using the command line interface



Step 1: Clone the Repository​


Start by cloning the repository to your local machine:



Code:
git clone <repository_url>
cd bicep

Step 2: Configure Parameters​


Edit the main.bicepparam parameters file to configure values for the parameters required by the Bicep templates. Make sure you set appropriate values for resource group name, location, and other necessary parameters in the deploy.sh Bash script.



Step 3: Deploy Resources​


Use the deploy.sh Bash script to deploy the Azure resources via Bicep. This script will provision all the necessary resources as defined in the Bicep templates.

Run the following command to deploy the resources:



./deploy.sh --resourceGroupName <resource-group-name> --location <location> --virtualNetworkResourceGroupName <client-virtual-network-resource-group-name>

How to Test​


By following these steps, you will have Azure AI Studio set up and ready for your projects using Bicep. If you encounter any issues, refer to the additional resources or seek help from the Azure support team.

After deploying the resources, you can verify the deployment by checking the Azure Portal or Azure AI Studio. Ensure all the resources are created and configured correctly.

You can also follow these instructions to deploy, expose, and call the Basic Chat prompt flow using Bash scripts and Azure CLI.

Continue reading...
 
Back
Top