Deploy a Phi-3 model in Azure AI, and consume it with C# and Semantic Kernel

[elbruno]

Hi friends!



Let's keep writing about Phi-3, and today's post is a demo about how to deploy Phi-3 model to Azure AI, and consuming it using C# and Semantic Kernel. This guide will walk you through the process, covering chat completion text scenarios and vision scenarios.

Why Phi-3 and Semantic Kernel?​

Phi-3 is a state-of-the-art model that brings advanced AI capabilities to your projects, enhancing both text and vision scenarios. Coupled with the Semantic Kernel, an orchestrator that handles authentication and interactions with the cloud, you have a robust setup for deploying and using AI models efficiently.



Bonus: Semantic Kernel allows literally zero code changes to move from a local Phi-3 deployment to a cloud Phi-3 deployment for your applications.

Step 1: Deploy Phi-3 on Azure AI​

First things first, let's get Phi-3 up and running on Azure AI. Follow these steps:

• Create an Azure AI Hub: Head over to the Azure AI Studio and create a new Azure AI Hub.
• Deploy the Phi-3 model: In the deployments, deploy a new base model, and select a Phi-3 model.

• Serverless API: select the server API option.

• Content Filter (Optional): Enable Content Filter if you want to use this model with a CF.

• Endpoint Data: After a short time the model will be deployed. In the Endpoint / API Routes section, copy the Chat Completion Name, and the Endpoint. Also copy the Key.

​

Step 2: Let's code!​

With Phi-3 deployed, we can write C# code. Below there are 2 samples on how to use the Azure model for text and vision scenarios. In both scenarios the Azure Model details are stored using User Secrets. In example, you need to setup your projects secrets with the following commands:

dotnet user-secrets init
dotnet user-secrets set "AZURE_AI_PHI3_MODEL" "phi3-mini-4k"
dotnet user-secrets set "AZURE_AI_PHI3_ENDPOINT" "< endpoint >"
dotnet user-secrets set "AZURE_AI_PHI3_APIKEY" "< api key > "

Chat Completion Text Scenario​

Here’s a simple example of how to use Phi-3 for generating chat responses.

#pragma warning disable SKEXP0010 
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var config = new ConfigurationBuilder().AddUserSecrets<Program>().Build();
var modelId = config["AZURE_AI_PHI3_MODEL"];
var endPoint = config["AZURE_AI_PHI3_ENDPOINT"];
var apiKey = config["AZURE_AI_PHI3_APIKEY"];

// create kernel
var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(modelId, new Uri(endPoint), apiKey);
var kernel = builder.Build();

// create chat
var chat = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory();

// run chat
while (true)
{
    Console.Write("Q: ");
    var userQ = Console.ReadLine();
    if (string.IsNullOrEmpty(userQ))
    {
        break;
    }
    history.AddUserMessage(userQ);

    Console.Write($"Phi3: ");
    var response = "";
    var result = chat.GetStreamingChatMessageContentsAsync(history);
    await foreach (var message in result)
    {
        Console.Write(message.Content);
        response += message.Content;
    }
    history.AddAssistantMessage(response);
    Console.WriteLine("");
}

​

Vision Scenario​

For vision scenarios, Phi-3 can analyze images and provide insights. Here’s an example of how to achieve this.

#pragma warning disable SKEXP0010 
using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;

var config = new ConfigurationBuilder().AddUserSecrets<Program>().Build();
var modelId = config["AZURE_AI_PHI3V_MODEL"];
var endPoint = config["AZURE_AI_PHI3V_ENDPOINT"];
var apiKey = config["AZURE_AI_PHI3V_APIKEY"];

// create kernel
var builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(modelId, new Uri(endPoint), apiKey);
var kernel = builder.Build();

// create chat
var chat = kernel.GetRequiredService<IChatCompletionService>();
var history = new ChatHistory();

var petsMusicImagePath = Path.Combine(Directory.GetCurrentDirectory(), "imgs", "petsmusic.png");

// get the mime type from the image
var mimeType = "image/png";

// create chat collection items
var collectionItems = new ChatMessageContentItemCollection
{
    new TextContent("What's in the image?"),
    new ImageContent(File.ReadAllBytes(petsMusicImagePath), mimeType)
};
history.AddUserMessage(collectionItems);

Console.Write($"Phi3: ");
var result = await chat.GetChatMessageContentsAsync(history);
Console.WriteLine(result[^1].Content);

Console.WriteLine("");

Recording​

The following 5 minute video shows the full process and running demos:

Wrapping Up​

Deploying and using Phi-3 on Azure AI with C# and Semantic Kernel opens up a world of possibilities in a simple way. Whether you're working on chatbots, vision applications, or any other AI-powered solution, this setup provides a scalable and efficient way to harness the power of Phi-3.

For more detailed instructions and examples, check out the Phi-3 Cookbook.



Happy coding!

Bruno Capuano

Continue reading...