N
nitya
In May, we announced a deepened partnership with Hugging Face and we continue to add more leading-edge Hugging Face models to the Azure AI model catalog on a monthly basis. We have 1550+ models in the Hugging Face collection and we added 20+ models in July including highly-ranked multilingual models, tuned for Asian languages like Mandarin, Japanese, Indonesian, Thai, Malay, Vietnamese. This blog posts kicks off a monthly roundup of the most recently added models to this collection, spotlighting notable metrics or features that may be relevant to your specific application development requirements.
We added 20+ models to the Hugging Face collection in the Azure AI model catalog in July. These included multilingual models (focus on Chinese, Dutch, Arabic, South-East Asian), embedding models, text generation (SLM and LLM) and models with a domain-specific focus (e.g., biomedical). The table below summarizes additions by task and notable features. Click model name to view related model cards on Azure AI for more details. In the next section, we’ll put the spotlight on a couple of models or model families that may be of particular interest to developers exploring SLMs or multilingual applications.
Multilingual models tap into the power of Large Language Models, but with support for queries created in multiple languages (beyond just English). This extends your application reach to new audiences and unlocks your ability to drive region-focused domain-specific solutions. Let’s look at some notable additions from the list above in more detail.
The Yi series of models are large language models trained from scratch by 01-AI to support bilingual (English, Chinese) text generation tasks, and are considered one of the strongest LLMs for language understanding, commonsense reasoning, reading comprehension and more. The model comes in three sizes - 5B (for personal use), 9B (for coding and math) and 34B (for personal, academic and commercial use) with both base and fine-tuned (chat) model options. We added the 01-ai/Yi-34B-Chat model to the catalog in July – here’s a look at how it performs against comparable LLMs on popular benchmarks.
The base model ranked first among all existing open-source models in both English and Chinese benchmarks. The chat model outperformed other LLMs (except GPT-4-Turbo) on the AlpacaEval Leaderboard in January 2024. The model performance was evaluated with zero-shot and few-shot prompting – ranking highly on most benchmarks, making it a proficient model for bilingual conversational use.
The Yi-1.5 series of models is an upgraded version of Yi with all those capabilities and a much stronger performance in coding, math, reasoning and instruction-following capabilities thanks to being continuously pre-trained on a high-quality corpus (500B tokens) and fine-tune on 3M diverse samples. The 01-ai/Yi-1.5-34B model was added to the Azure AI catalog in July. Based on the provided data, the model achieved parity with, or outperformed, comparable models across multiple popular benchmarks.
Sailor is a suite of Open Language Models tailored for South-East Asia (SEA) built from the Qwen 1.5 model which performs well for these languages. The models are designed to understand and generate text for the diverse languages dominating the SEA region.
The base models are fine-tuned with open-source datasets to get instruction-tuned Sailor-chat model variants. In July, we added the sail/Sailor-0.5B base model and the sail/Sailor-1.8B-Chat chat model to the Hugging Face collection in the Azure AI model catalog.
Sailor is continually pretrained on 200B-400B tokens over 7 languages (Indonesian, Vietnamese, Thai, Malay, Lao, English and Chinese) using well-known publicly-available corpus with aggressive de-duplication and data cleaning to create a high-quality dataset. The models are then evaluated 4 core tasks – using these well-known benchmarks.
The results shown below are taken from their website, with the highlighted row reflected the base model we just added to the catalog. The evaluation reports the 3-shot Exact Match (EM) for prompts provided in the local languages. Baseline models outperforming Sailor models are highlighted in green – with results showing this family’s proficiency in SEA languages across all four task categories. Note: Sailor models are licensed for research and commercial use - you should see the Qwen 1.5 license for exceptions.
Getting started with using Hugging Face models in Azure involves three steps:
Samples are available for core tasks – for instance this sample shown above walks you through the end-to-end process for a text generation task with streaming support. Simply replace the default model with a different Hugging Face model from the Azure AI catalog, and run.
Want to see a different Hugging Face hub model on the Azure AI catalog? Request it by visiting that model’s page on the Hugging Face hub. Click the “Azure ML” option under the Deploy dropdown (top right on page). If that model is not currently in our catalog (e.g., SmolLM-135M) you will see the “Request to add” option as shown. If it was already added to the catalog (e.g., SmolLM-1.7B) you will instead get a `Go to model in AzureML` option.
As we announced in May 2024, we will deepen Hugging Face hub and Azure AI platform integrations with model discoverability, custom deployment, and fine-tuning in focus. Here are a few actions you can take today to help us improve this developer experience.
Continue reading...
What’s New in July 2024
We added 20+ models to the Hugging Face collection in the Azure AI model catalog in July. These included multilingual models (focus on Chinese, Dutch, Arabic, South-East Asian), embedding models, text generation (SLM and LLM) and models with a domain-specific focus (e.g., biomedical). The table below summarizes additions by task and notable features. Click model name to view related model cards on Azure AI for more details. In the next section, we’ll put the spotlight on a couple of models or model families that may be of particular interest to developers exploring SLMs or multilingual applications.
| | Model Name · Inference Task | Notable Features |
| 01 | 01-ai/Yi-1.5-34B ﹒ Text Generation | Top-ranked bilingual language model (with English, Chinese) |
| 02 | 01-ai/Yi-34B-Chat ﹒ Text Generation | |
| 03 | BramVanroy/GEITje-7B-ultra ﹒ Text Generation | Bilingual (with English, Dutch) |
| 04 | BAAI/bge-m3﹒ Sentence Similarity | Multi-lingual, Multi-functional RAG |
| 05 | stanford-crfm/BioMedLM ﹒ Text Generation | Biomedical, MedQA (not for prod) |
| 06 | BioMistral/BioMistral-7B ﹒ Text Generation | Biomedical, MedQA (from Mistral) |
| 07 | m42-health/Llama3-Med42-70B ﹒ Text Generation | Clinical, MedQA (not for prod) |
| 08 | m42-health/Llama3-Med42-8B ﹒ Text Generation | Clinical, MedQA (not for prod) |
| 09 | shenzhi-wang/Llama3.1-8B-Chinese-Chat | Bilingual (with English, Chinese) |
| 10 | HuggingFaceTB/SmolLM-1.7B ﹒ Text Generation | SLM, high-quality training corpus |
| 11 | AI-MO/NuminaMath-7B-TIR ﹒ Text Generation | 1st progress prize - AI Math Olympiad |
| 12 | google-t5/t5-base ﹒Translation | Reframe NLP Tasks with one format |
| 13 | jbochi/madlad400-3b-mt ﹒ Translation | Multilingual (450 lang), uses T5 arch. |
| 14 | teknium/OpenHermes-2.5-Mistral-7B ﹒ Text Generation | Tackle complex conversation topics |
| 15 | Intel/neural-chat-7b-v3-1﹒ Text Generation | SlimOrca data, Intel Gaudi fine-tuned |
| 16 | hfl/chinese-llama-2-7b ﹒ Text Generation | Optimized for Chinese vocab |
| 17 | shibing624/mengzi-t5-base-chinese-correction ﹒ T2TG | T5 for Chinese Spelling Correction |
| 18 | GeneZC/MiniChat-1.5-3B ﹒ Text Generation | Fine-tuned Llama2-7B (outperforms) |
| 19 | sail/Sailor-0.5B ﹒ Text Generation | Tuned for SEA (South-East Asian lang) Indonesian, Thai, Malay, Vietnamese- |
| 20 | sail/Sailor-1.8B-Chat ﹒ Text Generation | |
| 21 | FreedomIntelligence/AceGPT-v1.5-13B ﹒ Text Generation | AceGPT family - Arabic lang domain |
Spotlight On: Multilingual Models
Multilingual models tap into the power of Large Language Models, but with support for queries created in multiple languages (beyond just English). This extends your application reach to new audiences and unlocks your ability to drive region-focused domain-specific solutions. Let’s look at some notable additions from the list above in more detail.
1. Yi Series by 01.AI
The information below is summarized from these model creator resources:
Model Website · Technical Report · Yi Series Models (GitHub) · Cookbook
Model Website · Technical Report · Yi Series Models (GitHub) · Cookbook
The Yi series of models are large language models trained from scratch by 01-AI to support bilingual (English, Chinese) text generation tasks, and are considered one of the strongest LLMs for language understanding, commonsense reasoning, reading comprehension and more. The model comes in three sizes - 5B (for personal use), 9B (for coding and math) and 34B (for personal, academic and commercial use) with both base and fine-tuned (chat) model options. We added the 01-ai/Yi-34B-Chat model to the catalog in July – here’s a look at how it performs against comparable LLMs on popular benchmarks.
The base model ranked first among all existing open-source models in both English and Chinese benchmarks. The chat model outperformed other LLMs (except GPT-4-Turbo) on the AlpacaEval Leaderboard in January 2024. The model performance was evaluated with zero-shot and few-shot prompting – ranking highly on most benchmarks, making it a proficient model for bilingual conversational use.
The Yi-1.5 series of models is an upgraded version of Yi with all those capabilities and a much stronger performance in coding, math, reasoning and instruction-following capabilities thanks to being continuously pre-trained on a high-quality corpus (500B tokens) and fine-tune on 3M diverse samples. The 01-ai/Yi-1.5-34B model was added to the Azure AI catalog in July. Based on the provided data, the model achieved parity with, or outperformed, comparable models across multiple popular benchmarks.
2. Sailor: Open Language Models for South-East Asia
The information below is summarized from these model creator resources:
Model Website · Technical Report · Sailor Language Models
Model Website · Technical Report · Sailor Language Models
Sailor is a suite of Open Language Models tailored for South-East Asia (SEA) built from the Qwen 1.5 model which performs well for these languages. The models are designed to understand and generate text for the diverse languages dominating the SEA region.
The base models are fine-tuned with open-source datasets to get instruction-tuned Sailor-chat model variants. In July, we added the sail/Sailor-0.5B base model and the sail/Sailor-1.8B-Chat chat model to the Hugging Face collection in the Azure AI model catalog.
Sailor is continually pretrained on 200B-400B tokens over 7 languages (Indonesian, Vietnamese, Thai, Malay, Lao, English and Chinese) using well-known publicly-available corpus with aggressive de-duplication and data cleaning to create a high-quality dataset. The models are then evaluated 4 core tasks – using these well-known benchmarks.
- Question Answering: XQuAD (Thai, Vietnamese) and TydiQA (Indonesian).
- Commonsense Reasoning: XCOPA (Indonesian, Thai, Vietnamese).
- Reading Comprehension: Belebele (Indonesian, Thai, Vietnamese).
- Examination: M3Exam (Javanese, Thai, Vietnamese).
The results shown below are taken from their website, with the highlighted row reflected the base model we just added to the catalog. The evaluation reports the 3-shot Exact Match (EM) for prompts provided in the local languages. Baseline models outperforming Sailor models are highlighted in green – with results showing this family’s proficiency in SEA languages across all four task categories. Note: Sailor models are licensed for research and commercial use - you should see the Qwen 1.5 license for exceptions.
Get Started Using Hugging Face Models on Azure
Getting started with using Hugging Face models in Azure involves three steps:
- Pick the right model from the catalog: Explore the Hugging Face Collection.
- Deploy the model to Azure: use Azure AI Studio or Python SDK or Azure CLI.
- Run a test inference: use Python Samples or Create your own test sample.
Samples are available for core tasks – for instance this sample shown above walks you through the end-to-end process for a text generation task with streaming support. Simply replace the default model with a different Hugging Face model from the Azure AI catalog, and run.
Request a Hugging Face Model for the Azure catalog
Want to see a different Hugging Face hub model on the Azure AI catalog? Request it by visiting that model’s page on the Hugging Face hub. Click the “Azure ML” option under the Deploy dropdown (top right on page). If that model is not currently in our catalog (e.g., SmolLM-135M) you will see the “Request to add” option as shown. If it was already added to the catalog (e.g., SmolLM-1.7B) you will instead get a `Go to model in AzureML` option.
Get Involved
As we announced in May 2024, we will deepen Hugging Face hub and Azure AI platform integrations with model discoverability, custom deployment, and fine-tuning in focus. Here are a few actions you can take today to help us improve this developer experience.
- Want to join a private preview program for these features? Complete this form.
- Want to try inference using these models, code-first? Explore the Python samples.
- Want to explore the Hugging Face models in Azure? Browse the Model catalog.
- Want to learn more about the Azure AI catalog & usage? Read the Documentation.
Continue reading...