Automate Markdown and Image Translations Using Co-op Translator: Phi-3 Cookbook Case Study

[Minseok_Song]

Automate Markdown and Image Translations Using Co-op Translator: Phi-3 Cookbook Case Study​

​

Overview​

In today's global environment, it's important to make technical content accessible to a wider audience. This is especially true for open source projects and technical documentation. In this blog post, I'll introduce you to Co-op Translator, an open source tool designed to automate multilingual translations for your projects.

I'll show you how I used the translation tool to effortlessly translate the Phi-3CookBook which is an official open source guide for Phi-3 / 3.5 small language model. You'll be able to follow the same process for your own projects, streamlining the translation of even large technical resources.

​

What is Co-op Translator?​

Co-op Translator is an open source tool designed to automate the translation of Markdown files and images containing embedded text into multiple languages. Powered by Azure AI Services, it streamlines the traditionally time-consuming translation process, allowing you to make your projects globally accessible with minimal manual effort.



Below is the architecture of this project:





 

 



The process begins with markdown and image files from your project folder, which are then processed by Azure AI Services. Azure OpenAI handles text translations from the markdown files, while Azure Computer Vision extracts text from the images. Once the text is extracted, Azure OpenAI translates the text from the images. The final translated files - both markdown and images - are saved in the designated translation folder, ready for use in multiple languages.

​

Key features​

• Translates both Markdown files and text within images.
• Supports multiple languages simultaneously.
• Leverages Azure Computer Vision and Azure OpenAI for high-quality translations.
• Can be easily integrated into your existing workflows.

​

Table of contents​

1. Phi-3 Cookbook translation: A case study
2. Set up Azure resources
3. Create the .env file in the root directory
4. Install the Co-op Translator package
5. Translate your project using Co-op Translator
6. Conclusion: Making your projects global

Phi-3 Cookbook translation: A case study​

The Phi-3 Cookbook is an official open-source guide that provides detailed instructions on Phi-3 and Phi-3.5 small language models. Given its technical nature and importance to the global AI community, translating it into multiple languages was a crucial step toward making this valuable resource accessible to non-English-speaking developers and researchers.

By using Co-op Translator, I was able to streamline the translation process, automating the conversion of both Markdown files and images containing embedded text into several languages. This case study explains how the tool was applied to the Phi-3 Cookbook, the challenges encountered, and the solutions implemented.

The translation process​

1. 
   Preparation: The first step was to organize the markdown files and image assets from the Phi-3 Cookbook. I removed the previously manually translated files to ensure a clean slate for the automated process, as leaving them would result in repeated translations. I also created an .env file in the root directory to securely store the necessary Azure API keys and configuration settings.
   
   
2. 
   Azure Setup: I configured Azure OpenAI to handle the markdown text translations and Azure Computer Vision for extracting and translating text from images. This setup allowed the tool to automatically detect and process both types of content seamlessly.
   
   
3. 
   Installing Co-op Translator: I installed the Co-op Translator package using Poetry to manage the dependencies. After installing, I ran the translate command with appropriate language codes to initiate the translation process.
   
   
4. 
   Execution with Co-op Translator: Using Co-op Translator, I initiated the translation process:
   
   • The markdown files were processed through Azure OpenAI for translation.
   • Azure Computer Vision was used to extract text from images, followed by Azure OpenAI for translating the extracted text.
   • The translated markdown and image files were saved in dedicated language-specific folders.
5. 
   Review: After the translations were completed, I reviewed the output for accuracy. The automated process produced high-quality translations, significantly reducing the need for manual adjustments.
   
   
   Note
   Once the translations are done, you'll find the translations and translated_images folders in the root directory. You can see an example of the folder structure in the Phi-3 Cookbook:
   
   
   
   
   
   
   
   

Challenges and Solutions​

While the process was mostly smooth, a few challenges arose:

• 
  Handling files with many code blocks: Some markdown files contained many code blocks. During translation, the tool splits the content into chunks, and if a split occurred within a code block, the translated output would sometimes break. To resolve this, we implemented a solution where code blocks are temporarily replaced with placeholders and skipped during translation, ensuring the integrity of both the code and the translation. In future versions, we plan to enhance this feature by translating comments within code blocks. This will involve separating code blocks from the rest of the markdown, accurately translating each part, and then reintegrating them into the final translated file.
  
  
• 
  Text from images: Extracting and translating text from complex images was challenging, especially when the images had a lot of text in small areas. This often led to the translated text in image being either overly stretched or compressed, affecting readability. While we managed to mitigate some of these issues, further improvements are needed in this area to ensure higher accuracy.
  

​

Results​

The translation of the Phi-3 Cookbook has been successful. You can now view the translated version in multiple languages. If you're interested in viewing the translated Phi-3 Cookbook, you can visit the Phi-3CookBook. This link will navigate you to the multilingual versions of the Phi-3 Cookbook. 

Below is a markdown example from Phi-3 Cookbook, translated into Korean using the Co-op Translator:

Set up Azure resources​

To translate markdown files and extract text from images, we need to set up two key Azure services: Azure OpenAI and Azure Computer Vision.

Create an Azure account​

If you don't already have an Azure account, you'll need to create one.

1. Navigate to the Azure Sign Up page.
2. Select Try Azure for free or Pay as you go.
3. Follow the on-screen instructions to create your account.
   • Provide your personal details and contact information.
   • Verification: You'll need to verify your identity using a credit card or phone number.

​

Create an Azure Computer Vision resource​

1. 
   Sign in to the Azure Portal.
   
   
2. 
   Type computer vision in the search bar at the top of the portal page and select Computer vision from the options that appear.
   
   
   
    
   
   
3. 
   Select + Create from the navigation menu.
   
   
   
   
   
4. 
   Perform the following tasks:
   
   • Select your Azure Subscription.
   • Select the Resource group to use (create a new one if needed).
   • Select the Region you'd like to use.
   • Enter Name. It must be a unique value.
   • Select the Pricing tier you'd like to use.
   
    
   
   
    
   
   
5. 
   Select Review + Create.
   
   
6. 
   Select Create.
   

​

Create an Azure OpenAI resource​

 ​

1. 
   Type azure openai in the search bar at the top of the portal page and select Azure OpenAI from the options that appear.
   
   
   
    
   
   
2. 
   Select + Create from the navigation menu. 
   
   
   
3. 
   Perform the following tasks:
   
   • Select your Azure Subscription.
   • Select the Resource group to use (create a new one if needed).
   • Select the Region you'd like to use.
   • Enter Name. It must be a unique value.
   • Select the Pricing tier you'd like to use.
     
   
4. 
   Select Next to move to the Network page.
   
   
5. 
   Select a network security Type you'd like to use.
   
   
   
    
   
   
6. 
   Select Next to move to the Tags page.
   
   
7. 
   Select Next to move to the Review + submit page.
   
   
8. 
   Select Create.
   
   
   
   
   

Deploy Azure OpenAI models​

1. 
   Navigate to the Azure OpenAI resource that you created.
   
   
2. 
   Select Go to Azure OpenAI Studio from the navigation menu.
   
   
   
   
   
   
3. 
   Inside Azure OpenAI Studio, select Deployments from the left side tab.
   
   
4. 
   Select + Deploy model from the navigation menu.
   
   
5. 
   Select Deploy base model from the navigation menu to create a new gpt-4o deployment.
   
   
   
   
   
   
6. 
   Perform the following tasks:
   
   • Inside Select a model page, select gpt-4o.
   • Select Confirm to navigate to the Deploy model gpt-4o page.
   • Inside Deploy model gpt-4o page, enter Deployment name. It must be a unique value. For example, gpt-4o.
   • Inside Deploy model gpt-4o page, select the Deployment type you'd like to use.
     
   
   
   
   
7. 
   Select Deploy.
   

Create an .env file in the root directory​

In this tutorial, we will guide you through setting up your environment variables for Azure services using an .env file. Environment variables allow you to securely manage sensitive credentials, such as API keys, without hard-coding them into your codebase.

​

Create the .env File​

In the root directory of your project, create a file named .env. This file will store all your environment variables in a simple format.



Warning
Do not commit your .env file to version control systems like Git. Add .env to your .gitignore file to prevent accidental commits.

1. 
   Navigate to the root directory of your project.
   
   
2. 
   Create an .env file in the root directory of your project.
   
   
   
   
   
   
3. 
   Open the .env file and paste the following template:
   
   
   
   
   

   # Azure Credentials
   AZURE_SUBSCRIPTION_KEY="your_azure_subscription_key"
   AZURE_AI_SERVICE_ENDPOINT="https://your_azure_ai_service_endpoint"
   
   # Azure OpenAI Credentials
   AZURE_OPENAI_API_KEY="your_azure_openai_api_key"
   AZURE_OPENAI_ENDPOINT="https://your_azure_openai_endpoint"
   AZURE_OPENAI_MODEL_NAME="your_model_name"
   AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="your_deployment_name"
   AZURE_OPENAI_API_VERSION="your_api_version"

   
   
   

​

Gather your Azure credentials​

You will need the following Azure credentials on hand to configure the environment:

1. 
   For Azure AI Service:
   
   • Azure Subscription Key: Your Azure subscription key, which allows you to access the Azure AI services.
   • Azure AI Service Endpoint: The endpoint URL for your specific Azure AI service.
2. 
   For Azure OpenAI Service:
   
   • Azure OpenAI API Key: The API key for accessing Azure OpenAI services.
   • Azure OpenAI Endpoint: The endpoint URL for your Azure OpenAI service.
   • Azure OpenAI Model Name: The name of the model you will be interacting with.
   • Azure OpenAI Deployment Name: The name of your deployment for Azure OpenAI models.
   • Azure OpenAI API Version: The version of the Azure OpenAI API you are using.

​

Add Azure environment variables​

1. 
   Perform the following tasks to add the Azure Subscription key and Azure AI Services Endpoint:
   
   • Type computer vision in the search bar at the top of the portal page and select Computer vision from the options that appear.
     
     
     
      
     
     
   • Navigate the Azure Computer Vision resource that you are currently using.
   • Copy and paste your Subscription key and Endpoint into the .env file.
     
     
     
     
2. 
   Perform the following tasks to add the Azure OpenAI API Key and Endpoint:
   
   • Type azure openai in the search bar at the top of the portal page and select Azure OpenAI from the options that appear.
     
     
     
     
     
   • Navigate the Azure OpenAI resource that you are currently using.
   • Select Keys and Endpoint from the left side tab.
   • Copy and paste your Azure OpenAI API Key and Endpoint into the .env file.
     
      
     
3. 
   Perform the following tasks to add the Azure OpenAI Deployment Name and Version:
   
   • 
     Navigate to the Azure OpenAI resource that you created.
     
     
   • 
     Select Go to Azure OpenAI Studio from the navigation menu.
     
     
     
     
     
     
   • 
     Inside Azure OpenAI Studio, select Deployments from the left side tab.
     
     
   • 
     Copy and paste your Azure OpenAI Name and model Version into the .env file.
     
     
     
     
     
4. 
   Save the .env file.
   
   
5. 
   Now, you can access these environment variables to use Co-op Translator with your Azure services.
   

Install the Co-op translator package​

The Co-op Translator is a command-line interface (CLI) tool designed to help you translate all the markdown files and images in your project into multiple languages. This tutorial will guide you through configuring the translator and running it for various use cases.

​

Create a virtual environment​

You can create a virtual environment using either pip or Poetry. Type one of the following commands inside your terminal.

​

Using pip​

python -m venv .venv

​

Using Poetry​

poetry init

​

Activate the virtual environment​

After creating the virtual environment, you'll need to activate it. The steps differ based on your operating system. Type the following command inside your terminal.

For both pip and Poetry​

• 
  Windows:
  
  
  

  venv\Scripts\activate.bat

• 
  Mac/Linux:
  
  
  
  
  

  source venv/bin/activate

  
  
  

​

Using Poetry​

1. 
   If you created the environment with Poetry, type the following command inside your terminal to activate it.
   
   
   
   

   poetry shell

​

Installing the Package and required Packages​

Once your virtual environment is set up and activated, the next step is to install the necessary dependencies.

​

Using Poetry (from pyproject.toml)​

1. 
   If you're using Poetry, type the following command inside your terminal. It will automatically install the required packages specified in the pyproject.toml file:
   
   
   
   
   
   poetry install
   
   
   

Translate your project using Co-op Translator​

The Co-op Translator is a command-line interface (CLI) tool that helps you translate markdown and image files in your project into multiple languages. This section explains how to use the tool, covers the various CLI options, and provides examples for different use cases.

​

CLI options overview​

The Co-op Translator CLI offers several options to customize the translation process:

• -l (or --language-codes): Space-separated list of language codes for translation (e.g., "es fr de" for Spanish, French, and German). Use "all" to translate into all supported languages.
• -r (or --root-dir): Specifies the root directory of the project (default is the current directory).
• -a (or --add): Adds new translations without deleting existing ones (default behavior).
• -u (or --update): Updates translations by deleting existing ones and re-creating them. Warning: This will delete all current translations.
• -img (or --images): Translates only image files.
• -md (or --markdown): Translates only markdown files.
• -chk (or --check): Checks translated files for errors and retries translation if needed.
• -d (or --debug): Enables debug mode for detailed logging.

---

​

Example scenarios and commands​

Here are a few common use cases for the Co-op Translator, along with the appropriate commands to run.

​

1. Basic translation (Single language)​

To translate your entire project (markdown files and images) into a single language, like Korean, use the following command:

translate -l "ko"

This command will translate all markdown and image files into Korean, adding new translations without deleting any existing ones.

​

Tip
Want to see which language codes are available in Co-op Translator? Visit the Supported Languages section in the repository for more details.

Example on Phi-3 CookBook​

In the Phi-3 CookBook, I used the following method to add the Korean translation for the existing markdown files and images.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko"
Translating images: 100%|████████████████████████████████████████████| 150/150 [45:53<00:00, 15.55s/it]
Translating markdown files: 100%|███████████████████████████████████| 95/95 [1:39:27<00:00, 125.62s/it]

​

2. Translating multiple languages​

To translate your project into multiple languages (e.g., Spanish, French, and German), use this command:

translate -l "es fr de"

This command will translate the project into Spanish, French, and German, adding new translations without overwriting existing ones.

Example on Phi-3 CookBook​

In the Phi-3 CookBook, after pulling the latest changes to reflect the most recent commits, I used the following method to translate newly added markdown files and images.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko ja zh tw es fr" -a
Translating images: 100%|███████████████████████████████████████████████████| 273/273 [1:09:56<00:00, 15.37s/it]
Translating markdown files: 100%|████████████████████████████████████████████████| 6/6 [24:07<00:00, 241.31s/it]

Note
While it's generally recommended to translate one language at a time, in situations like this where specific changes need to be added, translating multiple languages at once can be efficient.

​

3. Specifying the root directory​

By default, the translator uses the current working directory. If your project is located elsewhere, specify the root directory with the -r option:

translate -l "es fr de" -r "./my_project"

This command translates the files in ./my_project into Spanish, French, and German.

​

4. Add new translations without deleting existing ones​

The default behavior is to add new translations without deleting existing ones. You can explicitly specify this by using the -a option:

translate -l "ko" -a

This command will add new translations in Korean without affecting the existing translations.

​

Example on Phi-3 CookBook​

In the Phi-3 CookBook, to update the README.md translations, I first deleted the existing README.md translations and then used the following method to translate the updated content.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko ja zh tw es fr" -a
Translating markdown files: 100%|████████████████████████████████████████████████| 6/6 [24:07<00:00, 241.31s/it]

​

5. Updating translations (Deletes existing translations)​

To update existing translations (i.e., delete the current translations and replace them with new ones), use the -u option. This will delete all existing translations for the specified languages and re-translate them.

translate -l "ko" -u

Warning: This command will prompt you for confirmation before proceeding with deleting the existing translations.

​

Example on Phi-3 CookBook​

In the Phi-3 CookBook, I used the following method to update all translated files in Spanish. I recommend using this method when there are significant changes to the original content across multiple markdown documents. If there are only a few translated markdown files to update, it’s more efficient to manually delete those specific files and then use the -a method to add the updated translations.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l "es" -u
Warning: The update command will delete all existing translations for 'es' and re-translate everything.
Do you want to continue? Type 'yes' to proceed: yes
Proceeding with update...
Translating images: 100%|████████████████████████████████████████████| 150/150 [43:46<00:00, 15.55s/it]
Translating markdown files: 100%|███████████████████████████████████| 95/95 [1:40:27<00:00, 125.62s/it]

​

6. Translating only images​

To translate only the image files in your project, use the -img option:

translate -l "ko" -img

This command will translate only the images into Korean, without affecting any markdown files.

​

7. Translating only markdown files​

To translate only the markdown files in your project, use the -md option:

translate -l "ko" -md

​

8. Checking for errors in translated files​

If you want to check translated files for errors and retry the translation if necessary, use the -chk option:

translate -l "ko" -chk

This command will scan the translated markdown files and retry translation for any files with errors.

​

Example on Phi-3 CookBook​

In the Phi-3 CookBook, I used the following method to check for translation errors in the Korean files and automatically retry translation for any files with detected issues.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l"ko" -chk 
Checking translated files for errors in ko...
Checking files for ko: 100%|██████████████████████████████████████████████████| 95/95 [00:01<00:00, 65.47file/s]
Retrying vsc-extension-quickstart.md for ko:   0%|                                     | 0/17 [00:00<?, ?file/s]

This option checks for translation errors. Currently, if the difference in line breaks between the original and translated files is more than six, the file is flagged as having a translation error. I plan to improve this criterion for greater flexibility in the future.

For example, this method is useful for detecting missing chunks or corrupted translations, and it will automatically retry the translation for those files.

However, if you already know which files are problematic, it’s more efficient to manually delete those files and use the -a option to re-translate them.

9. Debug mode​

To enable detailed logging for troubleshooting, use the -d option:

translate -l "ko" -d

This command will run the translation in debug mode, providing additional logging information that can help you identify issues during the translation process.

Example on Phi-3 CookBook​

In the Phi-3 CookBook, I encountered an issue where translations with many links in markdown files caused formatting errors, such as broken translations and ignored line breaks. To diagnose this problem, I used the -d option to see how the translation process was works.

(.venv) C:\Users\sms79\dev\Phi-3CookBook>translate -l "ko" -d
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/chat/completions', 'headers': {'api-key': 'af04e0bea45747d8a7b8c131c1971044'}, 'files': None, 'json_data': {'messages': [{'role': 'user', 'content': "Translate the following text to ko. NEVER ADD ANY EXTRA CONTENT OUTSIDE THE TRANSLATION. TRANSLATE ONLY WHAT IS GIVEN TO YOU.. MAINTAIN MARKDOWN FORMAT\n\n# Phi-3 Cookbook: Hands-On Examples with Microsoft's Phi-3 Models [![Open and use the samples in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/microsoft/phi-3cookbook) [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%
...

​

10. Translating all languages​

If you want to translate the project into all supported languages, use the all keyword.



Warning
Translating all languages at once can take a significant amount of time depending on the project size. For example, translating the Phi-3 CookBook into Spanish took about 2 hour. Given the scale, it's not practical for one person to handle 20 languages. It's recommended to split the work among multiple contributors, each managing one or two languages, and update translations gradually.

translate -l "all"

This command will translate the project into all available languages. If you proceed, the translation may take a significant amount of time depending on the size of the project.

Conclusion: Making your projects global​

How can large-scale projects efficiently reach a global audience without getting bogged down in the complexities of translation? The Co-op Translator provides a compelling answer.





 

 

Based on my experience, here are a few key takeaways:

• 
  Time Management: Translating a large project can take considerable time. For example, translating the Phi-3 Cookbook into Spanish took about 2 hours. It’s important to account for this when working on large-scale translations to avoid being caught off guard by the time required. However, since this translation process only adds translated markdown or images once the translation is complete, you can pause the process at any time. Simply press Ctrl + C (or Cmd + C on macOS) at any time to stop the process, and when you're ready to resume, use the -a option to continue the translation from where you left off.
  
  
• 
  Team Collaboration: If you're part of a team, dividing the workload by assigning each member a specific language can help streamline the process. Co-op Translator's ability to easily add and update translations ensures smooth collaboration, allowing everyone to work without overwriting each other's progress. Or, if you're managing an open source project, I recommend encouraging contributors to use this tool to translate content into their respective languages, making the project more accessible to a global audience.
  

The Phi-3 Cookbook translation stands as a great example of how Co-op Translator can automate the localization process, making your project accessible to a global audience with minimal effort. Whether you're translating Markdown files or images, Co-op Translator simplifies the task, handling the complexities of translation seamlessly.

​

Clean Up Azure Resources​

Cleanup your Azure resources to avoid additional charges to your account. Go to the Azure portal and delete the following resources:

• The Azure Computer Vision resource.
• The Azure OpenAI resource

If you're interested in trying out Co-op Translator or contributing to its development, check out the Co-op Translator's GitHub repository. I'd love to hear your thoughts and experiences - feel free to contact me with any questions or feedback!

The Co-op Translator started as a proof of concept but has grown into a fully functional open source tool, thanks to community collaboration. Now, you can be part of this journey too!

• Global Impact: Help us make this tool accessible worldwide
• Contribute: Add new features, improve docs, or translate
• Join the Team: Become a collaborator in our next release

Contribute to Co-op Translator​

Your support will help bring translations to developers across the globe.

Explore on GitHub​

Continue reading...