Exploring the Advanced RAG (Retrieval Augmented Generation) Service

[freistli]

In the ever-evolving landscape of AI, LLM + RAG (Retrieval Augmented Generation) is a typical use scenario. Retrieving accurate related chunked data from complicated docs and then improving LLM response quality becomes challenge. There is no a silver bullet RAG can address all requirements so far. Developers need to verify different advanced RAG techs to find out which is a proper one for their scenarios considering accuracy, response speed, costs, etc. In order to solve this, with Azure Intelligent Document, Azure OpenAI, LlamaIndex, LangChain, Gradio..., I developed this AdvancedRAG service. This service is encapsulated in a Docker container, offers a streamlined way to experiment with different indexing techniques, evaluate their accuracy, and optimize performance for various RAG use cases. Whether you're building a quick MVP, a proof of concept, or simply exploring different indexing strategies, this service provides a versatile playground.

Introduction​

The Advanced RAG AI Service enables developers to quickly verify different RAG indexing techniques, assessing both accuracy and performance. From index generation to output verification through chat mode and proofreading mode, this service offers comprehensive support. It can run locally in a Docker container or be deployed to Azure Container Apps, providing flexibility for various deployment scenarios.



Docker Deploy Repo: GitHub - freistli/AdvancedRAG

Source Code Repo: GitHub - freistli/AdvancedRAG_SVC

Key Features​

• Knowledge Graph Indexing
• Recursive Retriever Query
• Tree Mode Summarization
• Semantic Hybrid Search + Sub Query Engine with Azure AI Search
• Microsoft GraphRAG (Local Search + Global Search)

Quick Start Guide​

To get started with the Advanced RAG service, follow these steps:

Cloning the Repository​

git clone https://github.com/freistli/AdvancedRAG.git

Setting Up Environment Variables​

Rename .env_4_SC.sample to .env_4_SC and input the necessary environment variables. Note that an Azure OpenAI resource and an Azure Document Intelligence resource are required. Azure AI Search is optional unless you plan to build or use an Azure AI Search index.

​

Building and Running the Docker Image​

Build the Docker image:

docker build -t docaidemo .

Run the image locally:

docker run -p 8000:8000 docaidemo

Access the service at [URL]http://localhost:8000[/URL].

​

Running on Azure Container App​

1. Publish your Docker image to Azure Container Registry or Docker Hub.
2. Create an Azure Container App, select the published Docker image, and deploy the revision pod without any extra commands.
3. Set environment variables in the Azure Container App if you didn't include .env_4_SC in the image.

​

Building an Index​

1. Click the "Index Build" tab.
2. Upload a file to the file section. (PDF format is recommended for complex content.)
3. Click "Submit" and wait for the index building to complete. Status updates will appear in real-time on the right pane.

4. Once you see the completion message, you can download the index for local use.

2024-06-13T10:04:54.120027: Index is persisted in /tmp/index_cache/yourfilename
/tmp/index_cache/yourfilename can be used as your Index name.

​

Setup Your Own Index in the Docker Image​

"rules" Index Name is predefined for Knowledge Graph Index of Japanese proofread demo in this solution. Developers can use their own indexes in other folders for the docker:

To make it work:

1. Move to the folder which contains the AdvancedRAG dockerfile
2. Create a folder to keep the index, for example, index123
3. Extract the index zip file you get from the step 6 in the "Build Index" section, save index files you downloaded into ./index123
4. Build the docker image again.

After this, you can use index123 as index name in the Chat mode.

​

Using the Advanced RAG Service​

Calling the Service through REST API​

• Endpoint: https://{BASEURL}/proofreadaddin/run/predict
• Method: POST
• Header: Content-Type: application/json
• Sample Data:

{
"data": [ 
"今回は半導体製造装置セクターの最近の動きを分析します。" , <---- Proofread Content
"False" <--- Streaming
]
}

• Endpoint: https://{BASEURL}/advchatbot/run/chat
• Method: POST
• Header: Content-Type: application/json
• Sample Data:

{
  "data": [
    "When did the Author convince his farther",  <----- Prompt
    "", <--- History Object, don't change it
    "Azure AI Search",  <------ Index Type
    "azuresearch_0",   <------- Index Name or Folder
    "You are a friendly AI Assistant"    <----- System Message
  ]
}

Consuming the Service REST API in Copilot Studio Power Autoflow Action​

Consuming the Service in Office Add-In​

For the Proofread Addin use case, refer to: Proofread Addin.

​

Chat Mode​

1. Click "Chat Mode."
2. Choose the index type.
3. Enter the index path or Azure AI Search Index name in the "Index Name" text field.
4. Interact with the document using various system messages if needed.

​

Proofread Mode​

Proofread mode is tailored for non-English languages and requires a Knowledge Graph Index. Steps to generate this index are the same as for other indices.

​

Viewing the Knowledge Graph Index​

1. Click the "View Knowledge Graph Index" tab.
2. Enter your Knowledge Graph Index name and click "Submit."
3. After a short wait, click "Download Knowledge Graph View" to see the result.

​

Conclusion​

The Advanced RAG Service is a convenient solution for developers looking to explore and optimize retrieval augmented generation techniques. By providing a flexible, Docker-based environment, it enables rapid experimentation and deployment, making it easier to find the best indexing strategies for specific use cases, and provide REST API Endpoints. Whether you're working locally or deploying to the cloud, this service streamlines the process of developing and testing advanced AI solutions.

Continue reading...