Business-in-a-box: Applying AutoGen and multi-agent systems to an enterprise context

[Guest LucaStamatescu]

Contributors: Priya Kedia, Shikha Agrawal, Nyuk Ching Lee, Ming Gu, Sunil Sattiraju, Naveen Gopalakrishna, Scott Holden, Daniel Csillag

 

 

 

Generative AI (GenAI) is revolutionizing how businesses engage with customers and streamline internal processes. However, scaling these innovations beyond a handful of use cases remains a significant challenge. Typically, enterprises build separate applications for different use cases— for example, a HR chatbot for the HR team, a legal advice RAG application for the legal team, and a customer service assistant for their frontline teams—each requiring its own knowledge base and tools. While a shared platform can be built, maximizing reuse is difficult, and each new application demands its own business case.

 

 

 

We believe the answer lies in agentic systems, where multiple agents collaborate to achieve a task. Instead of developing numerous standalone applications, you can build a single capability that unlocks potentially unlimited use cases. In this article, we will demonstrate how to apply multi-agent frameworks like AutoGen to various enterprise scenarios, to inspire you to implement this in your organization.

 

 

 

[HEADING=1]An overview of the Business-in-a-Box[/HEADING]

 

The Business-in-a-Box is comprised of three interfaces:

 

1. User Interface: Enables users to interact with the system and assign tasks to the agents.
   
2. Audit Interface: Allows you to observe how agents collaborate, each with specific domain knowledge and tools.
   
3. Manager Interface: Facilitates human-in-the-loop supervision, where a human supervisor can guide agents if they encounter obstacles or need approvals.
   

 

The agents in the Business-in-a-Box

 

There are many agents inside the box, spanning marketing, HR, procurement, technology, invoice auditing and more. Each of these agents has their own specialized set of knowledge and tools, making them experts in their domain.

 

 

 

[HEADING=1]Three levels of GenAI systems[/HEADING]

 

There are three key levels of interactive GenAI systems: chatbots, single agents and multi-agent systems.

 

 

 

[HEADING=2]Level 1: Informational Chatbots[/HEADING]

 

Retrieval Augmented Generation (RAG) chatbots are among the most common GenAI use cases. They enable Large Language Models (LLMs) to answer questions using your organization’s data.

 

 

 

 

Video overview: In this video, we ask the Business-in-a-Box “What is the dress code?” The HR Agent retrieves your organization’s specific policies from a knowledge base (e.g., Azure AI Search) and answers the question.

 

 

 

[HEADING=2]Level 2: Single Agents[/HEADING]

 

Single agent systems go beyond simple informational responses, possessing the ability to impact the world around them by automating tasks, delivering significantly more value.

 

 

 

 

Video overview: we ask the Business-in-a-Box to “Issue a badge.” The Procurement Agent identifies that human approval is required before proceeding. The human supervisor receives a message from the Procurement Agent, and approves the request. The Procurement Agent then issues the badge, which is mailed to the employee’s nearest office location.

 

 

 

[HEADING=2]Level 3: Multi-Agent Systems[/HEADING]

 

Multi-agent systems involve specialized agents collaborating to solve complex problems. Each agent is an expert in a specific domain. Similar to humans, no one agent can do everything, but together, they can achieve significant outcomes. The way these agents collaborate mirrors how people collaborate, as part of cross-functional teams, where each person has their own expertise.

 

 

 

 

 

Video overview: onboarding a new employee involves multiple areas of the business. The HR Agent identifies key onboarding steps, while the Procurement and Tech Agents complete their respective tasks such as setting up an Office 365 account or ordering a laptop.

 

 

 

[HEADING=1]So what?[/HEADING]

 

At this point, you may be wondering- couldn't we have automated this with a simple workflow? The power of multi-agent systems becomes evident, when you combine agents in different ways to solve use cases that you never anticipated.

 

 

 

By building one capability, you can solve almost an unlimited number of use cases. Here are a few examples of different ways these agents could be combined, to solve other problems:

 

By combining agents in different ways, one capability can unlock almost unlimited use cases.

 

 

 

Let’s make this real by exploring another use case.

 

 

 

 

 

 

Video overview: we ask the system to share updates on a new product release and obtain legal approval. The Product Agent provides updates, the Marketing Agent drafts the press release, and the Legal Agent obtains sign-off. The key takeaway is that one capability enables almost unlimited use cases.

 

 

 

This requires a different set of agents to work together to complete the task. Whilst the use case is interesting, the key takeaway is that one capability enables almost unlimited use cases.

 

 

 

[HEADING=1]An organizational shift[/HEADING]

 

We can further mirror human collaboration by organizing agents to align with an organization's existing structure. For instance, placing the Legal Agent in the legal business unit ensures the team closest to the information maintains its accuracy and relevance. The legal team then owns the roadmap for the legal agent, adding new information and capabilities to their agent. The Legal Agent can also collaborate directly with the lawyers, reaching out for approvals or guidance when necessary.

 

 

 

Agents can be arranged to mirror human organizations

 

 

 

[HEADING=1]Responsible AI[/HEADING]

 

It is important to consider the impacts of these types of systems and the automation they drive. The focus should be on using these systems to empower people by managing the overhead of coordinating tasks across a large enterprise and automating simple tasks, therefore allowing individuals to focus on leveraging their unique skills.

 

Let's consider the example of a legal team, who support the business by reviewing contracts and providing guidance. New work is accepted in the form of a brief that follows a set template. Often these templates are not complete or unclear in their intent, causing the legal team to spend a significant amount of time requesting additional information and fixing errors.

 

 

 

Now suppose a product team wishes to update their product terms ahead of a new product launch. In this scenario, they would ask their Product Agent to achieve this goal. The Product Agent reaches out to the Legal Agent and obtains the template. It then collaborates with the product team to gather all the relevant information before providing it to the marketing team. When the brief is submitted, the Legal Agent triages it, ensures it contains all the necessary information, and places it into the legal team's work queue.

 

 

 

In this way, the legal team is empowered to focus on their specialized skillset, and spend more time assessing risks and reviewing contracts, rather than guiding the rest of the business on how to write a proper brief.

 

 

 

[HEADING=1]The network effect[/HEADING]

 

The most incredible property of this system is the impact that adding just one new component can have (for example an API function or a new set of documents to a knowledge base). This small addition can unlock potentially multiple use cases, thanks to the network effect. For example, if the lawyers expose the ability to obtain legal sign-off as a function for their agent, this might unlock ten new use cases across the business that were previously not possible. These could include writing press releases, signing supply agreements, or updating product documentation. This provides an incredible level of reuse.

 

Furthermore, new use cases become an emergent property of the system. We are no longer building use cases individually but instead taking a data-centric view of the organization. By focusing on adding new data, we unlock use cases, rather than the other way around.

 

 

 

[HEADING=1]What are the limitations of this project?[/HEADING]

 

The demonstration shown is a real, live multi-agent system capable of handling unplanned use cases and taking action in the real world. There are a number of limitations that are important to note:

 

 

 

• Placeholder Functions: The demonstration shows agents correctly triggering functions with the right parameters; actual integrations were left as future work.
  
• Small number of documents: The knowledge base has a limited number of documents, making it easier to retrieve the right information. This is similar to challenges in existing RAG solutions but could compound if multiple retrieval steps are needed.
  
• Scaling to hundreds of agents: Further research is needed to understand how systems scale beyond a few agents. In this demo, seven agents worked together to solve tasks. As more agents are added, routing requests becomes more complex. New strategies, such as hierarchical structures similar to today's organizations, may be developed. This would involve multiple groups, of multi-agent systems working together. For instance, a high-level business objective could be assigned to a "leadership team" of agents, who oversee each of the key business units. The task would then be cascaded down to another multi-agent system, representing that business unit.
  

 

While the exact scalability of these systems is not yet known, it is clear that multi-agent systems have immense potential to transform organizational operations.

 

 

 

[HEADING=1]How does it work?[/HEADING]

 

This demo uses AutoGen, a multi-agent framework by Microsoft Research. Agents are wrapped using Prompt Flow, orchestrated with Azure AI Studio and Managed Online Endpoints. The knowledge base is built using Azure AI Search, and conversation history is managed with Cosmos DB.

 

 

 

[HEADING=1]Conclusion[/HEADING]

 

Multi-agent systems represent the next wave of Generative AI use cases, offering entirely new opportunities to drive efficiencies in your business. The Business-in-a-Box demonstrates several key benefits:

 

 

 

• Allows people to focus on what matters: by doing the heavy lifting involved with coordinating activities across an organization, peoples’ time is freed up to focus on their specializations.
  
• Enabling GenAI to scale: by not needing to build one application after another, organizations are able to reduce the friction of adopting GenAI across their entire organization. One capability can unlock almost unlimited use cases.
  
• Applicable to most industries: these are common challenges that most organizations face, across most industries.
  

 

Whilst still an emerging area, investing in agentic use cases, digitatization and developing tools will be key to ensuring you are able to leverage these new technologies and seize the GenAI moment.

 

 

 

[HEADING=1]Acknowledgements[/HEADING]

 

Thank you to the incredible team who helped deliver this project.

 

 

Continue reading...