Azure Bot Directline channel vs Directline speech channel

[meenakshiBalekar]

A comparison of two communication channels for Azure Bot Service

​

Introduction​

Azure Bot Service is a cloud platform that enables developers to create and deploy conversational agents, also known as bots, that can interact with users through various channels, such as web, mobile, or messaging applications. One of the key features of Azure Bot Service is the ability to connect your bot to multiple channels, such as Skype, Facebook Messenger, Slack, or Teams, using a single bot registration and code base.



However, not all channels offer the same level of functionality and user experience. Some channels, such as web chat or direct line, allow you to customize the look and feel of your bot, as well as the mode of communication, such as text, speech, or both. Other channels, such as Skype or Teams, have predefined user interfaces and only support text-based communication.



In this blog post, we will compare two communication channels that enable you to use speech as an input and output modality for your bot: direct line channel and direct line speech channel. We will explain what they are, how they differ, how to use them, and what are the benefits and limitations of each one.

​

What is direct line channel?​

Direct line channel is a REST API that allows you to communicate with your bot from any client application that can send and receive HTTP requests. You can use direct line channel to create your own custom user interface for your bot, or to integrate your bot with other services or applications. For example, you can use direct line channel to embed your bot in a web page, a mobile app, or a voice assistant.



Direct line channel supports both text and speech as input and output modalities for your bot. However, to use speech, you need to use additional services, such as Cognitive Services Speech SDK or Web Chat Speech Services, to handle the speech recognition and synthesis. You also need to handle the audio streaming and encoding between your client application and the speech service.

​

What is direct line speech channel?​

Direct line speech channel is a WebSocket-based API that allows you to communicate with your bot using speech as the primary input and output modality. You can use direct line speech channel to create voice-enabled user interfaces for your bot, such as voice assistants, smart speakers, or voice bots. For example, you can use direct line speech channel to connect your bot to Cortana, Alexa, or Google Assistant.



Direct line speech channel simplifies the speech integration for your bot, as it handles the speech recognition and synthesis, as well as the audio streaming and encoding, for you. You do not need to use any additional services or SDKs to use speech with your bot. You only need to use the direct line speech channel SDK for your client application, which is available for C#, JavaScript, and Java.

​

Similarities and difference of both​

Both direct line channel and direct line speech channel enable you to connect your bot to any client application that can communicate with them. They also enable you to use speech as an input and output modality for your bot, as well as to customize the user interface and the user experience of your bot.



However, there are some key differences between them, such as:

• Direct line channel supports both text and speech, while direct line speech channel only supports speech.
• Direct line channel requires additional services or SDKs to use speech, while direct line speech channel does not.
• Direct line channel uses REST API, while direct line speech channel uses WebSocket API.
• Direct line channel requires you to handle the audio streaming and encoding, while direct line speech channel does not.
• Direct line channel has more flexibility and control over the speech settings, such as the language, the voice, the rate, or the pitch, while direct line speech channel has less.
• Direct line channel has more latency and overhead for speech communication, while direct line speech channel has less.

​

How to use direct line speech channel with examples​

To use direct line speech channel with your bot, you need to follow these steps:

1. Create a bot using Azure Bot Service and enable the direct line speech channel in the bot settings. How to add directline speech channel to azure bot
2. Generate a direct line speech key and endpoint for your bot.
3. Create a client application using the direct line speech channel SDK for your platform (C#, JavaScript, or Java). Directline speech channel SDK
4. Initialize the direct line speech channel client with the direct line speech key and endpoint. Directline speech channel client
5. Start a conversation with the bot using the direct line speech channel client.
6. Send and receive speech messages from the bot using the direct line speech channel client.
7. End the conversation with the bot using the direct line speech channel client.

Here is an example of how to use direct line speech channel with C#:

<!DOCTYPE html>

<html lang="en-US">

  <head>

    <title>Web Chat: Using Direct Line Speech</title>

    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

    <script crossorigin="anonymous" src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>

    <style>

      html,

      body {

        height: 100%;

      }



      body {

        margin: 0;

      }



      #webchat {

        height: 100%;

        width: 100%;

      }

    </style>

  </head>



  <body>

    <div id="webchat" role="main"></div>

    <script>

      (async function() {

        const fetchCredentials = async () => {

          const res = await fetch('https://webchat-mockbot-streaming.azurewebsites.net/speechservices/token', {

            method: 'POST'

          });



          if (!res.ok) {

            throw new Error('Failed to fetch authorization token and region.');

          }



          const { region, token: authorizationToken } = await res.json();



          return { authorizationToken, region };

        };



        const adapters = await window.WebChat.createDirectLineSpeechAdapters({

          fetchCredentials

        });



        window.WebChat.renderWebChat(

          {

            ...adapters

          },

          document.getElementById('webchat')

        );



        document.querySelector('#webchat > *').focus();

      })().catch(err => console.error(err));

    </script>

  </body>

</html>

Reference : BotFramework-WebChat/samples/03.speech/a.direct-line-speech at main · microsoft/BotFramework-WebChat

​

How to use direct line channel with examples​

To use direct line channel with your bot, you need to follow these steps:

1. Create a bot using Azure Bot Service and enable the direct line channel in the bot settings. How to enable dierctline channel
2. Generate a direct line secret or token for your bot.
3. Create a client application that can send and receive HTTP requests.
4. Initialize the direct line channel client with the direct line secret or token.
5. Start a conversation with the bot using the direct line channel client.
6. Send and receive text or speech messages from the bot using the direct line channel client.
7. End the conversation with the bot using the direct line channel client.

Here is an example of how to use direct line channel with C# and Cognitive Services Speech SDK:

<!DOCTYPE html>

<html lang="en-US">

  <head>

    <title>Web Chat: Browser-supported speech</title>

    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

    <script crossorigin="anonymous" src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>

    <style>

      html,

      body {

        height: 100%;

      }



      body {

        margin: 0;

      }



      #webchat {

        height: 100%;

        width: 100%;

      }

    </style>

  </head>

  <body>

    <div id="webchat" role="main"></div>

    <script>

      (async function() {

        const res = await fetch('https://webchat-mockbot.azurewebsites.net/directline/token', { method: 'POST' });

        const { token };

        window.WebChat.renderWebChat(

          {

            directLine: window.WebChat.createDirectLine({ token }),

            webSpeechPonyfillFactory: window.WebChat.createBrowserWebSpeechPonyfillFactory()

          },

          document.getElementById('webchat')

        );



        document.querySelector('#webchat > *').focus();

      })().catch(err => console.error(err));

    </script>

  </body>

</html>

Reference : BotFramework-WebChat/samples/03.speech/f.web-browser-speech at main · microsoft/BotFramework-WebChat

​

What are pre-requisites to use direct line channel?​

To use direct line channel with your bot, you need to have the following:

• An Azure subscription.
• An Azure Bot Service resource.
• A direct line secret or token for your bot.
• A client application that can send and receive HTTP requests.
• Optionally, a speech service or SDK to use speech with your bot.

​

What are pre-requisites to use direct line speech channel?​

To use direct line speech channel with your bot, you need to have the following:

• An Azure subscription.
• An Azure Bot Service resource.
• A direct line speech key and endpoint for your bot.
• A client application that can use the direct line speech channel SDK for your platform (C#, JavaScript, or Java).

​

Sample references and documents​

For more information and examples on how to use direct line channel and direct line speech channel with your bot, you can refer to the following resources:

• Connect a bot to Direct Line
• Connect a bot to Direct Line Speech
• Speech SDK
• Web Chat samples
• Direct Line Speech Client sample

​

How to use direct line channel with cognitive services like speech​

To use direct line channel with cognitive services like speech, you need to use the Cognitive Services Speech SDK or the Web Chat Speech Services to handle the speech recognition and synthesis for your bot. You also need to handle the audio streaming and encoding between your client application and the speech service.



The Cognitive Services Speech SDK is a cross-platform library that enables you to use speech as an input and output modality for your bot. It supports various languages, voices, and speech settings. You can use the Speech SDK with any client application that can communicate with the direct line channel, such as a web, mobile, or desktop app.



The Web Chat Speech Services is a web-based service that enables you to use speech as an input and output modality for your bot. It supports various languages, voices, and speech settings. You can use the Web Chat Speech Services with any web-based client application that can embed the Web Chat component, such as a web page or a web app.

​

How to use direct line speech channel with cognitive services like speech​

To use direct line speech channel with cognitive services like speech, you do not need to use any additional services or SDKs, as the direct line speech channel handles the speech recognition and synthesis for you. You only need to use the direct line speech channel SDK for your client application, which is available for C#, JavaScript, and Java.



The direct line speech channel SDK is a cross-platform library that enables you to use speech as the primary input and output modality for your bot. It supports various languages, voices, and speech settings. You can use the direct line speech channel SDK with any client application that can communicate with the direct line speech channel, such as a web, mobile, or desktop app.

Continue reading...