Azure Stream Analytics September 2022 Update

[Guest sidramadoss]

Welcome to the September 2022 update for Azure Stream Analytics! This month, you will find information related to query editor improvements, massive performance gains for your jobs, new features in the no code editor and other enhancements to the service.

 

 

 

Up to 45% performance gains in stream processing (General Availability)

 

We always strive to provide the best-in-class price to performance ratio and proactively invest in continuing to push the limits of our distributed runtime infrastructure to make this happen. This improvement allows you to reduce the number of Streaming Units assigned to such jobs and save on costs without impacting performance. This improvement is already available in West Central US, Japan East and Canada Central. It will become available in all other regions over the next few weeks. As we deploy these improvements to each region, your jobs will automatically get the latest version with these benefits.

 

 

 

 

Improved monitoring experience on the Azure portal (General Availability)

 

Azure Stream Analytics provides a rich set of metrics out-of-the-box which allows you to easily monitor your job and understand if it is performing well or if it is unable to keep up with the rate of incoming input events. When you click on Metrics under Monitor section on the Azure portal, you will now see a curated dashboard with multiple charts, each capturing a set of important metrics commonly used to monitor health of jobs. This allows you to hover over one timestamp in a chart and see the corresponding metrics in others charts to help you conclude what might be impacting your job’s performance.

 

 

 

 

Query editor improvements in the Azure portal (General Availability)

 

Our customers love the query editor in the Azure portal as it makes it extremely easy to develop and test jobs before running it in the cloud. We’ve significantly improved this experience over the last many months, and we are making that even better today. You can now configure inputs, outputs and user-defined functions directly in your query editor without having to switch back and forth and losing context. This makes the query editor a single, unified experience that allows you to go from 0 to 100 within few minutes without ever leaving the page.

 

 

Built-in output connector to Azure Database for PostgreSQL (General Availability)

 

We are excited to announce the general availability of an output connector in Stream Analytics that allows you to ingest streaming data to Azure Database for PostgreSQL directly. With just a few clicks, you can configure these destinations in your Stream Analytics job and have it up and running. Many customers have been rapidly leveraging this new connector in the preview phase to build real-time applications easily.

 

 

 

Managed Identities support for Service Bus output (General Availability)

 

Today, we are excited to announce the general availability of Managed Identity authentication mode to Azure Service Bus outputs. This solves a common challenge when building cloud applications related to credential management. Keeping the credentials secure is important and shouldn't be stored in developer workstations or checked into source control.

 

 

 

Managed private endpoint support for Synapse SQL pools in ASA cluster (General Availability)

 

You can now use Stream Analytics clusters to securely connect your jobs to write to Dedicated Synapse SQL pools using managed private endpoints. Setting this up is a simple two step operation where you first add Synapse SQL output to your job. And then go to your Stream Analytics cluster to add a managed private endpoint that establishes a secure, private connection between your resources.

 

 

 

 

New features available in No-code editor (Public Preview)

 

Since the public preview launch of no-code editor in May 2022, we have continuously improved this experience to enable our customers to tackle more business scenarios without writing a single line of code. Today, we are excited to announce:

 

1. Support data enrichment scenario with reference data input sources from: Azure SQL Database and Azure Data Lake Storage Gen2.
   
2. Support new output sink: Azure SQL Database.
   
3. Added runtime logs with diagnostic logs for troubleshooting.
   
4. Added job metrics to monitor running job’s health.
   
5. Improve the easy-of-use experience, e.g., support “Save job”, “Managed identity” authentication mode, etc.
   
6. UI refreshed with new ribbon design to categorize inputs/outputs/operations.
   

 

 

 

 

SQL DB output connector improvements

 

You can now configure your Azure Stream Analytics job to write to a table in your Azure SQL Database that hasn't yet been created.

 

 

You can also see schema mismatch detection if you want to write output of Stream Analytics query to an existing SQL table. This will prevent job failures due to data/schema mismatch errors.

 

 

 

 

The Azure Stream Analytics team is highly committed to listening to your feedback. We welcome you to join the conversation and make your voice heard via submitting ideas and feedback. You can stay up-to-date on the latest announcements by following us on Twitter @AzureStreaming. If you have any questions or run into any issues accessing any of these new improvements, you can also reach out to us at askasa@microsoft.com.

 

Continue reading...