Azure Monitor Logs search job experience

[Guest RoyGal]

Introduction

 

 

With the introduction of new Azure Monitor Logs capabilities like Basic Logs and Logs Archive we are happy to introduce a new way to query and explore your logs - search job.

Search job allows users to run a reduced KQL a-synchronous query that explores big quantities of data and produces a persistent result set that may be the basis of further exploration.

 

Read more about Azure Monitor Logs new capacities in our Microsoft Ignite Announcement.

 

Basic Logs and Archive

 

 

Basic Logs and Archive offers a unique set of capabilities for Logs users.

Basic Logs a cheaper tier of logs that allows users to collect and retain Logs cheaply, while maintaining basic query capabilities.

 

Archive Logs allows cheap, long term retention of Logs for an extended period of time.

 

These new offerings empower users and organizations to optimize their logs cost planning to optimize logs cost and create a better suited logs estate.

 

These new capabilities require a new type of exploration experience - Search Job.

 

Search Job

 

 

Search job is an effective way to explore logs in the following cases:

 

1. A query in an Analytics table meets one of it's limitations - in this case, using a search job to create an interim table might help in reaching the desired insight.

 

2. When exploring Basic Logs for a period that exceeds the last 8 days

 

3. When exploring Archive Logs

 

 

 

Search Job uses reduced KQL and an a-synchronous query pattern to distribute the query and run multiple instances of the query to reduce time to result and explore a large data set.

 

Search Job ingests the result set to a new Analytics table - this allows persistency of results and allows the exploration of the result set using full KQL interactive experience.

 

Read more about how Search Job works in this article - Azure Monitor Logs Search Job.

 

The Search Job Experience

 

 

To use Search Job you must first enable the Search Job mode in Log Analytics.

 

To enable search job mode, go to the ellipsis menu on the right hand side of the screen and toggle Search Job mode on:

 

 

Enabling Search Job Mode will optimize your experience for ruining search jobs:

 

1. Run button will change it's appearance to indicate Azure Monitor Logs is in Search Job mode

 

2. Azure Monitor Logs intellisense will adjust to support reduced KQL and assist when composing a query:

 

 

Please note: it is recommended to compose and optimize your query before submitting a search job.

 

When you are ready, click the 'Search Job' button.

You will be asked to provide a name for the result set table:

 

 

Once you initiate the search job, Logs will create a new table in your workspace and will run your query.

 

Results will start flowing to the newly created results table:

 

 

As results become available, you will be able to explore and query the new results table, as with any other Log Analytics table.

While the query is running, the experience will show specific banners that update on the status of the results table:

 

Result Set

 

 

The Search Job results set will be ingested as a new, fully featured Log Analytics table, this means you may run full KQL analytics queries on the results table.

Additionally, the fact that the result set persists as a new table offers many advantages such as retention control for the results table and the ability to use the data in the results table with other results tables or other Log Analytics tables to achieve rich insights.

 

All search Job tables will appear under the 'Search Results' group in the tables side blade:

 

 

 

 

 

 

 

Summary and Feedback

 

 

We hope you enjoy this new addition to Azure Monitor Logs.

 

Have thoughts and comments about the feature? Please let us know what you think by commenting on this blog or using our feedback feature in Azure Monitor Logs.

 

Simply click the Feedback button and share your thoughts:

 

 

Continue reading...