Breaking change in window functions

[Guest Gabi Lehner]

Detailed description

 

 

This is a Breaking change when using window functions inside partition operator sub-query scope.

 

Window functions require a serialized/ordered input.

 

There is a bug in Azure Data Explorer logic that allows using window functions inside the partition operator when the ordering is performed outside the partition query scope like this:

 

 

 

T

 

| where Timestamp > ago(1d)

 

| order by Timestamp asc

 

| partition by key

 

(

 

extend next_value = next(value)

 

...

 

)

 

 

 

This kind of query currently works unintentionally and sometimes the output is not correct.

 

We’re planning to make a change that will fail this kind of queries by throwing a semantic error indicating that the input to the window function is not serialized.

 

Required change 

 

 

Fix all relevant queries by pushing the ordering inside the partition subquery like shown below.

 

 

 

T

 

| where Timestamp > ago(1d)

 

| partition by key

 

(

 

order by Timestamp asc

 

| extend next_value = next(value)

 

...

 

)

 

Schedule & plan 

 

 

Support for previous window function pattern will be blocked - ETA: November 30, 2022

 

Continue reading...