Query Details

Multiple Delayed Event Ingestion

Query

let query_frequency = 1h;
let query_period = 1d;
let percentile_threshold = 90;
let _ExpectedDelays =
    _GetWatchlist('DataType-IngestedTables')
    | project Type, Critical, ExpectedIngestionDelay = totimespan(IngestionDelay), DelayType, Notes
;
union withsource=_Type
    //This is a comment
    * // withsource= is just used to bypass the Analytics rule wizard
| where TimeGenerated > ago(query_period)
| where ingestion_time() between (ago(query_frequency) .. now())
| summarize IngestionDelay = percentile(ingestion_time() - TimeGenerated, percentile_threshold) by Type
| lookup kind=leftouter _ExpectedDelays on Type
| where not(DelayType == "offline" and Notes has "[LowVolume]")
| where IngestionDelay > ExpectedIngestionDelay * iff(DelayType == "offline", 2, 1)
| extend
    AlertSeverity = case(
    Critical == "true", "High",
    "Informational"
    )
| project Type, Critical, IngestionDelay, ExpectedIngestionDelay, AlertSeverity

Explanation

This query is used to analyze the ingestion delay of different types of data. It retrieves a watchlist of ingested tables and calculates the expected ingestion delay for each type. It then compares the actual ingestion delay to the expected delay and filters out any delays that are within an acceptable range. Finally, it assigns an alert severity level based on the criticality of the delay. The resulting data includes the type of data, criticality, actual ingestion delay, expected ingestion delay, and alert severity.

Details

Jose Sebastián Canós profile picture

Jose Sebastián Canós

Released: September 27, 2022

Tables

_ExpectedDelays

Keywords

Devices,Intune,User

Operators

letunionwithsourceprojectwhereagoingestion_timenowsummarizebylookupnothasiffextendcase

Actions