Multiple Ingestion Delays
Query
let query_period = 7d;
union *
| where TimeGenerated > ago(query_period)
| distinct Type, bin(IngestionTime = ingestion_time(), 30m)
| sort by Type asc, IngestionTime asc
| extend Difference = iff(Type == prev(Type), IngestionTime - prev(IngestionTime), 0s)
| summarize Frequency = max(Difference) by Type
| extend Frequency = iff(Frequency == 0s, query_period, Frequency)
| lookup kind=leftouter (
union *
| where TimeGenerated > ago(query_period)
| summarize percentiles(IngestionDelay = ingestion_time() - TimeGenerated, 50, 80, 95, 99) by Type
) on TypeExplanation
This query is designed to analyze data ingestion patterns over the past 7 days. Here's a simplified breakdown of what it does:
-
Define the Time Period: It sets a time period of 7 days to look back from the current time.
-
Combine Data Sources: It combines all available data sources into a single dataset.
-
Filter Recent Data: It filters the data to include only records generated within the last 7 days.
-
Identify Unique Records: It identifies distinct combinations of 'Type' and 'IngestionTime', with 'IngestionTime' rounded to the nearest 30 minutes.
-
Sort Data: It sorts the data by 'Type' and 'IngestionTime' in ascending order.
-
Calculate Time Differences: It calculates the time difference between consecutive ingestion times for each type. If the type is the same as the previous record, it computes the difference; otherwise, it sets the difference to zero.
-
Determine Frequency: It finds the maximum time difference (frequency) for each type. If the frequency is zero, it defaults to the entire query period (7 days).
-
Analyze Ingestion Delays: It performs a separate analysis to calculate the 50th, 80th, 95th, and 99th percentiles of the ingestion delay (the time difference between when data was generated and when it was ingested) for each type.
-
Combine Results: It performs a left outer join to combine the frequency data with the ingestion delay percentiles based on the 'Type'.
Overall, the query provides insights into how frequently data is ingested for each type and the typical delays in data ingestion over the past week.
Details
Jose Sebastián Canós
Released: July 14, 2026
Tables
Keywords
Operators