OpenAI - New tool / function first use per user
Open AI Tool Usage Drift
Query
let baselineWindow = 14d;
let recentWindow = 1h;
let known =
OpenAIChatCompletions
| where TimeGenerated between (ago(baselineWindow) .. ago(recentWindow))
| where isnotempty(ToolName)
| extend ActorUser = tostring(AdditionalFields.input_user)
| summarize by ModelName, ActorUser, ToolName;
OpenAIChatCompletions
| where TimeGenerated > ago(recentWindow)
| where isnotempty(ToolName)
| extend ActorUser = tostring(AdditionalFields.input_user)
| join kind=leftanti known on ModelName, ActorUser, ToolName
| summarize
FirstSeen = min(TimeGenerated),
Requests = count(),
Models = make_set(ModelName, 10)
by ActorUser, ToolName
| project FirstSeen, ActorUser, ToolName, Requests, Models
| order by FirstSeen descExplanation
This KQL query is designed to monitor the usage of tools or functions by OpenAI users over a 14-day period and alert when a new tool, not previously used by a user, is detected in the last hour. Here's a simple breakdown of what the query does:
-
Baseline Creation: It establishes a 14-day baseline of tools (or function-calling targets) that each OpenAI user has used per model. This is done by looking at the
ASimAgentEventLogsdata and recording which tools each user has interacted with. -
Recent Activity Monitoring: It then checks the tool usage in the last hour to identify any tools that were not part of the 14-day baseline.
-
Alert Generation: If a new tool is detected (one that wasn't used in the past 14 days by a specific user), it generates an alert. This helps in identifying:
- Capability drift (changes in what tools are being used).
- Newly granted access to functions.
- Potentially malicious tools introduced by attackers.
-
Severity and Tactics: The alert is categorized with a medium severity level and is associated with tactics like Execution and Privilege Escalation, which are common in cybersecurity contexts.
-
Entity Mapping and Incident Configuration: The query maps the user account involved and is configured to create incidents when such anomalies are detected. It groups alerts by user account and has settings to manage how incidents are reopened or grouped.
-
Operational Details: The query runs every hour (
queryFrequency: PT1H) and looks back over a 14-day period (queryPeriod: P14D). It is enabled and actively monitors for these conditions.
Overall, this query is a proactive measure to detect unusual tool usage patterns that could indicate security issues or changes in user behavior.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators
Severity
MediumTactics
Frequency: PT1H
Period: P14D