OpenAI - Sensitive tool / function invocation

Open AI Sensitive Tool Invocation

Query

let sensitiveTools = dynamic([
    "code_interpreter", "python", "shell", "bash", "cmd", "powershell",
    "exec", "os", "subprocess", "http", "requests", "fetch", "curl",
    "browser", "sql", "database", "query", "email", "sendmail", "smtp",
    "file", "filesystem", "read_file", "write_file", "delete", "secrets",
    "keyvault", "credentials", "ssh", "rdp", "admin"
]);
OpenAIChatCompletions
| where TimeGenerated > ago(15m)
| where isnotempty(ToolName)
| extend
    Tool = tolower(tostring(ToolName)),
    ActorUser = tostring(AdditionalFields.input_user)
| where Tool has_any (sensitiveTools)
| summarize
    Hits = count(),
    Models = make_set(ModelName, 10),
    Requests = make_set(EventRequestId, 5),
    FirstSeen = min(TimeGenerated),
    LastSeen = max(TimeGenerated)
    by ActorUser, Tool
| project
    FirstSeen, LastSeen, ActorUser, Tool, Hits, Models, Requests
| order by Hits desc

Explanation

This query is designed to monitor and detect potentially risky actions involving the OpenAI API. Here's a simplified breakdown:

Purpose: The query identifies OpenAI API requests that use certain sensitive tools or functions, which are considered high-risk. These tools are listed in a watchlist called AzureAI_SensitiveTools.
Context: It's part of a security measure to detect excessive autonomy or actions that might occur after a security breach (post-jailbreak). It complements another rule that flags the first use of any new tool.
Data Source: The query looks at logs from OpenAI API interactions, specifically focusing on the ToolName field in the ASimAgentEventLogs.
Sensitive Tools: The query checks for the use of tools like "powershell", "email", "sql", and others that could be used for executing commands, accessing files, or sending emails.
Detection Logic:
- It filters logs from the last 15 minutes.
- It checks if any of the sensitive tools were used.
- It counts how many times each tool was used and gathers related information like the models used and request IDs.
- It organizes the data by user and tool, showing when the tool was first and last seen in use.
Severity and Response: The severity of this detection is marked as high. If any such activity is detected, an incident is created for further investigation.
Entity Mapping: The query maps detected activities to user accounts and cloud applications for better tracking and response.
Alert Management: Alerts are grouped by user accounts, and incidents are created if necessary, but they won't reopen closed incidents.

Overall, this query helps in identifying and responding to potentially dangerous uses of the OpenAI API, ensuring that any misuse is quickly flagged and investigated.

Details

David Alonso

Released: June 8, 2026

Tables

OpenAIChatCompletions

Keywords

OpenAIAzureAIASimAgentEventLogsToolNameActorUserToolModelsRequestsAccountCloudApplication

Operators

letdynamicwhereisnotemptyextendtolowertostringhas_anysummarizecountmake_setminmaxbyprojectorder bydesc

Severity

High

Tactics

ExecutionImpact

MITRE Techniques

T1059

Frequency: PT15M

Period: PT15M

Actions

GitHub

KQL Search