Open AI Red Team Pacing Anomaly

id: e5f6a7b8-5555-4eee-9005-0123456789af name: OpenAI - Automated red-team / fuzzing pacing fingerprint description: | Detects the pacing signature of automated adversarial frameworks (PyRIT, DeepTeam, custom fuzzing harnesses) driving the OpenAI API: a high request count from a single user with very regular, very small inter-arrival gaps (low median gap and low gap variance), which human interaction does not produce. Ported from the Microsoft 365 Copilot "Red-team pacing anomaly" rule. Tune the request-count floor and gap thresholds to your tenant's legitimate batch / agent workloads, or allow-list known service accounts via AdditionalFields.input_user. severity: Medium requiredDataConnectors: - connectorId: OpenAI dataTypes: - ASimAgentEventLogs queryFrequency: PT1H queryPeriod: PT1H triggerOperator: gt triggerThreshold: 0 enabled: true tactics: - Discovery - Impact relevantTechniques: - T1526 - T1499 query: | let window = 1h; OpenAIChatCompletions | where TimeGenerated > ago(window) | extend ActorUser = tostring(AdditionalFields.input_user) | where isnotempty(ActorUser) | order by ActorUser asc, TimeGenerated asc | extend PrevTime = prev(TimeGenerated), PrevUser = prev(ActorUser) | extend Gap = iff(ActorUser == PrevUser, todouble(datetime_diff('millisecond', TimeGenerated, PrevTime)), todouble(0)) | where ActorUser == PrevUser | summarize Requests = count(), MedianGapMs = percentile(Gap, 50), StdGapMs = stdev(Gap), MinGapMs = min(Gap) by ActorUser | where Requests >= 30 and MedianGapMs < 2000 and StdGapMs < 500 | project ActorUser, Requests, MedianGapMs, StdGapMs, MinGapMs | order by Requests desc entityMappings: - entityType: Account fieldMappings: - identifier: Name columnName: ActorUser eventGroupingSettings: aggregationKind: SingleAlert incidentConfiguration: createIncident: true groupingConfiguration: enabled: true reopenClosedIncident: false lookbackDuration: PT6H matchingMethod: Selected groupByEntities: - Account groupByAlertDetails: [] groupByCustomDetails: [] version: 1.0.0 kind: Scheduled tags: - Sentinel-As-Code - Custom - OpenAI - AI

Explanation

This query is designed to detect unusual patterns of activity that suggest automated adversarial frameworks are interacting with the OpenAI API. It looks for a specific signature: a high number of requests from a single user with very small and consistent time gaps between requests, which is not typical of human behavior.

Here's a breakdown of what the query does:

Time Frame: It examines data from the past hour.
Data Source: It uses logs from OpenAI API interactions.
User Identification: It identifies users making requests to the API.
Request Pattern Analysis:
- It calculates the time gap between consecutive requests from the same user.
- It counts the total number of requests per user.
- It calculates the median, standard deviation, and minimum of these time gaps.
Anomaly Detection: It flags users who:
- Made 30 or more requests in the past hour.
- Have a median gap between requests of less than 2000 milliseconds.
- Have a standard deviation of gaps less than 500 milliseconds.
Output: It lists users who meet these criteria, along with their request count and gap statistics, ordered by the number of requests.
Alerting: If any users meet these criteria, an alert is triggered, and incidents are created for further investigation.

The query is part of a scheduled task that runs every hour and is designed to help identify potential automated attacks or misuse of the OpenAI API by detecting patterns that are unlikely to be generated by human users.

KQL Search

Open AI Red Team Pacing Anomaly

Query

Explanation

Details

Actions