Query Details
Foundry New Model First Seen
Query
id: 2b3c4d5e-1111-4310-9210-0123456789e0
name: Foundry - Agent uses a model never seen before
description: |
Detects a Foundry / Agent Service agent invoking a model
(gen_ai.request.model) that has not appeared for that agent in the
preceding 14-day baseline. A first-seen model is the shadow-model /
model-downgrade / unapproved-deployment shape: an agent silently
switched to a cheaper, unsanctioned or externally hosted model, or an
injection re-pointed it. Low noise by design - a model only fires once
per agent when it first appears, so legitimate roll-outs self-clear.
Reads gen_ai.request.model from the AppDependencies span property bag
(Properties), keyed on gen_ai.agent.name. The 14-day baseline is built
with a leftanti join, identical to the new-tool detection pattern. No
allowlist to maintain.
severity: Medium
requiredDataConnectors:
- connectorId: ApplicationInsights
dataTypes:
- AppDependencies
queryFrequency: PT1H
queryPeriod: P14D
triggerOperator: gt
triggerThreshold: 0
enabled: true
tactics:
- DefenseEvasion
- Execution
relevantTechniques:
- T1562
- T1059
query: |
let lookback = 1h;
let baselineWindow = 14d;
let recent =
AppDependencies
| where TimeGenerated > ago(lookback)
| where isnotempty(Properties["gen_ai.request.model"])
| extend
Agent = tostring(Properties["gen_ai.agent.name"]),
Model = tolower(tostring(Properties["gen_ai.request.model"])),
ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
ConvId = tostring(Properties["gen_ai.conversation.id"])
| where isnotempty(Model);
let baseline =
AppDependencies
| where TimeGenerated between (ago(baselineWindow) .. ago(lookback))
| where isnotempty(Properties["gen_ai.request.model"])
| extend
Agent = tostring(Properties["gen_ai.agent.name"]),
Model = tolower(tostring(Properties["gen_ai.request.model"]))
| where isnotempty(Model)
| distinct Agent, Model;
recent
| join kind=leftanti baseline on Agent, Model
| summarize
Calls = count(),
FirstSeen = min(TimeGenerated),
LastSeen = max(TimeGenerated),
AnyProject = take_any(ProjectId),
Convs = make_set(ConvId, 10)
by Agent, Model
| extend ProjectId = AnyProject
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
LastSeen, AccountName, Agent, Model, ProjectId, Calls, FirstSeen, Convs
| order by LastSeen desc
entityMappings:
- entityType: Account
fieldMappings:
- identifier: Name
columnName: AccountName
- entityType: CloudApplication
fieldMappings:
- identifier: Name
columnName: Model
eventGroupingSettings:
aggregationKind: SingleAlert
incidentConfiguration:
createIncident: true
groupingConfiguration:
enabled: true
reopenClosedIncident: false
lookbackDuration: PT6H
matchingMethod: Selected
groupByEntities:
- Account
groupByAlertDetails: []
groupByCustomDetails: []
version: 1.0.0
kind: Scheduled
tags:
- Sentinel-As-Code
- Custom
- Foundry
- AI
- OWASP-LLM10
Explanation
This query is designed to detect when a Foundry or Agent Service agent uses a machine learning model that it hasn't used in the past 14 days. Here's a simplified breakdown:
-
Purpose: The query identifies when an agent starts using a new model that hasn't been seen in the last 14 days. This could indicate a switch to a cheaper, unauthorized, or externally hosted model, or a potential security issue like an injection attack.
-
Data Source: It uses data from the
AppDependenciestable, specifically looking at thegen_ai.request.modelproperty to identify the model being used and thegen_ai.agent.nameto identify the agent. -
Time Frame: The query checks for new models every hour (
queryFrequency: PT1H) over a 14-day period (queryPeriod: P14D). -
Detection Logic:
- Recent Models: It first gathers data on models used in the last hour.
- Baseline Models: It then gathers data on models used in the 14 days before the last hour.
- Comparison: It compares these two sets of data to find models that appear in the recent data but not in the baseline data.
-
Output: For each new model detected, it summarizes the number of times it was called, the first and last time it was seen, and any associated project or conversation IDs.
-
Severity and Response: The severity of this detection is set to "Medium". If a new model is detected, it triggers an alert and potentially creates an incident for further investigation.
-
Tactics and Techniques: The query is associated with tactics like Defense Evasion and Execution, and techniques such as T1562 (Impair Defenses) and T1059 (Command and Scripting Interpreter).
-
Entity Mapping: It maps detected agents to accounts and models to cloud applications for better context in alerts.
-
Incident Management: If an incident is created, it can be grouped by the account involved, but won't reopen closed incidents.
Overall, this query helps monitor and alert on potentially unauthorized or unexpected changes in the models used by agents, which could indicate security risks or policy violations.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators