Foundry - Agent uses a model never seen before

Foundry New Model First Seen

Query

let lookback = 1h;
let baselineWindow = 14d;
let recent =
    AppDependencies
    | where TimeGenerated > ago(lookback)
    | where isnotempty(Properties["gen_ai.request.model"])
    | extend
        Agent     = tostring(Properties["gen_ai.agent.name"]),
        Model     = tolower(tostring(Properties["gen_ai.request.model"])),
        ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
        ConvId    = tostring(Properties["gen_ai.conversation.id"])
    | where isnotempty(Model);
let baseline =
    AppDependencies
    | where TimeGenerated between (ago(baselineWindow) .. ago(lookback))
    | where isnotempty(Properties["gen_ai.request.model"])
    | extend
        Agent = tostring(Properties["gen_ai.agent.name"]),
        Model = tolower(tostring(Properties["gen_ai.request.model"]))
    | where isnotempty(Model)
    | distinct Agent, Model;
recent
| join kind=leftanti baseline on Agent, Model
| summarize
    Calls     = count(),
    FirstSeen = min(TimeGenerated),
    LastSeen  = max(TimeGenerated),
    AnyProject = take_any(ProjectId),
    Convs     = make_set(ConvId, 10)
    by Agent, Model
| extend ProjectId = AnyProject
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
    LastSeen, AccountName, Agent, Model, ProjectId, Calls, FirstSeen, Convs
| order by LastSeen desc

Explanation

This query is designed to detect when a Foundry or Agent Service agent uses a machine learning model that it hasn't used in the past 14 days. Here's a simplified breakdown:

Purpose: The query identifies when an agent starts using a new model that hasn't been seen in the last 14 days. This could indicate a switch to a cheaper, unauthorized, or externally hosted model, or a potential security issue like an injection attack.
Data Source: It uses data from the AppDependencies table, specifically looking at the gen_ai.request.model property to identify the model being used and the gen_ai.agent.name to identify the agent.
Time Frame: The query checks for new models every hour (queryFrequency: PT1H) over a 14-day period (queryPeriod: P14D).
Detection Logic:
- Recent Models: It first gathers data on models used in the last hour.
- Baseline Models: It then gathers data on models used in the 14 days before the last hour.
- Comparison: It compares these two sets of data to find models that appear in the recent data but not in the baseline data.
Output: For each new model detected, it summarizes the number of times it was called, the first and last time it was seen, and any associated project or conversation IDs.
Severity and Response: The severity of this detection is set to "Medium". If a new model is detected, it triggers an alert and potentially creates an incident for further investigation.
Tactics and Techniques: The query is associated with tactics like Defense Evasion and Execution, and techniques such as T1562 (Impair Defenses) and T1059 (Command and Scripting Interpreter).
Entity Mapping: It maps detected agents to accounts and models to cloud applications for better context in alerts.
Incident Management: If an incident is created, it can be grouped by the account involved, but won't reopen closed incidents.

Overall, this query helps monitor and alert on potentially unauthorized or unexpected changes in the models used by agents, which could indicate security risks or policy violations.

Details

David Alonso

Released: June 8, 2026

Tables

AppDependencies

Keywords

FoundryAgentModelProjectIdConversationIdAccountNameApplicationInsightsAppDependenciesPropertiesTimeGenerated

Operators

letagoisnotemptytostringtolowerbetweendistinctjoinkind=leftantisummarizecountminmaxtake_anymake_setbyiffisemptyprojectorder bydesc

Severity

Medium

Tactics

DefenseEvasionExecution

MITRE Techniques

T1562 T1059

Frequency: PT1H

Period: P14D

Actions

GitHub

KQL Search