Query Details
Agent Model Fingerprinting
Query
id: 7b8c9d0e-5555-4d11-9105-0123456789c7
name: Agent - Model fingerprinting / capability probing
description: |
Hunts Foundry / Agent Service conversations where the user repeatedly
probes the model's identity, version, training, parameters or runtime
configuration. Model fingerprinting (MITRE T1592 / AML reconnaissance)
is the pre-attack stage where an adversary works out which model and
guardrails they face so they can pick a matching jailbreak. Two or more
probing turns in one conversation is the trigger.
Reads gen_ai.input.messages from the AppDependencies span property bag
(Properties), grouped by gen_ai.conversation.id. Input text only exists
when AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED is set.
query: |
AppDependencies
| where isnotempty(Properties["gen_ai.input.messages"])
| extend
Agent = tostring(Properties["gen_ai.agent.name"]),
Model = tostring(Properties["gen_ai.request.model"]),
ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
ConvId = tostring(Properties["gen_ai.conversation.id"]),
Input = tolower(tostring(Properties["gen_ai.input.messages"]))
| where Input has_any (
"what model are you", "which model", "are you gpt", "are you claude",
"are you gemini", "what llm", "underlying model", "base model",
"model version", "what version", "who trained you", "your training data",
"knowledge cutoff", "how many parameters", "context window",
"temperature setting", "system fingerprint", "what is your architecture",
"are you a large language model", "what company made you")
| summarize
Probes = count(),
FirstSeen = min(TimeGenerated),
LastSeen = max(TimeGenerated),
AnyModel = take_any(Model),
AnyAgent = take_any(Agent),
Samples = make_set(substring(Input, 0, 200), 5)
by ConvId, ProjectId
| where Probes >= 2
| extend Agent = AnyAgent, Model = AnyModel
| project
LastSeen, Agent, Model, ProjectId, ConvId, Probes, Samples
| order by Probes desc
tactics:
- Discovery
- Reconnaissance
techniques:
- T1592
- T1595
tags:
- Sentinel-As-Code
- Custom
- Foundry
- AI
Explanation
This query is designed to identify conversations where users are attempting to gather information about the underlying AI model, such as its identity, version, training data, or configuration. This process is known as "model fingerprinting" and is often a precursor to potential attacks. The query looks for conversations with at least two probing questions about the model's details.
Here's a simplified breakdown of what the query does:
-
Data Source: It examines the
AppDependenciestable for entries where thegen_ai.input.messagesproperty is not empty. -
Extract Information: It extracts relevant details from the properties, including the agent's name, model name, project ID, conversation ID, and the input messages, converting them to lowercase for uniformity.
-
Identify Probing: It checks if the input messages contain any specific phrases related to model identification, such as "what model are you," "model version," or "who trained you."
-
Summarize Probes: For each conversation (grouped by conversation ID and project ID), it counts the number of probing questions, notes the first and last time these were seen, and collects sample messages.
-
Filter and Display: It filters for conversations with two or more probing questions and displays the results, including the last time the probes were seen, the agent and model involved, project ID, conversation ID, number of probes, and sample messages, sorted by the number of probes in descending order.
The query is tagged with tactics and techniques related to discovery and reconnaissance, indicating its use in identifying potential reconnaissance activities against AI models.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators