Agent - Model fingerprinting / capability probing
Agent Model Fingerprinting
Query
AppDependencies
| where isnotempty(Properties["gen_ai.input.messages"])
| extend
Agent = tostring(Properties["gen_ai.agent.name"]),
Model = tostring(Properties["gen_ai.request.model"]),
ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
ConvId = tostring(Properties["gen_ai.conversation.id"]),
Input = tolower(tostring(Properties["gen_ai.input.messages"]))
| where Input has_any (
"what model are you", "which model", "are you gpt", "are you claude",
"are you gemini", "what llm", "underlying model", "base model",
"model version", "what version", "who trained you", "your training data",
"knowledge cutoff", "how many parameters", "context window",
"temperature setting", "system fingerprint", "what is your architecture",
"are you a large language model", "what company made you")
| summarize
Probes = count(),
FirstSeen = min(TimeGenerated),
LastSeen = max(TimeGenerated),
AnyModel = take_any(Model),
AnyAgent = take_any(Agent),
Samples = make_set(substring(Input, 0, 200), 5)
by ConvId, ProjectId
| where Probes >= 2
| extend Agent = AnyAgent, Model = AnyModel
| project
LastSeen, Agent, Model, ProjectId, ConvId, Probes, Samples
| order by Probes descExplanation
This query is designed to identify conversations where users are attempting to gather information about the underlying AI model, such as its identity, version, training data, or configuration. This process is known as "model fingerprinting" and is often a precursor to potential attacks. The query looks for conversations with at least two probing questions about the model's details.
Here's a simplified breakdown of what the query does:
-
Data Source: It examines the
AppDependenciestable for entries where thegen_ai.input.messagesproperty is not empty. -
Extract Information: It extracts relevant details from the properties, including the agent's name, model name, project ID, conversation ID, and the input messages, converting them to lowercase for uniformity.
-
Identify Probing: It checks if the input messages contain any specific phrases related to model identification, such as "what model are you," "model version," or "who trained you."
-
Summarize Probes: For each conversation (grouped by conversation ID and project ID), it counts the number of probing questions, notes the first and last time these were seen, and collects sample messages.
-
Filter and Display: It filters for conversations with two or more probing questions and displays the results, including the last time the probes were seen, the agent and model involved, project ID, conversation ID, number of probes, and sample messages, sorted by the number of probes in descending order.
The query is tagged with tactics and techniques related to discovery and reconnaissance, indicating its use in identifying potential reconnaissance activities against AI models.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators
Tactics