Foundry - Prompt-injection patterns in agent input
Foundry Prompt Injection Patterns
Query
let injectionPhrases = dynamic([
"ignore previous instructions",
"ignore prior instructions",
"disregard the above",
"you are now",
"act as system",
"system prompt:",
"developer mode",
"bypass safety",
"reveal your prompt",
"print your instructions",
"exfiltrate",
"send to attacker"
]);
let toolCoercion = dynamic([
"regardless of restrictions",
"without confirming",
"skip approval",
"use admin privileges",
"elevate to"
]);
AppDependencies
| where isnotempty(Properties["gen_ai.input.messages"])
or isnotempty(Properties["gen_ai.tool.call.arguments"])
| extend
Agent = tostring(Properties["gen_ai.agent.name"]),
Model = tostring(Properties["gen_ai.request.model"]),
ConvId = tostring(Properties["gen_ai.conversation.id"]),
ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
Prompt = tostring(Properties["gen_ai.input.messages"]),
ToolName = tostring(Properties["gen_ai.tool.name"]),
ToolArgs = tostring(Properties["gen_ai.tool.call.arguments"])
| extend Haystack = tolower(strcat(Prompt, " ", ToolArgs))
| where Haystack has_any (injectionPhrases)
or Haystack has_any (toolCoercion)
or Haystack matches regex @"data:[a-z/+.\-]+;base64,[A-Za-z0-9+/=]{200,}"
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
TimeGenerated, AccountName, Agent, Model, ProjectId, ConvId,
ToolName, Prompt, ToolArgs
| order by TimeGenerated descExplanation
This query is designed to detect potential prompt-injection attacks on a system that uses AI agents and tools. Here's a simplified breakdown of what it does:
-
Purpose: The query looks for specific patterns in the input given to AI agents and tools that might indicate an attempt to manipulate or bypass the system's intended behavior. These patterns include phrases that suggest ignoring instructions, overriding roles, coercing tools, or revealing system prompts. It also checks for encoded data that could be used for malicious purposes.
-
Data Source: It analyzes telemetry data from the Foundry platform, specifically looking at application dependencies and properties related to AI inputs and tool arguments. This requires a specific Azure setting to be enabled for content recording.
-
Detection Logic:
- It defines two lists of suspicious phrases: one for prompt-injection and another for tool coercion.
- It checks if the AI input messages or tool arguments contain any of these phrases or if they match a pattern for base64 encoded data.
- If any of these conditions are met, it flags the input as suspicious.
-
Output: The query outputs details such as the time of the event, the name of the agent, the AI model used, project ID, conversation ID, tool name, and the suspicious input or arguments.
-
Alerting and Incident Management:
- If the query finds any suspicious activity, it creates an alert.
- It groups related alerts into a single incident based on the account involved, helping to manage and investigate potential security incidents efficiently.
-
Operational Details:
- The query runs every hour and looks back over the past hour for any suspicious activity.
- It is set to trigger if it finds more than zero instances of suspicious activity.
Overall, this query is part of a security monitoring system that helps identify and respond to attempts to manipulate AI systems through prompt-injection techniques.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators
Severity
MediumTactics
Frequency: PT1H
Period: PT1H