Copilot Studio - Sensitive content in agent responses
Agent Sensitive Response Content
Query
let lookback = 1d;
AppEvents
| where TimeGenerated > ago(lookback)
| where Name == "BotMessageSend"
| extend
ConvId = tostring(Properties["conversationId"]),
ChannelId = tostring(Properties["channelId"]),
Output = tostring(Properties["text"])
| where isnotempty(Output)
| extend
AwsKey = array_length(extract_all(@"(AKIA[0-9A-Z]{16})", Output)),
PrivateKey = iff(Output contains "-----BEGIN" and Output contains "PRIVATE KEY-----", 1, 0),
Jwt = array_length(extract_all(@"(eyJ[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,})", Output)),
Emails = array_length(extract_all(@"([A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,})", Output)),
CardLike = array_length(extract_all(@"\b((?:\d[ \-]?){13,16})\b", Output))
| extend HitScore = AwsKey + PrivateKey + Jwt + iff(Emails >= 5, 1, 0) + iff(CardLike >= 1, 1, 0)
| where HitScore > 0
| project
TimeGenerated, UserId, ConvId, ChannelId,
AwsKey, PrivateKey, Jwt, Emails, CardLike, HitScore,
Output = substring(Output, 0, 1024)
| order by HitScore desc, TimeGenerated descExplanation
This query is designed to monitor and analyze outbound responses from Copilot Studio for potentially sensitive information. It looks for patterns that resemble secret keys, private keys, JWTs, multiple email addresses, and card-like numbers within the messages sent by the bot. Here's a simplified breakdown of what the query does:
-
Time Frame: It examines events from the last day (
lookback = 1d). -
Event Filtering: It focuses on events where the bot sends a message (
Name == "BotMessageSend"). -
Data Extraction: For each message, it extracts the conversation ID, channel ID, and the message text.
-
Pattern Matching: It checks the message text for:
- AWS keys
- Private keys
- JWTs (JSON Web Tokens)
- Email addresses (counts if there are five or more)
- Card-like numbers
-
Scoring: It calculates a "HitScore" based on the presence of these patterns. A higher score indicates more potential sensitive content.
-
Filtering: Only messages with a
HitScoregreater than 0 are considered, meaning they contain at least one type of sensitive information. -
Output: It displays the time the message was generated, user ID, conversation ID, channel ID, the count of each type of sensitive information found, the
HitScore, and a truncated version of the message text (up to 1024 characters). -
Ordering: Results are sorted by
HitScoreand time, with the highest scores and most recent messages appearing first.
The query is tagged with tactics and techniques related to data exfiltration and collection, indicating its use in identifying potential data leaks or unauthorized data collection.
Details
David Alonso
Released: June 8, 2026
Tables
Keywords
Operators
Tactics