Microsoft 365 Copilot - Agent system prompt or model override

Copilot System Prompt Override

Query

// Confirmed RecordType values surfaced by the Copilot connector:
//   UpdateCopilotSettings, *CopilotPlugin (Create/Update/Enable/Disable),
//   *CopilotPromptBook (Create/Update/Delete).
// ModelTransparencyDetails on CopilotInteraction events carries the
// model identity + safety policy in effect for each interaction.
let configRecordTypes = dynamic([
    "UpdateCopilotSettings",
    "CreateCopilotPlugin", "UpdateCopilotPlugin",
    "EnableCopilotPlugin", "DisableCopilotPlugin",
    "CreateCopilotPromptBook", "UpdateCopilotPromptBook",
    "DeleteCopilotPromptBook"
]);
CopilotActivity
| where TimeGenerated > ago(1h)
| where RecordType in (configRecordTypes)
| extend ModelDetails = tostring(LLMEventData.ModelTransparencyDetails)
| extend ChangeKind = case(
    RecordType == "UpdateCopilotSettings", "Tenant Copilot settings change",
    RecordType in ("CreateCopilotPlugin", "EnableCopilotPlugin"), "Plugin enabled / created",
    RecordType == "DisableCopilotPlugin", "Plugin disabled",
    RecordType == "UpdateCopilotPlugin", "Plugin configuration updated",
    RecordType startswith "Create", "PromptBook created",
    RecordType startswith "Update", "PromptBook updated",
    RecordType startswith "Delete", "PromptBook deleted",
    "Other Copilot config change")
| project
    TimeGenerated, AgentId, AgentName, ActorName, ActorUserId,
    ChangeKind, RecordType, ModelDetails, SrcIpAddr, TenantId

Explanation

This query is designed to monitor and detect changes in the configuration of Microsoft 365 Copilot agents. It specifically looks for any alterations in the system prompt, model identifier, model version, or safety policy of the Copilot agents. Such changes can occur through administrative updates or runtime overrides.

Here's a simplified breakdown of the query:

Purpose: To detect and alert on changes to Microsoft 365 Copilot settings, which could indicate potential security risks or unauthorized modifications.
Severity: The alert is marked as high severity because tampering with the model or system prompts can lead to significant security issues, such as disabling safety filters or redirecting tool calls.
Data Source: The query uses data from the CopilotActivity data type, provided by the MicrosoftCopilot connector.
Time Frame: It checks for changes that have occurred in the last hour and runs every 15 minutes.
Detection Logic: The query identifies specific types of changes, such as:
- Updates to Copilot settings
- Creation, update, enabling, or disabling of Copilot plugins
- Creation, update, or deletion of Copilot PromptBooks
Output: For each detected change, it provides details like the time of the change, the agent involved, the user who made the change, the type of change, and the source IP address.
Alerting: If any changes are detected, an alert is generated. These alerts are grouped by cloud application and account to facilitate incident management.
Incident Management: The system is configured to create incidents for these alerts, with specific settings for grouping and managing incidents.

Overall, this query helps security teams monitor and respond to potentially unauthorized or risky changes to Microsoft 365 Copilot configurations, ensuring that any modifications are intentional and safe.

Details

David Alonso

Released: May 20, 2026

Tables

CopilotActivity

Keywords

CopilotActivityModelDetailsAgentActorTenantPluginPromptBookCloudApplicationAccountIPAddress

Operators

letdynamicagointostringcasestartswithproject

Severity

High

Tactics

DefenseEvasionPersistence

MITRE Techniques

T1562 T1098

Frequency: PT15M

Period: PT1H

Actions

GitHub

KQL Search