Conditional Access Baseline Gap Detected Due to Policy Change

Conditional Access Baseline Gap Detected Due Policy Change

Query

let ChangedPolicies = AuditLogs
    | where TimeGenerated > ago(24h)
    | where OperationName has "conditional access policy"
    | where Result =~ "success"
    | extend InitiatingAppName = tostring(InitiatedBy.app.displayName)
    | extend InitiatingAppId = tostring(InitiatedBy.app.appId)
    | extend InitiatingAppServicePrincipalId = tostring(InitiatedBy.app.servicePrincipalId)
    | extend InitiatingUserPrincipalName = tostring(InitiatedBy.user.userPrincipalName)
    | extend InitiatingAadUserId = tostring(InitiatedBy.user.id)
    | extend InitiatingIPAddress = tostring(InitiatedBy.user.ipAddress)
    | extend CaPolicyName = tostring(TargetResources[0].displayName)
    | extend CaPolicyId = tostring(TargetResources[0].id)
    | extend NewPolicyValues = TargetResources[0].modifiedProperties[0].newValue
    | extend OldPolicyValues = TargetResources[0].modifiedProperties[0].oldValue
    | extend
        InitiatingAccountName = tostring(split(InitiatingUserPrincipalName, "@")[0]),
        InitiatingAccountUPNSuffix = tostring(split(InitiatingUserPrincipalName, "@")[1])
    | project-reorder
        TimeGenerated,
        OperationName,
        CaPolicyId,
        CaPolicyName,
        InitiatingAppId,
        InitiatingAppName,
        InitiatingAppServicePrincipalId,
        InitiatingUserPrincipalName,
        InitiatingAadUserId,
        InitiatingIPAddress,
        NewPolicyValues,
        OldPolicyValues;
let PreviousCheckResults = Maester_CL
    | where Block == "Conditional Access Baseline Policies"
    | summarize arg_min(TimeGenerated, *) by Id;
let NewestCheckResults = Maester_CL
    | where Block == "Conditional Access Baseline Policies"
    | summarize arg_max(TimeGenerated, *) by Id;
let FailedChecks = NewestCheckResults
    | join kind=inner PreviousCheckResults on Id
    | where Result != Result1
    | extend PreviousResultDetail = tostring(parse_json(ResultDetail1))
    | extend AffectedCaPolicyIds = extract_all(@'([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})', PreviousResultDetail)
    | project
        Id,
        Title,
        CurrentResult = Result,
        CurrentResultTimestamp = TimeGenerated,
        PreviousResult = Result1,
        PreviousResultTimestamp = TimeGenerated1,
        ResultDetail,
        PreviousResultDetail,
        AffectedCaPolicyIds;
ChangedPolicies
| join kind=inner (
    FailedChecks
    | mv-expand parse_json(AffectedCaPolicyIds)
    | project-rename MaesterId = Id
    | extend CaPolicyId = tostring(AffectedCaPolicyIds)
    | project
        MaesterId,
        CaPolicyId,
        TestTitle = parse_json(PreviousResultDetail).TestTitle,
        CurrentResult,
        TestFinding = ResultDetail.TestResult,
        PreviousTestFinding = parse_json(PreviousResultDetail).TestResult
    )
    on CaPolicyId
| extend FailedChecks = bag_pack_columns(MaesterId, TestTitle, CurrentResult, TestFinding, PreviousTestFinding)
| project
    TimeGenerated,
    OperationName,
    CaPolicyId,
    CaPolicyName,
    InitiatingUserPrincipalName,
    InitiatingAadUserId,
    InitiatingIPAddress,
    Id,
    LoggedByService,
    ActivityDisplayName,
    parse_json(FailedChecks),
    PreviousCaPolicy = parse_json(OldPolicyValues),
    CurrentCaPolicy = parse_json(NewPolicyValues),
    TestTitle,
    TestFinding

Explanation

This query is designed to monitor changes in Conditional Access policies within an organization and identify any critical gaps that these changes might introduce. Here's a simplified breakdown of what the query does:

Monitor Policy Changes: It looks at audit logs from the past 24 hours to find successful modifications to Conditional Access policies. It captures details about who made the change, what was changed, and the new and old values of the policy.
Check Against Baseline: It uses a tool called Maester to compare the current state of Conditional Access policies against a baseline (a set of recommended security configurations). It identifies any discrepancies or failures in maintaining this baseline.
Identify Failures: The query identifies policies that have failed the baseline check by comparing the most recent check results with previous ones. It extracts details about which policies are affected.
Correlate Changes and Failures: It correlates the identified policy changes with the failed baseline checks to determine if the changes are responsible for the failures.
Alert Generation: If a policy change results in a baseline failure, an alert is generated. The alert includes details about the policy change, the failed check, and the specific gap introduced.
Incident Management: The query is configured to create incidents for these alerts, grouping them by account and policy ID. This helps in managing and tracking the incidents effectively.
Severity and Frequency: The severity of the alert is set to medium, and the query runs daily, looking back over the past two days to ensure timely detection of issues.

Overall, this query helps organizations maintain their security posture by ensuring that any changes to Conditional Access policies do not inadvertently weaken their security baseline.

Details

Thomas Naunheim

Released: June 11, 2025

Tables

AuditLogsMaester_CL

Keywords

ConditionalAccessPolicyAuditLogsMaesterAccountIPAddress

Operators

letwherehas=~extendtostringsplitproject-reordersummarizearg_minarg_maxjoinkind=inner!=parse_jsonextract_allprojectmv-expandproject-renamebag_pack_columns

Severity

Medium

Tactics

DefenseEvasionInitialAccess

Frequency: 1d

Period: 2d

Actions

GitHub

KQL Search