KQL Search

Query Details

HomeAI ToolsDevice Query

Noise Syslog Facility

Query

// Syslog - Facility & Severity Noise Analysis
// Identifies high-volume Syslog facilities and severity levels
// that are candidates for DCR filtering. Informational and Notice
// logs are typically >60% of Syslog volume with low detection value.
// =====================================================================

let _Window = 7d;
let _TotalSyslog = toscalar(Syslog | where TimeGenerated > ago(_Window) | count);
// --- By Severity ---
let _BySeverity =
    Syslog
    | where TimeGenerated > ago(_Window)
    | summarize 
        EventCount = count(),
        DistinctHosts = dcount(HostName),
        DistinctFacilities = dcount(Facility)
        by SeverityLevel
    | extend 
        PctOfTotal = round(toreal(EventCount) * 100.0 / _TotalSyslog, 1),
        FilterAction = case(
            SeverityLevel in ("info", "notice", "debug"), "FILTER: Low-value severity - Strong DCR exclude candidate",
            SeverityLevel == "warning", "REVIEW: Evaluate if all warnings are needed",
            "KEEP: Security-relevant severity"
        ),
        EstDailyGB = round(toreal(EventCount) / 7.0 * 0.0005 / 1024.0, 3)  // rough ~500 bytes/event
    | project Section = "SEVERITY", Name = SeverityLevel, EventCount, PctOfTotal, EstDailyGB, DistinctHosts, FilterAction;
// --- By Facility ---
let _ByFacility =
    Syslog
    | where TimeGenerated > ago(_Window)
    | summarize 
        EventCount = count(),
        DistinctHosts = dcount(HostName),
        SeverityBreakdown = strcat(
            "err:", countif(SeverityLevel in ("err", "crit", "alert", "emerg")),
            " warn:", countif(SeverityLevel == "warning"),
            " info:", countif(SeverityLevel in ("info", "notice", "debug"))
        )
        by Facility
    | extend PctOfTotal = round(toreal(EventCount) * 100.0 / _TotalSyslog, 1)
    | extend FilterAction = case(
        Facility in ("cron", "ntp", "lpr", "uucp", "ftp") and PctOfTotal > 2, "FILTER: Non-security facility, high volume",
        Facility == "kern" and PctOfTotal > 15, "REVIEW: Kernel logs often noisy, evaluate host scope",
        Facility == "daemon" and PctOfTotal > 15, "REVIEW: Daemon logs - filter by process name in DCR",
        Facility in ("auth", "authpriv"), "KEEP: Authentication logs - security critical",
        PctOfTotal < 1, "OK: Low volume",
        "REVIEW: Evaluate detection coverage"
    )
    | project Section = "FACILITY", Name = Facility, EventCount, PctOfTotal, EstDailyGB = 0.0, DistinctHosts, FilterAction;
// --- By Facility + Severity combo (top noise) ---
let _TopCombos =
    Syslog
    | where TimeGenerated > ago(_Window)
    | summarize EventCount = count() by Facility, SeverityLevel
    | extend PctOfTotal = round(toreal(EventCount) * 100.0 / _TotalSyslog, 1)
    | where PctOfTotal > 3
    | extend FilterAction = case(
        SeverityLevel in ("info", "notice", "debug") and PctOfTotal > 10, "FILTER: High-vol low-severity combo - Top DCR candidate",
        SeverityLevel in ("info", "notice", "debug") and PctOfTotal > 5, "FILTER: Significant low-severity volume",
        "REVIEW"
    )
    | project Section = "TOP_COMBO", Name = strcat(Facility, "/", SeverityLevel), EventCount, PctOfTotal, EstDailyGB = 0.0, DistinctHosts = 0, FilterAction
    | top 15 by EventCount desc;
_BySeverity | union _ByFacility | union _TopCombos
| order by Section asc, EventCount desc

Explanation

This query is designed to analyze Syslog data over the past 7 days to identify high-volume sources that might be candidates for filtering, particularly focusing on facilities and severity levels that generate a lot of noise but have low detection value. Here's a simplified breakdown of what the query does:

  1. Total Syslog Count: It calculates the total number of Syslog events in the last 7 days.

  2. Analysis by Severity:

    • It counts the number of events, distinct hosts, and distinct facilities for each severity level.
    • It calculates the percentage of total events each severity level represents.
    • It suggests actions based on severity:
      • "info", "notice", "debug": Suggests filtering these as they are low-value.
      • "warning": Suggests reviewing these to see if they are necessary.
      • Other severities: Suggests keeping them as they are security-relevant.
    • It estimates the daily data volume generated by each severity level.

  3. Analysis by Facility:

    • It counts the number of events and distinct hosts for each facility.
    • It provides a breakdown of severity levels within each facility.
    • It calculates the percentage of total events each facility represents.
    • It suggests actions based on facility:
      • Certain non-security facilities with high volume are suggested for filtering.
      • High-volume kernel and daemon logs are suggested for review.
      • Authentication logs are suggested to be kept as they are critical for security.
      • Low-volume facilities are marked as okay.
    • It estimates the daily data volume generated by each facility.

  4. Top Facility and Severity Combinations:

    • It identifies the top combinations of facility and severity level that contribute significantly to the total volume.
    • It suggests filtering high-volume, low-severity combinations.
    • It lists the top 15 combinations by event count.

  5. Final Output:

    • It combines the results from the severity, facility, and top combinations analyses.
    • It orders the results by section (severity, facility, top combinations) and event count.

Overall, the query helps in identifying and suggesting actions for reducing noise in Syslog data by focusing on high-volume, low-value events.

Details

David Alonso profile picture

David Alonso

Released: April 8, 2026

Tables

Syslog

Keywords

SyslogFacilitySeverityHostNameTimeGeneratedEventCountDistinctHostsDistinctFacilitiesSeverityLevelFacilityFilterActionEstDailyGB

Operators

lettoscalarwherecountsummarizedcountextendroundtorealcaseprojectstrcatcountiftopunionorder by

Actions