Tifcepillar1 Data

Query

// ================================================
// TIFCE Pillar 1: Originality Score (Uniqueness)
// ================================================
// Purpose:
// - Measure how much a feed contributes indicators that are not already
//   broadly present across other feeds.
//
// Scoring logic:
// - Each IOC gets a weight of 1 / FeedCount
// - If only one feed has the IOC, it contributes 1.0 to that feed
// - If 5 feeds share the IOC, it contributes 0.2 to each feed
// - Feeds with more exclusive IOCs get higher originality scores

// Step 1: Build a deduplicated set of active IOCs per feed
let ActiveIndicators =
    ThreatIntelIndicators
    | where IsActive == true
      and IsDeleted == false
      and (isnull(ValidUntil) or ValidUntil > now())
    | where isnotempty(ObservableKey) and isnotempty(ObservableValue)
    | extend
        // Feed identifier used for scoring
        TIFeed = tostring(SourceSystem),
        // Canonical IOC format used consistently across the query
        // Normalization helps avoid mismatches caused by casing or extra spaces
        IOC = strcat(
            tolower(trim(" ", tostring(ObservableKey))),
            ":",
            tolower(trim(" ", tostring(ObservableValue)))
        )
    // Deduplicate so the same feed-IOC pair is only counted once
    | summarize by TIFeed, IOC;
// Step 2: Count how many distinct feeds report each IOC
let IOCFeedCounts =
    ActiveIndicators
    | summarize FeedCount = dcount(TIFeed) by IOC;
// Step 3: Join feed IOCs to IOC distribution and calculate originality
ActiveIndicators
| join kind=inner IOCFeedCounts on IOC
| summarize
    // Sum of fractional IOC contributions for the feed
    OriginalityScore = sum(1.0 / FeedCount),
    // Total distinct IOCs contributed by the feed
    TotalIOCs = count(),
    // Count of IOCs seen only in this single feed
    ExclusiveIOCs = countif(FeedCount == 1)
    by TIFeed
| extend
    // Average originality contribution per IOC
    AvgOriginalityPerIOC = round(OriginalityScore / TotalIOCs, 4),
    // Same idea expressed as a percentage for easier comparison
    OriginalityPct = round(100.0 * OriginalityScore / TotalIOCs, 2)
| order by OriginalityScore desc
| project
    Feed = TIFeed,
    OriginalityScore,
    TotalIOCs,
    AvgOriginalityPerIOC,
    OriginalityPct,
    ExclusiveIOCs

Explanation

This query is designed to evaluate the uniqueness of threat intelligence feeds by calculating an "Originality Score" for each feed. Here's a simplified breakdown of what the query does:

Purpose: The goal is to measure how much unique information (indicators of compromise, or IOCs) each feed provides compared to others. Feeds that provide more exclusive IOCs receive higher scores.
Scoring Logic:
- Each IOC is given a weight based on how many feeds report it. If an IOC is reported by only one feed, it contributes fully (1.0) to that feed's score. If it's shared by multiple feeds, its contribution is divided among them.
- Feeds with more exclusive IOCs (those not reported by other feeds) get higher originality scores.
Steps in the Query:
- Step 1: Create a list of active, non-duplicate IOCs for each feed. This involves filtering out inactive or deleted IOCs and normalizing the data to ensure consistency.
- Step 2: Count how many different feeds report each IOC.
- Step 3: Calculate the originality score for each feed by joining the IOC data with the feed count data. This step involves:
  - Summing the fractional contributions of IOCs for each feed.
  - Counting the total number of distinct IOCs each feed contributes.
  - Counting how many IOCs are exclusive to each feed.
  - Calculating the average originality contribution per IOC and expressing it as a percentage for easier comparison.
Output: The query orders the feeds by their originality scores in descending order and presents the results, including:
- The feed identifier.
- The originality score.
- The total number of IOCs contributed.
- The average originality per IOC.
- The originality percentage.
- The count of exclusive IOCs.

In summary, this query helps identify which threat intelligence feeds provide the most unique and valuable information by calculating and comparing their originality scores.

Details

Michalis Michalos

Released: June 3, 2026

Tables

ThreatIntelIndicators

Keywords

ThreatIntelIndicators

Operators

let|whereandorisnullnowisnotemptyextendtostringstrcattolowertrimsummarizedcountjoinkindsumcountcountifbyroundorder byproject

Actions

GitHub