Skip to main content
{ Security Operations }

Datadog Detection Tuning

Finds the Datadog Cloud SIEM detection rules that produce the most noise, reads each one against the signals it raised, and proposes a specific tuning change with the evidence and what it would have missed.

What this agent does

This read-only agent ranks the detection rules in Datadog Cloud SIEM by the noise they produce. It measures each rule's signal volume, the share archived as false positive, the time to close, and the entities that repeat. It reads the rule and the signals behind the noise, and it proposes one tuning change per rule. The change is a query filter, a threshold, a suppression rule, a group-by change, or a longer signal duration. Each proposal shows the signals it would have removed and the true positives it would have hidden. A person applies the change.

The challenge

A handful of detection rules produce most of the signals, and most of those are archived the same way every day. Analysts know which rules are noisy. A safe suppression needs the logs behind hundreds of signals read first, and the triage queue takes that time every day. A suppression written in a hurry hides a real detection along with the noise.

The solution

The agent finds the attribute values behind each noisy rule's signals. It proposes the narrowest change that removes them. It replays the changed rule as a Historical Job to show what the change would have hidden. It presents every proposal as a candidate and changes nothing itself. Provides a ranked tuning queue where each item is ready for a detection engineer to review and apply.

Workflow

  1. 01

    Measure noise

    For each detection rule, count signals in the window, the share archived as false positive, time to close, and repeated entities.

  2. 02

    Rank

    Rank rules by false positive and benign count times median time to close, and take the top 5.

  3. 03

    Read the signals

    Read the signals and the logs that triggered them for each rule, and find the attribute values that explain the noise.

  4. 04

    Propose

    Write the narrowest change that removes the pattern. Run the proposed rule as a Historical Job over the window, and say how many days of the window the indexed logs cover.

  5. 05

    Report

    Publish the ranked queue with each proposal and its evidence.

Agent template 

# Datadog Detection Tuning

## Measurable outcomes

Every run produces a ranked list of noisy detection rules, each with one specific proposal and the evidence for it. Track the signal volume and the false positive share of each rule on every run. On the next run, compare each tuned rule's volume and false positive share with the run before its last change.

## Procedure

Each run, review the last 30 days of security signals in Datadog Cloud SIEM, unless I set another window. For each detection rule, count the signals it raised and the share archived with a false positive or other benign reason. Also count the median time to close and the number of distinct and repeated hosts, users, and source addresses. Report the share of closed signals with no disposition. Rank on volume alone when that share is above half. Otherwise, rank rules by false positive and benign count times median time to close. Take the top 5, unless I set another number. Skip rules I mark as tuned recently. For each one, read the rule's query, cases, group-by attributes, and existing suppressions, and read its signals with the logs that triggered them. Find the attribute values that explain the noise. Typical causes are a service account, a scanner's address range, a deploy window, a hostname pattern, or a threshold the normal baseline crosses. State the pattern as attributes, values, and the share of signals they cover. Propose the narrowest change that removes that pattern, and show the exact text. The change can be a filter added to the query, a higher threshold in the case, or a suppression rule with its exact query. It can also be a different group-by. Where one actor opens many signals, it can be a longer keep alive or maximum signal duration. Run the proposed rule as a Historical Job over the window, and say how many days of the window the indexed logs cover. Report the signals it would have removed and every true positive or open signal it would have hidden. For each proposal, name what an attacker could do inside the excluded scope without an alert, such as acting as the excluded service account. When no change removes the pattern without hiding a true positive, say that no safe change exists. Never propose a change that hides a true positive without saying so. Never propose disabling a rule. Present every proposal as a candidate, and never change a rule or create a suppression. When a rule's noise comes from a log quality problem, say so and point at the pipeline or source instead of proposing a filter.

## Requirements

It needs read-only Datadog API access to security monitoring rules, suppressions, signals, and the logs in scope, and permission to run Historical Jobs, and nothing more. It never changes rules, suppressions, or signals. Reports quote only the field values a proposal excludes, with raw events, other users, hosts, and alert IDs left out.