What Good Detection Engineering Looks Like

Good detection engineering treats each detection as a small product. It has a reason to exist, an owner, a test, a quality measure and an end date. A large pile of rules does not qualify, however much of it there is.

The pattern in programs that struggle is consistent. Telemetry flows in, rules are added when someone remembers, and nobody can say with confidence what the organization would and would not catch. This explainer describes the practices that change that.

Start from behavior and a response, not from a rule

A detection begins with an adversary behavior the organization cares about, and a clear idea of what an analyst will do when it fires. The backlog should draw from threat intelligence, findings from past incidents, red and purple team results, and the organization’s own exposure, such as its cloud services or identity platform.

MITRE ATT&CK gives teams a shared vocabulary for those behaviors. It works best as an index of what to consider. It works badly as a scoreboard. A detection that nobody knows how to triage is a liability, since it spends analyst time and teaches the team to ignore alerts.

Check the data before writing the logic

Many detections fail before the logic is ever evaluated. The questions are plain. Is the event generated at the source? Is it collected? Is it parsed into the fields the logic needs? Does it arrive soon enough to matter? Is it kept long enough to investigate?

Detecting suspicious application consent grants in a cloud identity platform, for example, depends on the right audit events being enabled and ingested with the relevant fields. If they are not, no amount of clever correlation fixes it. ATT&CK’s data sources and components help state the requirement precisely, and NIST SP 800-92 remains a sound baseline for log management practice.

Write a specification for every detection

A short specification is the difference between a detection and a query someone once wrote. Palantir’s published Alerting and Detection Strategy framework is a widely used starting point, and most teams adapt it. A workable version covers:

• Goal: the behavior being detected and why it matters here.

• Technique mapping: the ATT&CK technique or techniques, with the procedures actually covered.

• Data required: sources, fields and acceptable latency.

• Logic: the rule or analytic, with its assumptions stated.

• Known false positives: what legitimate activity looks similar, and how it is handled.

• Validation: how the detection was tested and when it was last confirmed to work.

• Triage and response: the first steps an analyst takes, and who owns escalation.

• Owner and review date: so the detection does not outlive its usefulness.

Prefer behaviors that cost the attacker something to change

David Bianco’s Pyramid of Pain makes a durable point. File hashes are trivial for an attacker to change. IP addresses and domains are slightly harder. Tools take more effort. The attacker’s underlying techniques are the hardest to replace. Detections built on indicators age quickly. They still have uses, mainly for retrospective hunting and known campaigns, but they should not carry the program.

Behavior-based detections last longer and also produce more noise, because legitimate administrators do unusual things. The usual answer is context: asset criticality, identity risk, the user’s normal pattern. MITRE’s Center for Threat-Informed Defense has published work called Summiting the Pyramid that assesses how robust an analytic is against evasion, which is a useful lens for reviewing existing rules.

Test detections the way engineers test code

Detection-as-code is the practice most associated with maturity, and the reasons are practical. Rules sit in version control. Changes are peer reviewed. Each rule has tests with sample events that should trigger it and events that should not. Deployment runs through a pipeline, and a bad change can be rolled back.

Unit tests on sample events confirm the logic. They do not confirm that the technique, run in your environment, produces those events. For that, emulate the technique with a tool such as Atomic Red Team or MITRE Caldera in a controlled setting, then check the whole chain: the event was generated, the rule fired, the alert reached the right queue and an analyst could act on it. Purple team exercises do the same with the defenders in the room.

Sigma, a vendor-neutral rule format, helps with portability and with sharing rules across platforms. It does not remove the work of mapping to your own log schema and field names, which is where most of the effort sits.

Measure quality and coverage honestly

Coverage numbers are easy to inflate. A single rule that matches one procedure of a technique can turn a heatmap cell green, while the technique has dozens of ways of being carried out. A more truthful view records, for each technique, whether it was tested, whether the data exists, and how confident the team is.

The broader mapping of techniques to evidence deserves its own treatment, and it is covered in the related reading below.

Run the full lifecycle, including retirement

Detections decay. A log format changes, an agent is upgraded, a field is renamed, and a rule that once worked stops firing without any error. Health monitoring on data sources and scheduled re-testing catch this. Without both, silence looks like safety.

Every incident is also a source of backlog. After containment, the useful question is what should have caught this, why it did not, and what data would have helped. Analyst feedback closes the loop in the other direction: which alerts waste time, which lack context, which have unclear response steps. Detections that no longer map to a relevant threat or a live system should be retired. A smaller set of tested rules beats a larger set nobody trusts.

Signs a program is not there yet

• Nobody can say which detections are tested, or when.

• Rules were imported from a vendor or community pack and never tuned.

• Analysts routinely close a rule’s alerts without reading them.

• A new technique takes weeks to reach a working detection, if it arrives at all.

• The team learns a data source broke when an investigation needs it.

Where to start

Pick the five behaviors that would hurt most in your environment. Confirm the data for each exists and arrives in time. Write a specification for each, test it by emulating the technique, and record the result. That small set will show where the process breaks, whether in data, logic, triage or ownership, and tell you what to fix before scaling up.