Cloud Detection Engineering

In the cloud, the main telemetry is the provider’s record of API activity, and much of what you can detect depends on settings someone has to turn on. A rule cannot fire on a log that was never collected.

That changes where the work starts. Cloud detection engineering begins with a log inventory, moves to behavior-based logic, and ends with measurement that shows whether a detection works. Rule writing is the middle step, not the first.

This article covers cloud-specific detection engineering. General practice, coverage mapping and alert overload are covered in the related pieces.

Start with the log, not the rule

Cloud platforms differ in what they log by default. The examples below show why a log inventory comes first.

Providers set a floor, and the floor has been questioned publicly. After the Storm-0558 intrusion into Microsoft’s cloud email service, the US Cyber Safety Review Board noted that a customer, not the provider, raised the alarm. The Board recommended a minimum standard for default audit logging in cloud services. Until that exists, your detection coverage depends on what you add above the default.

Two practical consequences follow. First, data-plane logging has a cost, so spend it where it counts. CloudTrail’s advanced event selectors let you log data events only for the buckets and functions that hold something worth protecting. Second, latency counts. Some logs arrive minutes after the event, and delivery of EKS control plane logs is described as best effort. A detection that fires late has a smaller response window.

Detect behavior, not single calls

Most single cloud API calls are ordinary. Creating an access key, listing buckets and changing a policy all happen daily. Rules that alert on each one produce noise, and teams respond by switching them off.

The signal is in sequence and context. Consider a hypothetical chain in AWS: a new access key created for an existing user, first use of that key from a network the account has never used, a burst of list and describe calls, then a change to logging configuration, such as StopLogging, DeleteTrail or an edit to event selectors. No step is proof alone. Together they describe an intruder discovering the environment and reducing visibility. Tampering with logging deserves a high-priority alert on its own.

Identity-centred logic works well because cloud access is granted through identities: role assumption chains, tokens used from new locations, MFA changes, and consent grants to unfamiliar applications. Build detections around those relationships.

MITRE ATT&CK’s Cloud matrix covers four platforms: IaaS, SaaS, Office Suite and Identity Provider. Use it as a map for planning. Treat a technique as covered only when three things are true: the log source exists, the data arrives in time and a tested detection fires on it.

The engineering lifecycle

1. Choose the threat. Start from attack paths that matter for your environment, not from a list of techniques.

2. Confirm the data. Check that the needed log exists, is retained and arrives quickly enough.

3. Write the logic. Prefer behavior and sequence. Record what the rule assumes about the environment.

4. Test it. Replay sample logs or emulate the behavior in a test account. An untested rule is a hypothesis.

5. Deploy as code. Keep rules in version control with review, tests and rollback. Vendor-neutral formats such as Sigma help where your platforms allow.

6. Tune with real alerts. Fix logic before suppressing alerts. Record why each exclusion exists.

7. Measure and retire. Remove rules that no longer detect anything useful.

Cloud environments change faster than traditional estates. New accounts, services and regions appear without a ticket to the SOC. Detections need an owner who also watches for new log sources and for logging that has been switched off.

Measures that mean something

None of these needs a benchmark. Each is a comparison against your own priorities, taken over time.

Where to start, and who owns it

Pick the three cloud attack paths most likely to cause real harm in your environment. For each, confirm the log exists, arrives promptly and feeds a detection that has been tested. Then fix whichever of the three fails first.

For leadership, the decision is ownership of the pipeline. Platform teams control cloud logging settings. The SOC consumes the output. Detection engineers sit between them. If no one owns the whole path from event to alert, a switched-off log goes unnoticed until the incident that needed it.

Use this to improve detection coverage and quality: choose three attack paths, check the log, the latency and the test for each, and treat what fails as the first backlog.

Sources

• AWS CloudTrail User Guide, Logging data events. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html

• Amazon EKS User Guide, Amazon EKS control plane logging. https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html

• MITRE ATT&CK, Cloud Matrix. https://attack.mitre.org/matrices/enterprise/cloud/

• Cyber Safety Review Board (CISA), report on the Microsoft Exchange Online intrusion attributed to Storm-0558. cisa.gov

• Google Cloud Documentation, Cloud Audit Logs overview. docs.cloud.google.com

• Microsoft Azure Documentation, Azure Monitor activity log and diagnostic settings. learn.microsoft.com