Detecting CI/CD pivots before they reach prod

A build runner is a machine that executes arbitrary code, holds credentials for production, and is trusted by everything downstream of it. Most organisations monitor their domain controllers closely and their runners not at all. That asymmetry is the whole problem, and it does not require an attacker to do anything exotic — a pipeline that has been made to run someone else's code behaves almost exactly like a pipeline running yours.

The pivot is rarely dramatic. Someone opens a pull request against a repository whose workflow triggers on pull_request and runs with the repository's secrets. Or a dependency picks up a post-install script. Or a developer's token, scraped from a laptop, is used to push a branch that edits the workflow file that runs on push. From there the runner does what runners do: authenticate to a cloud provider, pull artifacts, sign things, deploy. Every one of those actions is legitimate in isolation. The detection problem is that the runner's normal behaviour is already indistinguishable from an intrusion's objectives.

Baseline first, or none of this works

The three detections below all compare present behaviour against expected behaviour, which means they are useless until you know what a given pipeline is supposed to do. That inventory is the actual work, and it is unglamorous: for each workflow, which secrets does it hold, which external hosts does it legitimately contact, and which artifacts does it produce.

Doing this once produces something more valuable than the detections. Most estates discover during the exercise that a meaningful number of workflows hold credentials they no longer use, contact hosts nobody can explain, and run on triggers that were appropriate two years ago. Fixing that removes more risk than any alert will.

Detection one — egress the pipeline has no reason to make

A build for a Java service talks to your artifact registry, your package mirrors, and your cloud provider's API endpoints. That set is small, stable, and knowable. Anything outside it is either a new legitimate dependency — which someone can confirm in a minute — or the interesting case.

The signal is strongest when scoped per workflow rather than per runner fleet, because a fleet's aggregate egress is the union of every project's needs and is consequently enormous. Per workflow it is usually a dozen hosts.

title: CI runner contacted an unbaselined destination
logsource:
  category: network_connection
detection:
  runner:
    Image|contains:
      - '/runner/'
      - '/actions-runner/'
  known_good:
    DestinationHostname|endswith:
      - '.internal-registry.acme.local'
      - '.blob.core.windows.net'
      - 'registry.npmjs.org'
  condition: runner and not known_good
falsepositives:
  - A genuinely new dependency host. Confirm with the workflow owner, then baseline it.
level: medium

Two implementation notes matter more than the rule text. Resolve and log the destination hostname, not only the address, or a CDN-hosted destination will look like a hundred different things across a week. And apply this to the runner's network namespace specifically — running it across the whole build cluster produces a volume nobody will read past the first day.

Egress detection catches the crude end of the spectrum: pulling a second-stage payload, posting stolen secrets somewhere, opening a channel out. It does not catch an attacker who confines themselves to infrastructure you already trust, which is why it is not the only rule here.

Detection two — token scope drift

This is the highest-value signal of the three and the one most often missing entirely.

Pipeline credentials are issued with a scope. A workflow that builds and pushes a container image needs registry write and nothing else. When that same identity starts enumerating IAM, reading secrets it has never read, or calling services outside its blast radius, the credential is being used for something other than its purpose. The account is authorised, so the cloud provider allows it and returns success. The event is invisible unless you are comparing calls against a per-identity profile.

Build that profile from the audit log. Thirty days of history for a given workflow identity yields a set of API actions that is short and highly repetitive; a build does the same handful of things every time it runs. Alert on first-observation of an action outside it.

let baseline =
    CloudAudit
    | where TimeGenerated between (ago(30d) .. ago(1d))
    | where PrincipalType == "workload_identity"
    | distinct Principal, Action;
CloudAudit
| where TimeGenerated > ago(1d)
| where PrincipalType == "workload_identity"
| join kind=leftanti baseline on Principal, Action
| project TimeGenerated, Principal, Action, Resource, SourceIP

Reconnaissance is what makes this land. Anyone who lands on a runner and does not already know the environment has to look around first, and looking around means listing things. List*, Describe* and Get* calls against services a build has never touched are the first thing that happens and the cheapest thing to catch. An attacker who knows exactly which single API call they want will slip past it — but that attacker had to learn the environment somewhere, and that is a different detection's problem.

The same rule catches a considerable amount of ordinary drift: over-broad roles, copied-and-pasted workflow files, credentials shared between pipelines that should not share them. Treat those findings as the rule paying for itself rather than as noise.

Detection three — the artifact does not match the source

The end state of a compromised pipeline is usually a modified build output, signed by your infrastructure and trusted by everything downstream. If the change is small enough, no human reviews it, because humans review source and not binaries.

Reproducibility is the clean answer and it is hard. Comparing successive builds of the same component is the achievable one: build two adjacent commits, diff the outputs, and expect the delta to correspond to the source delta. A one-line documentation change that alters the dependency tree, adds a network call, or changes the entrypoint is worth a question.

• Diff dependency manifests between builds, resolved rather than declared. A transitive change is still a change.
• Watch for new outbound calls or new spawned processes introduced by a commit that did not add either.
• Alert when a build's output digest changes for a commit that has already been built. Deterministic inputs should produce the same artifact.
• Record which workflow, runner, and identity produced every artifact you promote, so a bad one can be traced back to the run that made it rather than to a rough time window.

A pipeline that can deploy to production is production. Monitor it like production, or accept that you are monitoring the least defended path into it.

Where to start

If you have capacity for one of these, take token scope drift. It requires no agent on the runner, the data is already in your cloud audit log, and it is the only one of the three that catches an attacker operating entirely within infrastructure you trust. Egress baselining is second, and cheap, once the inventory exists. Artifact comparison is the most work and the most durable.

Then test them. A detection nobody has fired deliberately is a hypothesis, not a control — which is the subject of a different note.