To manage risk, first line teams will typically have to prove a control was applied (and presumably protected us from some risk). It’s common that an auditor would ask for “evidence” and for the most part, they will look in some sort of compliance database (aka Compliance System of Record) for that evidence. More often than not, that’s ServiceNow.

Auditors look for evidence retrospectively to understand if a control was executed and is effective

The problem is that most of the time, the evidence doesn’t have any context. It’s the output of some process with no advocacy how it was produced or what policy or process it actually followed.

Evidence != Proof

If you want to check I travelled to Brighton, you could look at my train ticket but you’d have no way to know if I actually travelled. If you have actual proof, perhaps CCTV footage of me boarding and departing the train, you wouldn’t actually need the ticket stub. The ticket stub is evidence, but it doesn’t prove I travelled. The CCTV footage is proof.

In modern governance systems, if we demonstrate the control can not be avoided (e.g. show a sealed pipeline is the only way to achieve an action), then we don’t need to rely on the output so much. Audit the system, not the output.

Attestations

Attestations in the general English sense have been a way for teams to “evidence” that they have done something (like met a control standard). We’re used to a human saying “I attest” or “I approve” and use it as evidence in the way it was described above. Audit functions rightly point out the weaknesses of this in the risk world - we can’t rely on the human. So “attestation” becomes a naughty word in the GRC world and first line teams strive for more automation.

However, attestations as a term can take on a much stronger meaning when we look at SLSA and modern Governance Engineering. In this context, an attestation is much more formal.

Attestations are trusted and verifiable statements or metadata about a software artifact or group of artifacts. These statements help us answer important compliance questions.

Attestations are observations made against events. They are opinionless. They are just statements that something happened, usually against some form of artifact. For example, the code was compiled. They capture metadata about the event, for example, the version of the compiler used but they do not determine compliance. Attestations provide inputs needed to evaluate compliance later. To be able to evaluate them, you need to know the policy. Together this can form a coherent set of steps to implement a control.

To continue the example, we might have a policy which says only version 1.2 or above of the compiler is allowed. We record what version was used at the point of compilation and they later feed the policy that metadata to decide if we are compliant or not.

Key points:

• Attestations are facts
• Attestations are opinionless, they declare something has happened and don’t care what impact that has on compliance
• Attestations are distinct from governance, they must be applied to policy to derive compliance

Policy

When we ask if a control is compliant (passes or fails our governance tests), we are evaluating the control. We need to have enough information from the attestations in order to do that. Simplistically, the result of an evaluation is always pass or fail.

The way we capture the governance tests that must pass or fail, is in the policy. It’s just a set of rules that define what is considered compliant or non-compliant. It takes the attestations as input and applies the rules to determine if the control meets the required standards. Rego is an expression language that allows us to write these rules in a structured and machine-readable way which really means we can write policy-as-code. When we evaluate a Rego policy, we pass in attestations and get a compliance status back.

Key points:

• Evaluations consume attestations, they do not alter them (an evaluation itself however, might produce new attestations)
• Because attestations are artifact-bound, evaluations are made against a specific artifact or group of artifacts
• Evaluations are always either a pass (compliant) or fail (non-compliant)

The point around artifact-binding is interesting. It means evaluations prove something is true for a given artifact and that is a powerful tool when we want to demonstrate provenance of artifacts, especially to a chain of compliance statements.

Evidence

Given the above, evidence is the supporting documents or materials which summarise the evaluation of policy. It should be the information that helps a person or system evaluate whether a claim is true. The issue I pointed out earlier is that in many governance systems, the evidence often doesn’t prove the claim. By building evidence that includes inputs, specific policy (think versioned) and the result of the evaluation, we can provide a more complete picture. Evidence like this captures the state of the system at a specific point in time, making the process transparent rather than opaque. That means evidence can be replayed and independently verified. Honestly, this is what auditors never knew they needed.

Pulling it Together

Attestations record what happened. Policy defines what should have happened. Evaluation compares the two. Evidence is the durable record of that comparison, it’s really a side effect. It has the inputs, the versioned policy and the result, bound to a specific artifact and point in time.

Which brings us back to the auditor and the compliance database. The problem was never that there’s too little evidence, it’s that what’s there arrives with no account of how it was produced. A ticket stub, not CCTV.

Get the chain right and nobody has to argue about the output, because anyone can replay the decision and get the same answer. Audit the system, not the output.