Policy evaluation
A CBOM records facts. A judgement about those facts (for example, whether an interface is post-quantum-safe) is a separate matter. Such judgements should be produced by an external policy applied to the facts at evaluation time, rather than stored as attributes within the CBOM, because the criteria on which they depend change over time.
Facts versus evaluations
A CBOM states facts: the protocols, versions, algorithms, key material, and certificate signature algorithms an interface uses. These are primary data, observed or declared by the producer.
An evaluation is a judgement derived from those facts against a set of criteria. "Is this interface post-quantum-safe?" is not a fact recorded by the producer. The answer is a conclusion drawn from the algorithms the interface uses, assessed against current guidance. The conclusion depends on criteria that are external to the facts and that are revised as guidance develops.
Why the evaluation must be external to the data
If a derived value such as post-quantum posture is stored as an attribute within the CBOM, three problems follow:
- The value becomes stale. When the criteria change (for example, when an algorithm is reclassified), the stored value no longer reflects current guidance, yet the CBOM presents it as though it did.
- Producers evaluate inconsistently. Different producers apply different or outdated criteria. They therefore record different verdicts (conclusions such as "post-quantum-safe") for equivalent facts, and their CBOMs cannot be compared.
- The record and the judgement are conflated. A CBOM is intended to be a durable record of facts. Embedding a time-dependent judgement in it undermines that role.
Keeping the evaluation external avoids these problems. The CBOM remains a stable record of facts. A single, versioned policy evaluates all CBOMs consistently. The same unchanged facts can be re-evaluated at any time under the criteria in force at that time.
Examples of evaluations
| Evaluation | Derived from (facts) | Against (criteria, external) |
|---|---|---|
| Post-quantum posture | the key-exchange and authentication algorithms an interface uses, and whether they are classical, hybrid, or post-quantum | current guidance on which algorithms and combinations are considered quantum-safe, which is revised over time |
| Algorithm transition | every algorithm a CBOM discloses, with its family and security strength | a dated transition schedule, such as the one in the NIST IR 8547 initial public draft |
| Readiness gap | what the product record says the product can do, what the configured record permits, and what the observed record saw, joined one interface at a time | a target, such as ML-KEM for key establishment and ML-DSA or SLH-DSA for entity authentication |
| Cryptographic maturity | algorithm strength, key sizes, deprecation status, library currency, and cryptographic agility | a maturity rubric that is revised as recommendations and deprecations change |
All of these evaluations draw on the same kind of primary data, and the same point applies to each. Their criteria are not fixed. The evaluation must therefore be applied externally and dated, rather than frozen into the CBOM. The working group has written the algorithm-transition and readiness-gap evaluations as concrete example policies. No example policy exists yet for cryptographic maturity.
Vulnerability status is another case. It is the case where the pressure to store a verdict in the CBOM is strongest, because the format offers a field for it. Vulnerability status also differs from the evaluations in the table. Its criteria can be revised, and in addition someone other than the producer holds them: whether an interface is affected depends on an advisory that a third party publishes. The same reasoning applies. Vulnerabilities works through the consequences. It also sets out what a profile must require so that a consumer can match a CBOM against an advisory at all.
Consequence for the profiles
For this reason, the working group removed pqcPosture from the baseline's attributes. The attribute was carried in an early working draft, before v0.1, and nobody ever authored a document against that draft. The changelog records only changes between published versions. The removal therefore appears in the profile specification's revision rationale rather than in its changelog. Decision 0002 records the removal. The profile requires disclosure of the facts from which posture is derived. These facts are the protocol, the key exchange, encryption and authentication algorithms, and the implementing library. An external policy computes the posture itself at evaluation time. The same applies to any cryptographic maturity assessment. The baseline lists derived judgements among its exclusions for the same reason.
Policies are artefacts
Decision 0002 is now concrete. A policy is a file in the repository. It holds versioned data plus a named evaluation method. The data says what the criteria are. The method says how an evaluator applies them. Policies sit outside CBOMs and outside profiles, because the criteria change while the facts do not.
| Artefact | File (in docs/methodology/) | What it is |
|---|---|---|
| Policy format | policy.schema.json | The structure of a policy: identifier, version, status, source, method, a classification of algorithm names, and either a dated schedule or a set of targets per purpose. |
| Readiness-gap policy | policies/policy-readiness-gap.policy.json | Example policy pkic.example.policy.readiness-gap, version 0.1, method readiness-gap. |
| Algorithm-transition policy | policies/policy-algorithm-transition.policy.json | Example policy pkic.example.policy.algorithm-transition, version 0.1, method algorithm-transition. |
| Edge-posture policy | policies/policy-edge-posture.policy.json | Example policy pkic.example.policy.edge-posture, version 0.1, method edge-posture. |
| Evaluator | evaluate_policy.py | Implements the three methods. Usage: evaluate_policy.py <policy> <cbom...> --date YYYY-MM-DD. |
| Result format | policy-result.schema.json | The structure of a result. |
| Results | examples/results/ | The results for the Keycloak records, all evaluated on 2026-10-25. |
Both example policies carry the status illustrative. They show the mechanism. They are not advice.
What a result records
A result is the judgement, written down. It records five things:
- the policy it applied, by identifier, version and SHA-256 digest of the policy file;
- the evaluation date;
- every CBOM it read, each by file name, serial number, version, subject and SHA-256 digest;
- the findings, and a summary count per class or status;
- what the result does not assert.
The last item matters as much as the findings. Every example result states that it does not assert that the disclosed values are true, because the evaluator applied the policy to what the CBOMs state. It asserts nothing about interfaces or algorithms the CBOMs do not disclose. It does not assert that the policy is right. It asserts nothing after the evaluation date, because the same facts give a different result under a later date or a later policy version.
The direction of reference follows time. A CBOM exists before anyone evaluates it, so the result points to the CBOM by digest. A CBOM never points to a result. Context Pointers applies the same rule to conformance claims.
The evaluation date is fixed in the examples, at 2026-10-25. The test suite runs each evaluation again at that date and checks that the output is identical to the published result. An evaluator that used the current date would produce a different result each day. A real evaluation states its own date for the same reason: so that anyone can reproduce it.
Two evaluation methods
Algorithm transition
The algorithm-transition method reads every algorithm a CBOM discloses at an interface: key exchange, encryption and authentication, and, where present, the supported sets. It looks each name up in the policy's classification, which gives its family, its security strength in bits, and whether a quantum computer could break it. It then applies the policy's schedule at the evaluation date. Each finding states the status on that date and, where there is one, the next transition and its date.
The example schedule follows the NIST IR 8547 initial public draft (November 2024). NIST had not published a final version as of October 2026. Under that draft, a quantum-vulnerable algorithm at 112-bit strength is deprecated after 2030 and disallowed after 2035. A quantum-vulnerable algorithm at 128 bits or more is disallowed after 2035. The policy repeats this schedule as data, and its source block names the draft. Check the schedule against the NIST publication before relying on it.
The method does not classify a literal. It reports none as not-applicable, deployment-defined as not-fixed-by-product, and peer-defined as decided-by-peer.
Readiness gap
The readiness-gap method joins a product record (implemented), a configured record and an observed record along correspondsTo. Lifecycle Data describes that join. For each interface, the method looks at two purposes: key establishment and entity authentication. The policy names the target for each purpose: ML-KEM for key establishment, and ML-DSA or SLH-DSA for entity authentication. The method then places each interface and purpose in one class.
| Class | Meaning | Who acts |
|---|---|---|
target-met | Every value observed (or, without an observation, every value configured) meets the target. | Nobody. |
enabled-not-negotiated | The deployment permits a target value, and at least one session still negotiated a classical value. | The other endpoint or a fallback path is the limit. |
capable-not-enabled | The product can do it, and the deployment does not permit it. | The operator, by configuration. |
capable | The product can do it. No deployment record was supplied. | Not evidenced: without a deployment record, nobody knows whether it is enabled. |
not-capable | The product cannot do it. The product record says what blocks it. | The supplier, or the operator by upgrading or replacing the product. |
unprotected | The interface applies no cryptography at all at the latest stage supplied. | Not fixed by the class. The traffic is exposed whatever the product can do. |
not-deployed | The product has the interface, and the deployment records do not. | Nobody, in this deployment. |
not-applicable | No algorithm does this job at this interface. | Nobody. |
unknown | A stage needed to decide is missing, or declares the value unknown. | Whoever can supply the missing record. |
The readiness gap is computed from the join each time. No CBOM stores it.
Edge posture
The edge-posture method joins the two ends of a connection along peerInterface (deployment disclosure I4, decision 0026). Each end is an interface in its own operator's records. The configured record says what the end permits. The observed record, where one is supplied, says what was negotiated. For each purpose the method places the connection in one class, and names the end that limits it where there is one.
| Class | What it means |
|---|---|
target-met | Every value negotiated meets the target. Without an observation, both ends permit a common value that meets it. |
limited-by-end | One end permits a target value and the other does not. The evaluation names the end that does not, with what its product record says blocks it. |
neither-end-permits | Neither end permits a target value. Both have to change. |
fallback-or-intermediary | Both ends permit a target value, and a session negotiated one that does not meet it. The limit is a fallback path or something between the ends. |
no-common-target | Both ends permit a target value, but not the same one. |
limit-unknown | An end has no configured record, so the limit cannot be placed. |
The readiness gap compares one interface across stages. The edge posture compares the two ends of one connection. Where the readiness gap says the limit is the other endpoint, the edge posture says which endpoint that is.
The Keycloak results
The working group evaluated the Keycloak records with both example policies on 2026-10-25. The operator, Example Org, is illustrative.
Result (in examples/results/) | Records read | Summary |
|---|---|---|
result-readiness-gap-sso-prod.json | product, configured and observed records | 28 findings: 14 not-capable, 4 unprotected, 4 not-deployed, 4 not-applicable, 1 enabled-not-negotiated, 1 unknown. |
result-algorithm-transition-keycloak-product.json | product record | 156 findings. 126 algorithms are acceptable on the evaluation date. Of these, 53 are quantum-vulnerable at 112 bits and become deprecated after 2030, and 67 are quantum-vulnerable at 128 bits or more and become disallowed after 2035. 13 values are decided-by-peer, 2 are not-fixed-by-product, and 15 are not-applicable. |
result-algorithm-transition-sso-prod-observed.json | observed record | 31 findings. 18 algorithms are acceptable on the evaluation date. 6 of them (RSA-2048, RS256 and RSA-SHA256) become deprecated after 2030, and 6 become disallowed after 2035. 13 values are not-applicable. |
result-edge-posture-sso-db.json | the Keycloak product, configured and observed records, and the database team's configured record | 2 findings for the connection from Keycloak to PostgreSQL. Key establishment is limited-by-end: the database permits X25519MLKEM768, and Keycloak's int-db does not, because its Java provider is blocked (planned, JEP 527). Entity authentication is neither-end-permits. |
The readiness-gap result says who acts. Key establishment at the ingress is enabled-not-negotiated: the configured record permits X25519MLKEM768, and 29% of sessions still negotiate X25519 alone. The plaintext hop from the ingress to Keycloak and the management port are unprotected, so Example Org acts. Token signing is not-capable: ML-DSA signing is committed upstream (issue #43690) and blocked by the JOSE standard. Keycloak's own TLS is not-capable: post-quantum key exchange is planned and blocked by the Java runtime (JEP 527). The product's HTTPS listener is not-deployed, because TLS ends at the ingress. Lifecycle Data shows the findings as a table.
The algorithm-transition results show why the date belongs in the result. On 2026-10-25 every classified algorithm is acceptable. Evaluated on 2031-01-01 with the same policy, the six 112-bit values in the observed record (RSA-2048, RS256 and RSA-SHA256) are deprecated. The facts in the CBOM are the same.
Relationship to profiles and governance
An evaluation policy is versioned, dated, and governed in the same manner as a profile (see Versioning and Governance). A conformance claim (a producer's statement that a CBOM conforms to a profile) references the profile version. In the same way, a policy result references the policy identifier, version and digest, and the date on which the evaluator applied the policy. Anyone can then reproduce the result and audit the basis on which it was made.