Litatoli evidence format — specification (v1)¶

This specification is licensed under Apache License 2.0, separately and deliberately, while the reference implementation is under the Business Source License (see ../LICENSE).

The reason is the point of the document. An auditor must be able to verify evidence without depending on us, and an independent implementation must be writable by anyone — including a competitor, including a certification body. Restricting the format would make the evidence worth less, not more.

The restriction on the implementation is narrow and elsewhere: you may not run our code as a hosted evidence service for third parties. You may implement this specification and do exactly that.

This document specifies the on-disk evidence format precisely enough to build an independent verifier with no litatoli code. A reference second implementation (Python, blake3 + cryptography only) lives at pimatika/tools/verify_bundle.py and reproduces every check below; it was validated byte-for-byte against real logs. If the CLI and an independent implementation disagree, the bug is in one of them — this spec is the contract.

Everything is UTF-8 text: JSON Lines for the log, JSON for the registry. Primitives: BLAKE3 (keyed MAC + unkeyed digests) and Ed25519 (signatures, RFC 8032). No ASN.1, no certificates, no server.

1. The evidence log — append-only JSON Lines¶

One JSON object per line. Fields of an entry:

Field	Type	Notes
`seq`	u64	strict 1-based, +1 per entry
`prev_hash`	string?	absent on the first entry; else = previous entry's `blake3_signature`
`timestamp`	string	`YYYY-MM-DDTHH:MM:SSZ` (UTC, exactly 20 chars — lexicographic order = chronological order)
`evidence_type`	string	e.g. `command_run`, `ai_inference`, `checkpoint`
`payload`	object	type-specific content
`blake3_signature`	string (hex)	keyed BLAKE3 MAC over the canonical signing payload (§2)
`key_id`	string	first 16 hex chars of BLAKE3(MAC key)
`algorithm`	string	`"blake3-keyed"`
`ed25519_signature`	string (hex)?	Ed25519 over the same canonical signing payload
`ed25519_pubkey`	string (hex, 64)?	the signer's public key, self-contained
`run_id`	string?	correlation id, present iff `payload.run_id` is a non-empty string, lifted to the top level at append so a whole run is queryable without searching the payload

Chain rules (verifiable without any key): - seq at position i (0-based) must equal i + 1; - first entry has no prev_hash; every later entry's prev_hash equals the previous entry's blake3_signature field (string equality). This catches reordering and middle deletion.

Tail truncation is not detectable from the file. Dropping the last N entries leaves a prefix in which seq still runs 1..n, every prev_hash still matches, and every signature is still genuine — because nothing was forged. Checkpoints (§4) do not close this on their own: a cut deep enough to matter removes them too, and what remains is a shorter but perfectly valid log.

A verifier can only exclude truncation by comparing against something recorded outside the file. Two anchors, and an implementation should support at least one:

A pinned head — the last entry's blake3_signature, recorded when the log was closed and supplied to the verifier out of band. Any cut changes it.
A freshness witness (§7) — an external registry that only ever moves a workload's head forward, so a truncated log presents a head behind what the witness already saw.

A verifier MUST report which of these it applied. Reporting only pass/fail lets a reader conclude "nothing was removed" from a check that never looked.

2. Canonical signing payload (what both signatures sign)¶

The MAC and the Ed25519 signature both sign the UTF-8 bytes of one compact JSON object whose field order is fixed (it is a struct, not a map):

{"seq":<u64>,"prev_hash":<null|string>,"timestamp":<string>,
 "evidence_type":<string>,"payload":<value>,"key_id":<string>,
 "algorithm":"blake3-keyed","ed25519_alg":"ed25519","ed25519_pubkey":<string>,
 "run_id":<string>}

prev_hash is always present here (JSON null on the first entry) even though it is omitted from the stored entry.
The two ed25519_* fields are present iff the entry carries an Ed25519 signature — so the MAC also authenticates the key binding (stripping ed25519_pubkey from a stored entry breaks verification).
run_id comes last and is present iff the entry carries one. An entry without it signs to exactly the same bytes as one written before the field existed, so older chains stay verifiable unchanged. When present it is authenticated like every other field: the verifier reads the entry's stored top-level run_id, so stripping or editing it breaks verification — it is not free-floating metadata.
key_id and algorithm are covered by both signatures — the signing payload is the whole entry minus the two signature fields. Flipping either one's value or renaming the field breaks verification; only blake3_signature and ed25519_signature are self-excluded.
payload is serialized with recursively sorted object keys and compact separators (, and :, no spaces), UTF-8 (not ASCII-escaped).
Numbers are rendered exactly as Rust serde_json (ryu): integers verbatim; a float uses its shortest round-tripping decimal, in fixed notation when its scientific exponent E is in [-5, 15] and scientific otherwise (d.dddde±NN, sign always present, exponent not zero-padded, e.g. 1e-7, 0.00005, 1e+20). A naive json.dumps/repr does NOT match (1e-07, 5e-05); an independent verifier must reproduce this rule (the Python reference does, validated on 200 000+ doubles). .inf/.nan cannot appear (not valid JSON).

Third-party verification checks the Ed25519 signature over these bytes with the entry's ed25519_pubkey, then authorizes that key via the registry (§3). The keyed MAC is not third-party-verifiable by design (it needs the operator's secret); it serves the writer's own tamper detection and the chain linkage.

3. Signed key registry (trust anchor + rotation)¶

registry.json:

{ "payload": { "registry_version": "1",
               "root_pubkey": "<hex64>",
               "issued_at": "YYYY-MM-DDTHH:MM:SSZ",
               "keys": [ { "pubkey": "<hex64>", "role": "evidence" | "bim",
                           "not_before": "…Z", "not_after": "…Z"?, "revoked_at": "…Z"? } ] },
  "signature": "<hex — Ed25519 by root_pubkey>" }

Canonical bytes = the payload object as compact JSON with recursively sorted keys; signature is Ed25519 over those bytes by root_pubkey.
The auditor pins root_pubkey out of band — a registry vouching for its own root proves nothing.
A key authorizes an entry at time t (its timestamp) iff not_before ≤ t, t ≤ not_after (when set), and t < revoked_at (when set). All comparisons are plain string comparisons on the fixed 20-char form.

4. Merkle checkpoints (`evidence_type: "checkpoint"`)¶

Payload: { "merkle_root": "blake3:<hex>", "through_seq": N, "entry_count": N }.

The root commits to all prior entries' blake3_signature strings (their ASCII bytes), in seq order, with domain separation:

leaf: BLAKE3(0x00 ‖ leaf_bytes)
node: BLAKE3(0x01 ‖ left ‖ right)
odd level: duplicate the last node;
zero leaves: root = BLAKE3(0x00).

through_seq and entry_count must both equal the number of covered entries (everything before the checkpoint). A checkpoint can then be RFC 3161 timestamped (stamp, sidecar timestamps/<seq>.tsr, imprint = SHA-256(canonical signing payload)), which prevents a later key compromise from silently re-signing history.

5. Interop projections¶

export-attestations maps this format onto in-toto v1 Statements and signed DSSE envelopes (see the CLI reference in the README). That is a projection for Sigstore-family tooling: DSSE verifiers check the per-statement signature; chain continuity (§1), registry windows (§3) and checkpoints (§4) remain this spec's semantics and require a verifier that implements them.

6. Known equivalences an implementer must honour¶

JSON: compact separators, UTF-8 output, object keys sorted recursively (the writer's maps are sorted; the signing struct of §2 is fixed-order).
YAML (BIM side, see pimatika/docs/BUNDLE_FORMAT.md): YAML 1.2 semantics — timestamps stay strings (a YAML 1.1 loader that produces datetime objects will diverge).
Strings are NFC-normalized only where the BIM canonicalization says so; log entries are used as-is.
Hex is lowercase; pubkey comparisons are case-insensitive, trimmed.

7. Freshness witness — the external anchor¶

A witness is an append-only, monotonic registry of heads, keyed by workload. It answers the one question a self-contained file cannot: is this the whole log, or a shortened version of it?

It is also what gives an auditor a signature from something other than the tool under review: the witness signs with its own key, which the producer does not hold.

7.1 `Head` — what a producer submits¶

Six fields, fixed set:

Field	Type	Notes
`workload_id`	string	What the chain belongs to. Monotonicity is enforced per workload.
`seq`	u64	`seq` of the terminal checkpoint entry.
`merkle_root`	string	The checkpoint's Merkle root (§4).
`head_hash`	string (hex)	The checkpoint entry's own `blake3_signature`.
`evidence_pubkey`	string (hex, 64)	Public key of the evidence signer.
`timestamp`	string	The checkpoint's timestamp, verbatim.

7.2 Canonical bytes a witness signs¶

Unlike §2, this is a sorted-key compact JSON object — not a fixed-order struct:

{"evidence_pubkey":…,"head_hash":…,"merkle_root":…,"seq":…,"timestamp":…,"workload_id":…}

Compact separators (, and :, no spaces), keys sorted lexicographically, UTF-8. In Python this is exactly json.dumps(head, sort_keys=True, separators=(",", ":")).encode().

An implementation that re-renders these values rather than reproducing them byte-for-byte will produce a different signature. In particular timestamp is a string and must not be parsed into a date and re-formatted.

7.3 `WitnessedHead` — what a witness returns¶

Field	Type	Notes
`head`	object	The `Head` of §7.1, verbatim as submitted.
`witness_pubkey`	string (hex, 64)	The witness's own key. Pin this.
`witness_signature`	string (hex, 128)	Ed25519 over §7.2 by `witness_pubkey`.
`observed_at`	string	`YYYY-MM-DDTHH:MM:SSZ` — when the witness signed.

7.4 The monotonicity rule¶

For a given workload_id, a witness MUST refuse any submission whose seq is at or below the highest it has already recorded. That single rule is the entire mechanism: a rolled-back or truncated log necessarily presents a lower seq, and the refusal is what makes the cut visible.

A witness MUST NOT overwrite a recorded head, and SHOULD retain every submission rather than only the newest — "what did you see, and when" is the question an auditor brings, and the latest record alone cannot answer it.

Refusal is a distinct outcome from failure. An implementation must let a caller tell apart: the submission lost against a record (refused), this workload has never been seen (no record), and the witness is unavailable (failure). An auditor draws opposite conclusions from the second and the third.

7.5 HTTP protocol¶

Two endpoints, relative to a base URL:

POST {base}/submit                      body: Head (§7.1)
     → 2xx      recorded
     → 409      refused (non-monotonic)
     → 5xx      witness unavailable

GET  {base}/latest?workload_id=<urlencoded>
     → 200      WitnessedHead (§7.3), bare — no envelope
     → 404      no record for this workload
     → 5xx      witness unavailable

GET /latest returns the WitnessedHead unwrapped. A response nested inside a status envelope is not this protocol.

7.6 Comparing a log against a witness¶

Given a log's terminal head and a WitnessedHead for the same workload:

Condition	Verdict
witnessed `seq` > log `seq`	rollback — the log is behind what the witness saw. Reject.
witnessed `seq` = log `seq`, `head_hash` differs	fork — two different chains claim the same position. Reject.
witnessed `seq` = log `seq`, `head_hash` matches	ok
witnessed `seq` < log `seq`	ok, log ahead — not a rollback; the witness may simply be lagging.
no record	unknown — freshness was not established. Not a pass.

A verifier MUST check witness_signature against a pinned witness_pubkey before trusting any of this. A witness vouching for its own key proves nothing, exactly as an unpinned log signed end to end by one key proves nothing.

7.7 A local file registry is not a countersignature¶

The file: backend signs at read time, with whatever key the reader holds. Its witness_signature therefore attests that this reader saw this file — not that some independent party recorded the head when it was submitted. Pinning witness_pubkey against it is meaningless: it is the reader's own key.

A file registry is still useful for rollback detection on one host, because the monotonicity rule (§7.4) is enforced at submission and the file is append-only. What it does not provide is independence, which is the whole point of a countersignature for an auditor.

An independent witness therefore requires a remote registry, signing at submission with a key the producer does not hold. An implementation SHOULD refuse --witness-pubkey against a file: backend rather than appear to enforce a pin it cannot.

8. What a verification result must state¶

Two independent axes. A result that collapses them is misleading even when every check passed:

Axis	Established by	If absent
Who signed	A pinned `ed25519_pubkey`, or a registry (§3) rooted in a pinned trust anchor	The log is only self-consistent — which a log forged end to end by one key also is
Whether anything is missing	A pinned head, or a witness (§7)	Entries cut from the end are invisible

An implementation MUST report which axes a given run established, in the result itself. A caveat that lives in documentation is not delivered: the reader of a green check does not open the manual.