How to Verify a Coding Agent’s Runtime Evidence

01Subject

Identify the process that was actually observed.

Begin with the launcher’s record of the intended process. A process name, a test-suite name or the first matching entry in a process list is not enough to distinguish it from another worker. For a Linux snapshot, useful checks include the process ID, start time, user identity, executable path and a digest of the argument bytes. Record when those checks occurred and whether the identity was checked again after collection.

A run label supplied to a collector is an association to investigate. It does not prove that this process performed that run. Connect it to the actual launch record and the retained attempt. Keep argument bytes private when they contain sensitive information; a digest does not justify publishing the original command line.

• Match the intended launch and attempt
• Check identity before and after collection
• Keep supplied labels distinct from observed facts

02Capture

Check who could write the evidence.

The process being evaluated should not be able to replace the report or choose a convenient set of measured files. Review the collector’s identity, its source version, its output directory and the permissions of that directory’s ancestors. A protected report written by a separate host observer answers a different question from a JSON file produced by the candidate itself.

File collection also has limits. A bounded snapshot can record mapping paths, device and inode identity, permissions, sizes and content hashes, then compare them after reading. Repeated byte reads can detect changes that metadata checks miss. Matching observations still do not prove that no transient write occurred between them. The collector and host require their own trust review.

03Origin

Follow the signature chain all the way to the file bytes.

An installed package name is a lookup hint. To check a distribution claim, retain the signed repository metadata, the package index identified by that metadata, the exact package archive and the file selected from that archive. Verify the signature, the signed index hash, the archive’s hash and version, and finally the selected regular file’s size and content hash against the observation.

A signature is checked using a particular key. A supplied keyring and a matching signature do not establish that an independent reviewer approved that publisher key. Record the fingerprint and the policy used to trust it. Metadata age is another decision: an old signed index may help reconstruct an earlier installation while still being unsuitable for approval of a current deployment.

• Retain original signed metadata and package bytes
• Match exact version and architecture
• Name the archive member that matched
• Review publisher-key trust and freshness separately

04Example

Two matching libraries do not establish the whole runtime.

Consider an illustrative capture with three file-backed mappings: an executable, a system library and a dynamic loader. The distribution replay matches the library and loader to regular files inside one signed package archive. Its useful conclusion is that those two observed backing files match the checked distribution bytes. The executable remains outside that proof until its own origin and contents are checked.

Now suppose someone replaces the observed library hash and updates the surrounding manifest. A separately retained observation digest should detect that substitution. If the observation itself is deliberately re-reviewed under a new digest, the package-member comparison must still detect the changed file bytes. Test both boundaries; passing only the outer checksum check leaves the inner comparison untested. This example describes a review method, not a reported TraceAxiom production run.

05Coverage

Write down what the snapshot cannot see.

Backing-file hashes are not a measurement of every byte in loaded memory. Anonymous mappings, generated code and application imports may require other collection methods. A native file-mapping snapshot does not enumerate all JavaScript modules or show that a model’s context, credential boundary or network behavior was correct. Read the collector’s declared scope before treating a result as runtime verification.

Review the files outside the proof as carefully as the matches. An empty excluded-files list can reflect a narrow collection method rather than complete coverage. Ask which inputs should exist for this workload, which were observed, which were verified and which remain unknown. Preserve the unknowns in the record passed to the next reviewer.

• Backing files and loaded memory are different subjects
• Package provenance does not prove application behavior
• A test worker is not evidence of genuine provider execution
• Missing coverage remains visible in the decision

06Decision

Use a reproducible bundle to inform approval.

Keep the original observation, its separately reviewed digest, the collector source identity, signed distribution inputs, file bindings and replay result together. The replay should work from those retained bytes without silently substituting current downloads. If the original mirror no longer serves an exact package, an archive can help retrieve it; the retrieved bytes still need the same signed-hash checks.

Finish the review with a concrete statement of scope: which process was observed, which files matched, which trust assumptions were approved and which requirements remain open. A successful replay supplies evidence for that decision. It does not create execution authority, approve a customer deployment or independently accept a model-produced change.