00Field guide / code verification
AI Code Verification: How to Check AI-Generated Pull Requests
AI code verification is not a second model saying the first model looks correct. It is a chain of independently executable checks that identify the exact candidate, preserve failures, and separate technical verdicts from human acceptance.
01Subject
Begin by identifying exactly what is being verified.
A verdict is only useful when it names its subject. For a source candidate, that means hashing the actual files and bytes under review with deterministic ordering and explicit inclusion rules. The build output, browser run, security scan, and acceptance evaluation must all point back to that same identity.
This prevents evidence laundering: attaching a passing result from a fixture, another branch, or an earlier commit to the current change. The verifier should reject missing files, unexpected files, dirty-tree drift, and implausibly small inventories rather than quietly hashing whatever happens to be visible.
- Deterministic byte-level source identity
- Explicit inventory rules
- Minimum file-count and non-empty checks
- Drift detection before and after evaluators
02Gates
Each claim needs a method that can fail for the right reason.
A build gate answers whether the candidate compiles in a defined environment. Behavior checks answer whether named journeys or contracts hold. Browser evidence observes rendered states. Accessibility and security require their own methods. One green test command should not be stretched into claims it did not evaluate.
Before trusting a new gate, mutate its input or implementation so the bad condition exists and confirm the gate rejects it. Keep that failure as a regression test. A check that has never demonstrated its ability to fail may only be re-reading the value the producer already emitted.
- One claim boundary per evaluator
- Mutation tests prove rejection
- Independent inputs instead of self-reported fields
- Failures retain a precise cause
03Order
Run cheap structural checks before expensive behavioral ones, without hiding later evidence.
A practical gate chain verifies lineage and source integrity before materialization, then builds the candidate before launching browser or accessibility work that depends on a runnable output. Security and catalog checks may remain independently runnable even if the build fails, depending on what they inspect.
The aggregate status should distinguish failed, blocked, and open. If a build fails, a browser journey may be blocked rather than failed. A human acceptance gate may remain open even when every automated check passes. These distinctions preserve meaning and prevent a single status word from erasing the reason work did not advance.
- Lineage before mutation
- Build before dependent runtime checks
- Independent checks continue when safe
- Failed, blocked, and open remain distinct
04Evidence
A result should be re-checkable from retained inputs and artifacts.
A verification record should include the subject digest, evaluator identity and version, declared parameters, timestamps, structured findings, verdict, limitations, and the digest of the evidence artifact itself. Numeric claims should be derived from the artifact by code rather than copied into prose where they can drift.
Append-only, content-addressed artifacts create a chain of custody across planning, execution, candidate materialization, gates, and release decisions. That does not make every claim true, but it makes substitutions and unsupported edits visible.
- Content-addressed evidence envelopes
- Inputs and parameters retained
- Published claims derived from artifacts
- No credentials or unrelated private data in evidence
05Acceptance
Technical verification informs a decision; it does not impersonate one.
Automated gates can establish properties within their scope. They cannot prove that a product tradeoff is desirable, that a visual direction is prestigious, or that a customer accepts the outcome unless an authorized person actually makes that judgment. Those decisions need named human authority and should remain open until it exists.
The final record should therefore show both the strongest verified result and its boundaries. A failed candidate can still produce a complete, useful record. A passing candidate can still be ineligible for release. Verification is strongest when it makes that difference obvious.
- Machine verdicts stay within method scope
- Human acceptance is explicit
- Release authority is separate
- Limitations travel with every published result
Design partners
Make every pass belong to one candidate and one method.
Run the change against its base, check its claims, and record which checks ran and which did not. TraceAxiom Verify is a GitHub Action, in development and not installable yet, that runs a pull request’s tests on base and head, checks its claims, and posts one verdict with its reasons.
Join the waitlist