00Comparison / running vs reading
A review bot reads the change. Verify runs it.
AI review bots and Verify answer different questions. A bot asks whether a diff looks right. Verify asks whether the change does what it claims when you actually run it. Teams with coding agents will want both, and should know which one to trust for what.
01Review bots
What a review bot is good at.
An AI review bot reads the diff, often with the surrounding code, and comments like a reviewer would: naming, structure, likely mistakes, missing edge cases, unclear intent. It is fast, it never gets tired, and it catches things a busy person skims past.
Its limit is the method. Reading is an opinion about code that has not been run. A bot can say a change looks safe and still be wrong, and it rarely tells you what it did not check.
- Fast, tireless reading
- Style, structure and intent
- Opinions, not execution
02Verify
What running the change adds.
Verify checks out the branch a pull request targets and the pull request itself, and runs the project's own tests on both. It plants control tests so a change cannot switch the tests off or fake their results, breaks the changed lines on purpose to see whether the tests notice, and checks claims such as “dependency bump only” or “every new npm package exists” mechanically.
Then it posts one verdict with its reasons, and lists the checks that did not run. Verify is in development and not installable yet.
- Tests on base and head
- Mutation testing of changed lines
- Claim checks
- States what it did not check
03What each catches
Defects that look fine are where reading runs out.
A test that passes because the change weakened it looks fine in a diff. So does an import of a package that was never published, a version bump that quietly edits source files, or a change whose tests would still pass if the new lines were wrong. These are the defects running catches and reading tends to miss.
Equally, running catches nothing about naming, readability or whether the design is sensible. A change can pass every check Verify runs and still deserve a reviewer's comment.
- Weakened tests
- Made-up packages
- Claims that do not match the diff
- Tests too thin to notice a bug
04Together
Use a bot for opinions and Verify for evidence.
The practical setup for a team with coding agents: let a review bot comment on readability and design, let Verify decide whether the change does what it claims, and let a person make the final call with both in front of them.
Verify's verdict is meant to be the part you can rely on without re-reading the diff: it names the exact commits it ran, the checks behind the verdict, and the ones that did not run.
- Bot: readability and design
- Verify: does it run and do what it says
- A person: the final call
The waitlist