00Glossary / term
Independent tests: tests written apart from the change, from what it says it does.
Independent tests are written apart from the change they check, from its stated intent, by another author. Why a change's own tests miss some of its bugs.
01Definition
Tests the author of the change never saw.
An independent test is written by someone, or something, other than the author of the change, from what the change claims to do rather than from its code. The point is separation: the same mind that wrote a bug often writes a test that misses it.
In AI terms, that means a different model, ideally from a different family, writing tests from the pull request's description and the code's public behavior.
- Different author
- Written from intent
- Separation is the point
02Why it matters
A change's own tests share its blind spots.
When an agent writes code and tests together, the tests tend to confirm the code as written, including its mistakes. Mutation testing shows whether those tests would notice a broken line; it cannot show behavior the change forgot entirely.
Tests that must fail before the change and pass after it are evidence that the change does what it says, not merely that it does what it does.
- Shared blind spots
- Fail before, pass after
- Evidence of intent, not just behavior
03In Verify
Not built yet.
Independent tests are the next major stage of Verify: tests written from a pull request's stated intent by a model from a different family than the one that wrote the change. They must fail on the base and pass on the head, and they add evidence without ever rejecting a change on their own.
This is planned and not built. Verify itself is not installable yet.
The waitlist
See this check on your own pull requests.
Verify is not installable yet. Join the waitlist to hear the day it is; teams with coding agents can also become design partners.
Join the waitlist