Test surface: everything the tests themselves depend on.

01Definition

The tests, and everything that shapes their result.

A project's test surface is the set of files that decide what its tests do and report: the test files, fixtures and snapshots, test configuration, helper modules, and the test command itself.

Change any of these and you change the measuring instrument, not just the thing being measured.

• Test files and fixtures
• Configuration and helpers
• The test command

02Why it matters

If the ruler changes, the measurement means less.

A pull request that changes code and also edits its tests can make the tests pass by weakening them. That is not always wrong: behavior sometimes changes on purpose and tests must follow. But a passing run can no longer be taken at face value, because the change helped decide what passing means.

The fair comparison is to run the old tests against the new code. If everything that passed before still passes, the test edits hid nothing.

• Edits can weaken tests
• Sometimes legitimate
• Run the old tests on the new code

03In Verify

Built: the test-surface check.

Verify checks whether a change touches what the tests depend on. When a change edits existing tests, Verify also runs the base's versions of those tests against the change; if every test that passed on the base still passes, the edits hide nothing. Otherwise the comment names the tests involved.

Built; Verify is not installable yet.