Direct answer
Give the agent a behavior contract, existing test conventions, and a precise test boundary before asking for code. Require each new test to demonstrate the failure it protects against, avoid private implementation coupling, and pass independently under the repository’s real test command.
Key takeaways
- Choose the test level from the behavior boundary rather than asking for maximum coverage.
- Make a new regression test fail against the unfixed behavior before accepting it.
- Review fixtures, mocks, time, randomness, and shared state for hidden flakiness.
- Treat line coverage as navigation evidence, not proof of meaningful assertions.
Key facts
| Fact | Verified value | Evidence |
|---|---|---|
| Test discovery | pytest discovers tests by documented file, class, and function naming conventions. | View source |
| Repository work | Codex can inspect existing code and run repository commands within its configured environment. | View source |
| Workflow input | Claude Code common workflows recommend supplying context and verification criteria for implementation tasks. | View source |
| Execution boundary | Codex sandbox and approval settings control whether generated test workflows may execute commands or write files. | View source |
What this guide helps you solve
Create a controlled workflow for generating unit, integration, and regression tests with coding agents.
Verification scope and limits
- Evidence basis
- Official documentationLocal verification
- Verification scope
- Covers behavior selection, test-level choice, generation, failure confirmation, flakiness review, and suite integration.
- Limits and invalidation conditions
- - The workflow does not infer missing business rules; maintainers must provide expected behavior.
- - Framework-specific discovery, fixture, and isolation rules must be checked against the repository version.
Select one observable behavior to protect
Begin with an input, state transition, output, or side effect that a caller can observe. State normal cases, boundaries, invalid inputs, and the required error behavior. If expected behavior is disputed, stop and resolve the contract before generating assertions.
Ask the agent to locate neighboring tests and explain their conventions before editing. Existing fixtures, naming, helpers, and test placement are part of the repository contract.
Choose the narrowest credible test level
Use a unit test for deterministic logic with a stable function boundary, an integration test for component contracts, and an end-to-end test only when the user-visible path cannot be validated lower in the stack. Adding every level by default creates maintenance cost without necessarily adding a new signal.
- Prefer public APIs and observable outcomes over private fields.
- Reuse maintained fixtures before inventing a parallel setup layer.
- Mock external boundaries, not the code whose behavior the test should exercise.
- Keep secrets and live production services out of the test path.
Generate small tests with explicit assertions
Ask for a small batch grouped by behavior, then review names and assertions before expanding. Each test should have a clear reason to fail. Snapshot tests are appropriate only when the serialized output is itself a reviewed contract.
Require the agent to explain any fixture or mock it adds. Hidden global setup can make a test pass while bypassing the path it claims to exercise.
Prove that a regression test detects the defect
For a bug fix, run the new test against the failing revision or temporarily reverse the targeted condition to confirm that the assertion detects the intended behavior. Then restore the implementation and verify the test passes. Record both commands and outcomes.
A test that passes before and after the change may still be useful, but it is not evidence that it protects the reported regression.
Run isolation and quality gates
Run the focused test repeatedly, then run it with related tests and the standard suite shard. Check ordering, clock use, randomness, temporary files, network calls, and cleanup. Review whether failures point to the behavior rather than opaque helper code.
- 01Focused runExecute the new test alone with the documented runner.
- 02Failure proofConfirm the assertion fails when the protected behavior is absent.
- 03Neighbor runExecute related tests in different orders where the framework permits.
- 04Suite gateRun the repository checks required for the changed package.
Review generated tests as production code
Keep test intent readable, remove duplicated setup, and assign ownership through the same review process as implementation code. When behavior changes, update the contract and test together; do not let an agent silently rewrite expected results to match a regression.
Official sources and verification scope
This guide is based on public official documentation. Commands and configuration may change between client versions, so verify the linked sources before use.
Read the verification methodology