00Field guide / delivery cost
How to Calculate Cost per Accepted Coding-Agent Change
A cheap model response can still lead to an expensive change. Calculate cost over a declared group of tasks, including unsuccessful attempts and the work required to verify the result. The worked example below is invented to explain the method; it is not a TraceAxiom benchmark or price.
01Scope
Freeze the task group and acceptance rule first.
Choose the decision this measurement should support: for example, whether one workflow reduces delivery cost on a particular class of maintenance tasks. Record the repositories, task types, starting revisions, permitted tools, retry limits and observation window before comparing outcomes. Keep the same acceptance requirements across alternatives.
Count every eligible task in that group, including failed, cancelled and capped attempts. Give repairs stable links to their original task so retries cannot quietly become extra successful results. An independently accepted change is a different event from a model claiming completion or a build passing. Record which event supplies the denominator.
02Cost record
Collect the costs incurred by the whole group.
Keep model and tool charges, sandbox compute, verification, storage and other attributable service costs. Include unsuccessful attempts and integration work. State how shared infrastructure is allocated, which period is covered, and whether charges are observed, estimated or still awaiting billing. Unavailable costs are missing data; recording zero would change the meaning of the result.
Track active human review, coordination, correction and support time separately, including founder assistance. Record observed minutes before attaching an assumed hourly rate. Keep taxes and any other excluded categories explicit so a service-cost figure cannot be mistaken for a complete business budget.
- Task and attempt IDs connect usage to outcomes
- Billing and usage records identify the measurement window
- Shared cost allocation has a recorded rule
- Estimated and unreconciled charges remain labelled
03Worked example
Ten tasks, six accepted changes, and $60 of service cost.
Suppose an illustrative group contains ten eligible tasks. Six have independent acceptance, two fail their required checks, one is cancelled and one reaches its retry cap. All ten remain in the task inventory. Their combined service costs are $32 for model and tool usage, $18 for sandbox and verification, and $10 for attributable storage and other service usage: $60 in total. Assume these amounts include the unsuccessful attempts and all repairs.
The accepted-result rate is 6 / 10 = 60%. Service cost per accepted change is $60 / 6 = $10. Dividing by ten would give $6 per attempted task, which answers a different question. Publish both labels and raw counts if both are useful.
Suppose the group also takes 90 minutes of active human work. That is 15 minutes per accepted change after allocating all group review time across the six accepted outcomes. At an explicitly assumed $40 per hour, human service cost is $60 for the group. The combined illustrative service-plus-human cost is ($60 + $60) / 6 = $20 per accepted change. The assumed rate is not an observed payment or a claim about labor-market pricing.
04Credits
Show cash cost alongside cost without promotional credits.
Now suppose a promotional credit covers $18 of that illustrative $60 service bill. Under the example’s stated coverage assumption, service cash cost becomes $42, or $7 per accepted change. The service cost without the promotional credit remains $60, or $10 per accepted change. Keep the human-cost calculation separate in both views.
This distinction makes the expiry effect visible. A credit can fund useful experiments without establishing that the same workflow is economical after the award ends. Actual coverage, taxes, expiry and delayed charges must come from the relevant account records; do not infer them from the nominal credit balance.
05Missing outcomes
Zero accepted changes means the ratio is not measurable.
If none of the ten tasks has an accepted result, retain the $60 cost and the zero accepted count. Report cost per accepted change as not measurable because its denominator is zero. Do not turn it into $0, substitute attempted tasks, or drop the unsuccessful group from the comparison.
When acceptance is pending, identify those tasks separately. Likewise, a cost record awaiting provider billing supports a provisional estimate with its missingness stated. A completed test run does not resolve either missing acceptance or missing costs.
06Decision
Compare cost with quality, elapsed time and human attention.
Use matched task groups and the same acceptance rules when comparing workflows. Retain regressions, safety failures, repair counts and time spent waiting or integrating. A lower cost figure may come with fewer accepted changes or more human effort; publish those tradeoffs together. A ten-task example is too small and artificial to establish a market-wide advantage.
Keep a reviewable record containing the frozen cohort, exact candidates, attempt history, acceptance decisions, usage inputs, allocation policy, missing data and calculation. This guide describes an evaluation method and reports no measured TraceAxiom cost figure.
Design partners
Measure the cost of reaching an accepted result.
Use the coding-agent evaluation guide to define the comparison, then review the exact candidate and its evidence before recording acceptance. Preserve unsuccessful work and missing information in the same record.
Join the waitlist