Black vs Dodger

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Dodger studies your market and your users, writes the specs, hands your agents the full context, watches what ships, and takes the next action once you approve it.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Dodger

Dodger studies your market and your users, writes the specs, hands your agents the full context, watches what ships, and takes the next action once you approve it. A demo of Dodger's workspace: a request built from cited evidence is sent, Dodger drafts a PRD in Documents, the change ships as a pull request, and the release is scored against its goal. It covers the problem, the scope, twelve requirements, and the two metrics this should move: checkout completion and repeat purchase rate.