Black vs Wasted Cycles

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

A local-first wall-clock profiler for coding agents and GitHub Actions. Find the machine time blocking progress.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Wasted Cycles

A local-first wall-clock profiler for coding agents and GitHub Actions. Find the machine time blocking progress. Wasted Cycles, find machine time blocking agent work Skip to content. See how much agent time goes to builds, tests, CI, containers, packages, sub-agents, and repeated work, then fix the largest machine stall first. Wasted Cycles measures elapsed time, keeps human wait outside the metric, and gives you a small ranked list instead of a speculative ROI number. Read existing Codex, Claude Code, Cursor, and Grok Build traces in place.