Black vs Cookbook AI

A black-box evaluation of how AI-generated tests find functional bugs in live APIs.

Turn your recipes into a print-ready cookbook. The AI cookbook maker reads your files and designs every page. Try it free, no design skills.

About Black

A black-box evaluation of how AI-generated tests find functional bugs in live APIs. The harder question is whether those tests find bugs. Each system receives only a JSON schema and one valid sample payload, then must generate API test cases that expose failures in a live reference API. The evaluation uses APIEval-20 v1.0, a black-box benchmark contributed by KushoAI. Because KushoAI is also one of the evaluated systems, this report includes the methodology, workflow definitions, repeated-run setup, and robustness checks so readers can understand where the performance difference comes from.

About Cookbook AI

Turn your recipes into a print-ready cookbook. The AI cookbook maker reads your files and designs every page. Try it free, no design skills. How to make a cookbook with the AI Cookbook Maker. Edit any page, then export a print-ready PDF or send a share link. Turn Grandma's handwritten cards and scattered family recipes into a family recipe book everyone's proud to keep, designed and laid out for you, no layout tool to learn.