Braintrust

Open-source LLM evaluation and prompt management platform with a collaborative playground.

Overview

Braintrust provides a framework for writing, running, and versioning LLM evaluations across datasets. It includes a UI for side-by-side prompt comparison, rubric-based grading, and integration with CI/CD pipelines to prevent regression.

Best for

• prompt A/B testing
• CI/CD evaluation gates
• collaborative prompt iteration

Trade-offs

What XeroHack pre-wires

When you pick Braintrust in the interview, the engine emits these files into your scaffold:

• evaluation templates
• scorecard grading
• dataset version control