The top 10 most loved features of 2024
This year was all about making Braintrust simpler, faster, and more powerful for you. Here are the top 10 features of 2024:
1. Custom scorer, tool, and prompt functions
Create and manage your own scorers, tools, and prompts, all within the SDK.
2. AI proxy and hybrid self-hosting
We bundled the AI proxy with the main API, introduced temp credentials for front-end requests, and broadened provider support (Cerebras, Fireworks, Lepton, etc.). Self-hosters also got simpler Docker deployments and better config options.
3. Structured outputs in the playground
We added structured outputs to the playground, plus improved the UX so you can parse and analyze raw data more easily. No more wrestling with JSON—just clean, organized feedback.
4. Monitoring improvements
Sparkline charts on project home, flexible resizable charts, and a dedicated monitor page for performance metrics. Keeping an eye on system health has never been simpler (or prettier).
5. Faster experiment and log loading
Major speed boosts for experiments and logs, even at scale. We also added new grouping options and CSV/JSON exports, so you can comb through data at whatever level of detail you need.
6. Human review
Sometimes you just need a real human’s opinion on LLM responses. Define custom human review scores in the project configuration, quickly label data in “review mode,” and feed those insights back into automated scoring for richer, more accurate results.
7. Custom provider configuration
You can now configure endpoints to stream results even if the model doesn’t natively support it. Think “instant streaming” from your custom providers to the Braintrust playground.
8. Improved logs and search
The logs viewer now supports exporting rows as CSV or JSON, plus advanced BTQL search and UI filters across metadata fields.
9. Attachment uploads and previews, plus Realtime API support
Log, preview, and analyze file attachments directly in Braintrust. Use the OpenAI Realtime API securely in serverless environments. Whether it’s data dumps, audio, images, or debug logs, everything can live under one roof.
10. Flexible visualizations
We added a grid view for experiment comparisons, an expanded timeline and tree view for traces, and built-in side-by-side diff modes. You can also render custom iframe URLs as fields in trace spans, add tags, and define custom columns. Visualizing performance differences is quick and intuitive.
Thank you
To everyone who used Braintrust, shared feedback, or reported issues—thank you. Your input guided every improvement we made this year. We can’t wait to share even more with you soon!