Logs
Logs are the recorded data and metadata from an AI routine. We record the inputs and outputs of your LLM calls to help you evaluate model performance on set of predefined tasks, identify patterns, and diagnose issues.
In Braintrust, logs consist of traces, which roughly correspond to a single request or interaction in your application. Traces consist of one or more spans, each of which corresponds to a unit of work in your application, like an LLM call, for example. You typically collect logs while running your application, both in staging (internal) and production (external) environments, using them to debug issues, monitor user behavior, and gather data for building datasets.
Why log in Braintrust?
By logging in Braintrust, you can create a feedback loop between real-world observations (logs) and offline evaluations (experiments). This feedback loop is crucial for refining your model's performance and building high-quality AI applications.
By design, logs are exactly the same data structure as experiments. This leads to a number of useful properties:
- If you instrument your code to run evals, you can reuse this instrumentation to generate logs
- Your logged traces capture exactly the same data as your evals
- You can reuse automated and human review scores across both experiments and logs
Where to go from here
Now that you know the basics of logging in Braintrust, dig into some more complex capabilities: