Our approach to hybrid deployment
When it comes to using Braintrust as part of your LLM development workflow, we want to make sure you have full flexibility and control over your data. To accomplish this, we designed a hybrid deployment model that lets you take advantage of our newest UI and platform features while simultaneously keeping your sensitive data secure.
At a high level, Braintrust's architecture has two main components: the data plane and the control plane. The data plane is the component that handles the actual data, while the control plane serves the UI along with metadata. When you deploy Braintrust in hybrid mode, you host the data plane (experiment logs, dataset records, etc.) in your own environment, while the control plane (web app and metadata) is hosted by Braintrust.
This gives you the best of both worlds: security and compliance and UI updates. All your data, like experiment inputs/outputs, logs, and sensitive customer information, lives securely in your own environment. You can make sure it’s behind a firewall or VPN and meets your organization’s compliance requirements. And since the UI and metadata are hosted by Braintrust, you can simply visit our site to see the latest features and improvements, just like any other SaaS product. Our product engineering team can continuously polish every little detail and fix bugs instantly, and you don’t have to manually update anything.
Keeping your data secure
Because the data plane runs entirely in your environment, Braintrust’s servers and employees do not require access to it. All data requests from the Braintrust UI go directly from your browser to your self-hosted data plane (via CORS), bypassing our servers. By default, the data plane also does not send telemetry back to the control plane.
Data storage
The data plane contains your data:
- Experiment records (input, output, expected, scores, metadata, traces, spans)
- Log records
- Dataset records
- Prompt playground prompts and completions
- Human review scores
The control plane stores metadata:
- Experiment and dataset names
- Project names and settings
- Organization info
- API keys (hashed)
- Encrypted LLM provider secrets
Auth credentials are managed through our external authentication service (Clerk).
Constraining the SDK
In most setups, the Braintrust SDK will communicate with both your data plane and the control plane to retrieve various metadata, but it’s also possible to constrain all SDK communication solely to your data plane. You can configure it so that the data plane acts as a proxy to the control plane, eliminating any need for outbound connections from the SDK to Braintrust’s servers.
Advantages of hybrid deployment
- You get access to the latest Braintrust features automatically.
- You decide where to host the data plane and what data to purge, so you can meet any data residency requirements like GDPR.
- You can configure rate limits, custom URLs, and domain proxies to ensure the deployment works within your IT environment and security policies.
Getting started
We provide a Docker Compose configuration file that you can use as-is or adapt to your infrastructure. After starting the containers, you’ll point to your newly hosted API in your Braintrust settings. From there, any time you load Braintrust via braintrust.dev, your browser will connect to your self-hosted data plane to retrieve and store data.
You can also self-host on AWS via a CloudFormation template.
For more information, check out the self-hosting guide. We’re also happy to help set up advanced deployment scenarios— just reach out to support@braintrust.dev.