Advanced topics related to self-hosting

This guide covers advanced topics related to self-hosting.

Data plane vs. control plane

Braintrust's architecture has two main components: the data plane and the control plane. The data plane is the component that handles the actual data, while the control plane is the component that serves the UI along with metadata.

Data storage

Braintrust self-hosting splits data into a data plane and a control plane. We often refer to this as "hybrid" self-hosting. When you deploy Braintrust in hybrid mode, you host the data plane (API) in your own environment, while the control plane (web app and metadata database) is hosted by Braintrust.

To clarify which data is stored in which location, here is a breakdown of the data stored in each place:

Securing sensitive customer data

Braintrust's servers and employees do not require access to your data plane for it to operate successfully. That means that you can protect it behind a firewall/VPN and physically isolate it from access. When you use the Braintrust web application, it communicates directly with the data plane (via CORS), and the data does not flow through any intermediate systems (the control plane, or otherwise) before reaching your browser. The data plane is also configured by default not to send any telemetry back to the control plane. Because of this architecture, our self-hosted customers do not generally list us as a subprocessor.

Like any third-party software, it is important that you establish the appropriate controls to ensure that your deployment is secure, and we're very happy to help you do so. Ultimately, the goal of the control plane and data plane split is to provide you with the highest levels of security and compliance.

Telemetry

By default, the Braintrust API server does not send any telemetry to the control plane; however, you should be aware of the following:

• There are a few endpoints that Braintrust's engineering team can access to debug issues and monitor system health. Specifically, the /brainstore/backfill/* endpoints which report system metrics about the backfill and compaction status of Brainstore segments. Note that these endpoints do not access or expose any data, just metadata from Brainstore. You can disable these endpoints by setting the DISABLE_SYSADMIN_TELEMETRY environment variable to true.
• There is an optional, TELEMETRY_ENABLED flag which sends billing and usage data to Braintrust. This is disabled by default, but it may be required depending on your contract with Braintrust. It may default to enabled in the future.

Customizing the webapp URL

The SDKs guide users to https://www.braintrust.dev (or the BRAINTRUST_APP_URL variable) to view their experiments. However, in certain advanced configurations, you may want to reverse proxy traffic to the BRAINTRUST_APP_URL from the SDKs while pointing users to a different URL.

To do this, you can set the BRAINTRUST_APP_PUBLIC_URL environment variable to the URL of your webapp. By default, this variable is set to the value of BRAINTRUST_APP_URL, but you can customize it as you wish. This variable is only used to display information, so even its destination does not need to be accessible from the SDK.

Constraining SDK to the data plane

If you're self-hosting the data plane, it may also be advantageous to constrain the SDKs to only communicate with your data plane. Normally, they communicate with the control plane to:

• Get your data plane's URL
• Register and retrieve metadata (e.g. about experiments)
• Print URLs to the webapp

The data plane can proxy the endpoints that the SDKs use to communicate with the control plane, allowing your SDKs to only communicate with the data plane directly. Simply set the BRAINTRUST_APP_URL environment variable to the URL of your data plane and BRAINTRUST_APP_PUBLIC_URL to "https://www.braintrust.dev" (or the URL of your webapp).

Allow-list URLs

In some cases, you may want to restrict the URLs that the SDKs or API server can communicate with. If so, you should include the following URLs:

www.braintrust.dev
braintrust.dev

Configuring rate limits

By default, the Braintrust API server imposes rate limits against any external domains it reaches out to, such as the BRAINTRUST_APP_URL. The purpose of rate-limiting is to prevent unintentionally overloading any external domains, which may block the API server IP in response.

By default, the rate limit is 100 requests per minute per user auth token. The API server exposes the following variables to configure the rate limits:

• OUTBOUND_RATE_LIMIT_MAX_REQUESTS: Configure the number of requests per time window. This can be set to 0 to disable rate limiting. In the braintrust CLI, this variable can be set with the --outbound-rate-limit-max-requests flag, or the OutboundRateLimitMaxRequests CloudFormation template parameter.
• OUTBOUND_RATE_LIMIT_WINDOW_MINUTES: Configure the time window in minutes before the rate limit resets. In the braintrust CLI, this variable can be set with the --outbound-rate-limit-window-minutes flag, or the OutboundRateLimitWindowMinutes CloudFormation template parameter.

Data residency (EU and others)

In the Hybrid (API) deployment:

• All customer data lives wherever you choose to host the data plane.
• All prompts are run on the data plane and in your region of choice.
• If you log a customer's data to Braintrust, it will only touch the servers in your data plane.
• You have API-level and even database-level control to purge customer data to comply with regulations like GDPR.
• Braintrust user info (e.g. your employees who sign into the Braintrust web application) is hosted globally by us, in the US. However, if you need this data to be hosted in your region, reach out to us and we can figure something out.

Audit headers

When integrating with Braintrust, especially in environments where actions need to be attributed to specific users or for compliance purposes, you might want to enable audit headers. These headers provide additional metadata about the request and the resources it touched.

To enable audit headers, include the x-bt-enable-audit: true header in your API request. When this header is present, the API response will include the following additional headers:

• x-bt-audit-user-id: The ID of the user who made the request (based on the provided API key or impersonation).
• x-bt-audit-user-email: The email of the user who made the request.
• x-bt-audit-normalized-url: A normalized representation of the API endpoint path that was called. Path parameters like object IDs are replaced with placeholders (for example, /v1/project/[id]).
• x-bt-audit-resources: A JSON-encoded, gzipped, and base64-encoded string containing a list of Braintrust resources (like projects, experiments, datasets, etc.) that were accessed or modified by the request. Each resource object includes its type, id, and name.

The x-bt-audit-resources header requires specific parsing due to its encoding. Here's an example of how to parse it using the Python SDK:

import os
 
import braintrust
import requests
 
API_URL = "https://api.braintrust.dev/v1"
# Ensure BRAINTRUST_API_KEY is set in your environment.
headers = {
    "Authorization": "Bearer " + os.environ["BRAINTRUST_API_KEY"],
    "x-bt-enable-audit": "true",  # Enable audit headers
}
 
# Example: Create a project.
response = requests.post(f"{API_URL}/project", headers=headers, json={"name": "audit-test-project"})
response.raise_for_status()
 
project_data = response.json()
print(f"Project created: {project_data['name']} (ID: {project_data['id']})")
 
# Access and parse audit headers.
user_id = response.headers.get("x-bt-audit-user-id")
user_email = response.headers.get("x-bt-audit-user-email")
normalized_url = response.headers.get("x-bt-audit-normalized-url")
resources_header = response.headers.get("x-bt-audit-resources")
 
print(f"Audit User ID: {user_id}")
print(f"Audit User Email: {user_email}")
print(f"Normalized URL: {normalized_url}")
 
if resources_header:
    try:
        # Use the provided utility to parse the resources header.
        resources = braintrust.parse_audit_resources(resources_header)
        print("Accessed/Modified Resources:")
        for resource in resources:
            print(f"  - Type: {resource['type']}, ID: {resource['id']}, Name: {resource['name']}")
    except Exception as e:
        print(f"Error parsing resources header: {e}")
else:
    print("No resources header found.")

This feature is useful for building audit logs or understanding resource usage patterns within your applications that interact with the Braintrust API.