Guides/Self-hosting

Setting up Braintrust in your AWS Account

This guide walks through the process of setting up Braintrust in your AWS account using Terraform. This is the recommended way to self-host Braintrust on AWS.

Setting up the Data Plane

Configure the Terraform module

First, you'll need to instantiate the Braintrust Terraform module in your own repository. The module contains all the necessary resources for a self-hosted Braintrust data plane. We recommend created a dedicated AWS account for your Braintrust deployment, but it is not required.

We provide an example configuration in the terraform-braintrust-data-plane repository. Copy the entire contents of the examples/braintrust-data-plane directory into your own repository.

You'll need to modify several files:

  1. provider.tf: Configure your AWS account and region
  2. terraform.tf: Set up your remote backend (typically S3 and DynamoDB)
  3. main.tf: Customize the Braintrust deployment settings

The default settings in main.tf are suitable for a small development deployment. Adjust them based on your needs.

Initialize AWS account

If you're using a new AWS account, you'll need to create service-linked roles first:

./scripts/create-service-linked-roles.sh

This script ensures all necessary IAM service-linked roles are created for the deployment.

Configure Brainstore license

You'll need a Brainstore license key to complete the deployment. You can find this in the Braintrust UI under Settings > Data Plane.

Brainstore License Key

Do not commit the license key to your git repository. Instead, use one of these methods:

  • Set TF_VAR_brainstore_license_key=your-key in your environment
  • Pass it via command line: terraform apply -var 'brainstore_license_key=your-key'
  • Add it to an uncommitted terraform.tfvars or .auto.tfvars file
  • Store it in AWS Secrets Manager and load it at runtime using the aws_secretsmanager_secret_version_source data source

Deploy the module

Initialize and apply the Terraform configuration:

terraform init
terraform apply

This will create all necessary AWS resources including:

  • VPC with public & private subnets
  • Lambda functions for the Braintrust API
  • Public Cloudfront endpoint and API Gateway
  • EC2 Auto-scaling group for Brainstore
  • Postgres database, Redis cache, and S3 buckets
  • KMS key for encryption

Get your API URL

After the deployment completes, get your API URL from the Terraform outputs:

terraform output

You should see output similar to:

api_url = "https://dx6atff6gocr6.cloudfront.net"

Save this URL - you'll need it to configure your Braintrust organization.

Configure your organization

Point your organization to your data plane

  1. Go to the Braintrust UI
  2. Click your user icon in the top right
  3. Navigate to Settings > Data Plane
  4. Click "Edit"

If you're testing, we highly recommend creating a new Braintrust Organization for testing your data plane. Changing your live organization's API URL might break existing users.

Setting the API URL in Braintrust

Configure the API URL

  1. Paste your API URL into the text field
  2. Leave the Proxy and Realtime URL fields blank
  3. Click "Save"

Paste the API URL

Verify the connection

The UI will automatically test the connection to your new data plane. Verify that the ping to each endpoint is successful.

Verify Successful Ping

Maintaining your installation

Updating the deployment

If you are pointing at the latest root module, you can update the deployment by simply re-running terraform apply. This will apply any changes to the infrastructure while preserving your data.

It is always recommended to carefully review the output of terraform plan before applying any changes to your deployment. If you see something unexpected, like deletion of a database or S3 bucket, please reach out to the Braintrust team for help.

If you prefer to point at a specific version of the root module, you can do so by adding a ?ref=<version> to the module source:

module "braintrust-data-plane" {
  source = "github.com/braintrustdata/terraform-braintrust-data-plane?ref=vX.Y.Z"

  # ... other configuration ...
}

We publish regular releases of the root module which you can find in the GitHub Releases page.

Debugging issues

If you encounter issues, you can use the provided dump-logs.sh script to collect logs:

./scripts/dump-logs.sh <deployment_name> [--minutes N] [--service <svc1,svc2,...|all>]

For example, to dump 60 minutes of logs for the bt-sandbox deployment, run:

./scripts/dump-logs.sh bt-sandbox

This will save logs for all services to a logs-<deployment_name> directory, which you can share with the Braintrust team for debugging.

Advanced configurations

Customizing the deployment

While we recommend using the root module for easier support and upgrades, you can customize the deployment by using individual submodules. The main module serves as a reference for how to configure these submodules.

For example, if you want to use an existing VPC:

  1. Remove the module.main_vpc block
  2. Pass your existing VPC's ID, subnets, and security group IDs to the services, database, and redis modules

VPC connectivity

If you need to connect Braintrust's VPC to other internal resources (like an LLM gateway), you can do one of the following:

  1. Create a VPC Endpoint Service for your internal resource. Then create a VPC Interface Endpoint inside of the Braintrust "Quarantine" VPC
  2. Setup VPC peering with the Braintrust "Quarantine" VPC

CloudFormation (Deprecated)

We used to support a CloudFormation template for setting up the Braintrust data plane. This method is deprecated for new deployments.

If you have an existing deployment and need to migrate to Terraform, please reach out to the Braintrust team for help.

The docs for the deprecated CloudFormation template can be found here.

On this page