Skip to main content

Documentation Index

Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt

Use this file to discover all available pages before exploring further.

Administer your Braintrust organization, control access to resources, configure integrations, and manage self-hosted deployments.

​
Manage organizations

Organizations represent your team or business. Configure organization-wide settings including:
  • Members: Invite users and assign permission groups
  • API keys: Create and manage authentication credentials
  • Service tokens: Set up system integrations with service accounts
  • AI providers: Configure API keys for OpenAI, Anthropic, Google, and others
  • Environment variables: Set secrets for functions across your organization
Go to Settings to manage these settings. See Manage organizations for details.

​
Control access

Braintrust provides flexible access control at multiple levels:
  • Organization level: Assign users to Owners, Engineers, or Viewers groups
  • Project level: Grant specific permissions to custom permission groups
  • Object level: Control access to individual experiments, datasets, or prompts
Create custom permission groups to match your team’s needs. Service accounts enable secure system integrations with granular permissions. See Control access for details.

​
Manage projects

Projects organize AI features in your application. Each project contains logs, experiments, datasets, and functions. Configure project settings including:
  • Tags: Organize and filter logs across your project
  • Human review scores: Define manual review criteria
  • Aggregate scores: Combine multiple metrics into single values
  • Online scoring: Automatically evaluate production logs
  • Comparison keys: Customize experiment comparisons
See Manage projects for details.

​
Separate production and staging

There are several ways to handle production vs. staging data: Use separate projects (recommended): Split production and staging into different projects so they’re isolated and code changes to staging cannot affect production. This also allows you to enforce access controls at the project level. Use tags within one project: If it’s easier to keep everything in one project (e.g., to triage issues in one place), use tags to separate environments. Filter by tags to view production or staging data independently. Use separate organizations: For physical isolation, create separate organizations for production and staging, each mapping to a different deployment.
Experiments, prompts, and playgrounds can use data across projects. For example, you can reference a prompt from your production project in your staging logs, or evaluate using a dataset from staging in a different project.

​
Set up automations

Automate routine tasks and stay informed about production issues:
  • Alerts: Get notified when metrics exceed thresholds or errors spike
  • Data management: Configure retention policies and archiving rules
Automations run in the background, keeping your data clean and your team informed. See Set up automations for details.

​
Self-host Braintrust

Deploy Braintrust in your own cloud infrastructure:
  • AWS: Deploy to Amazon Web Services
  • GCP: Deploy to Google Cloud Platform
  • Azure: Deploy to Microsoft Azure
Self-hosting keeps your data within your environment while providing the full Braintrust platform. Configure authentication, networking, and advanced options to match your infrastructure requirements. See Self-host Braintrust for details.

​
Manage billing

Control your Braintrust subscription and costs:
  • Change plans: Upgrade or downgrade between Starter, Pro, and Enterprise
  • Monitor usage: Track trace spans, processed data, and scores
  • Cost alerts: Get notified when spending exceeds thresholds
  • Payment methods: Update billing information and access invoices
See Manage billing for details.

​
Configure authentication

Braintrust supports multiple authentication methods:
  • Email/password: Standard authentication for individuals
  • SSO: Integrate with your identity provider (Okta, Google Workspace, Azure AD)
  • API keys: Authenticate SDK and API requests
  • Service tokens: Authenticate service accounts for system integrations
See Configure authentication for details.