Skip to main content
Administer your Braintrust organization, control access to resources, configure integrations, and manage self-hosted deployments.

​
Manage organizations

Organizations represent your team or business. Configure organization-wide settings including:
  • Members: Invite users and assign permission groups
  • API keys: Create and manage authentication credentials
  • Service tokens: Set up system integrations with service accounts
  • AI providers: Configure API keys for OpenAI, Anthropic, Google, and others
  • Environment variables: Set secrets for functions across your organization
Go to Settings to manage these settings. See Manage organizations for details.

​
Control access

Braintrust provides flexible access control at multiple levels:
  • Organization level: Assign users to the Owners, Engineers, Viewers, and All AI Provider Access built-in groups
  • Project level: Grant specific permissions to custom permission groups
  • Object level: Control access to individual experiments, datasets, or prompts
Create custom permission groups to match your team’s needs. Service accounts enable secure system integrations with granular permissions. See Control access for details.

​
Manage projects

Projects organize AI features in your application. Each project contains logs, experiments, datasets, and functions. Configure project settings including:
  • Tags: Organize and filter logs across your project
  • Human review scores: Define manual review criteria
  • Aggregate scores: Combine multiple metrics into single values
  • Online scoring: Automatically evaluate production logs
  • Comparison keys: Customize experiment comparisons
See Manage projects for details.

​
Separate production and staging

There are several ways to handle production vs. staging data: Use separate projects (recommended): Split production and staging into different projects so they’re isolated and code changes to staging cannot affect production. This also allows you to enforce access controls at the project level. Use tags within one project: If it’s easier to keep everything in one project (e.g., to triage issues in one place), use tags to separate environments. Filter by tags to view production or staging data independently. Use separate organizations: For physical isolation, create separate organizations for production and staging, each mapping to a different deployment.
Experiments, prompts, and playgrounds can use data across projects. For example, you can reference a prompt from your production project in your staging logs, or evaluate using a dataset from staging in a different project.

​
Set up automations

Automate routine tasks and stay informed about production issues:
  • Alerts: Get notified when metrics exceed thresholds or errors spike
  • Data management: Configure retention policies and archiving rules
Automations run in the background, keeping your data clean and your team informed. See data management and alerts for details.

​
Choose a deployment option

Braintrust supports several deployment options that differ in where your data plane runs and who operates it:
  • SaaS: Braintrust operates both the control plane and the data plane in Braintrust-managed infrastructure.
  • BYOC: Braintrust operates the data plane inside your own cloud account or project.
  • Self-hosted: Your team deploys and operates the data plane in your cloud using Braintrust’s published artifacts.
Each option keeps the same managed control plane. See Deployment options to compare them and choose the right fit.

​
Manage billing

Control your Braintrust subscription and costs:
  • Change plans: Upgrade or downgrade between Starter, Pro, and Enterprise
  • Monitor usage: Track trace spans, processed data, and scores
  • Cost alerts: Get notified when spending exceeds thresholds
  • Payment methods: Update billing information and access invoices
See Manage billing for details.

​
Configure authentication

Braintrust supports multiple authentication methods:
  • Email/password: Standard authentication for individuals
  • SSO: Integrate with your identity provider (Okta, Google Workspace, Azure AD)
  • API keys: Authenticate SDK and API requests
  • Service tokens: Authenticate service accounts for system integrations
See Configure authentication for details.

​
Review audit logs

Audit logs show who changed projects, access controls, API keys, etc. Members of the Owners permission group can read audit logs by default. To let anyone else read them, grant the Read audit logs permission to a permission group. See Audit logs for the event reference.