Skip to main content
Applies to:
  • Plan -
  • Deployment -

​
Summary

Issue: After migrating to data plane v2, custom code scorers calling an internal LLM proxy via BRAINTRUST_API_KEY fail with an authentication error. The provider secret is configured to send the stored credential as an API key, but the upstream OpenAI-compatible gateway expects it as a Bearer token. The Braintrust API key in the scorer may still be valid. The failure can come from the provider secret Braintrust uses when forwarding the model request to the upstream gateway. Cause: AI provider secrets configured with auth_format: api_key are incompatible with gateways that require Authorization: Bearer after the data plane v2 migration. Resolution: Update the affected provider secret to use auth_format: bearer with auth_type: api_key, ensuring the stored secret value is a raw token with no Bearer prefix.

​
Resolution steps

​
Step 1: Identify the affected provider secret

Check the auth_format on the secret tied to your custom model. Use the appropriate command based on whether the secret is org-level or project-level. Org-level: 
curl -s -H "Authorization: Bearer <braintrust-api-token>" \
  "$BRAINTRUST_API_URL/v1/ai_secret" \
  | jq --arg m "<model-name>" '
    .objects[]
    | select((.metadata.customModels // {}) | has($m))
    | { id, name, provider_type: .type, preview_secret, metadata }
  '
Project-level: 
curl -s -H "Authorization: Bearer <braintrust-api-token>" \
  "$BRAINTRUST_API_URL/v1/env_var?object_type=project&object_id=<PROJECT_ID>&secret_category=ai_provider" \
  | jq --arg m "<model-name>" '
    .objects[]
    | select((.metadata.customModels // {}) | has($m))
    | { id, name, provider_type: .secret_type, preview_secret, metadata }
  '
Look for metadata.auth_format in the response. If it is api_key, proceed to Step 2.

​
Step 2: Update the provider secret (requires org admin)

Update the secret to use Bearer authentication. The correct combination for an OpenAI-compatible provider is:
  • auth_type: api_key
  • auth_format: bearer
Note: Users without org admin permissions cannot update ai_secret via the API. An admin must make this change.
Important: The stored secret value must be the raw token only. Do not include a Bearer prefix in the stored value.

​
Step 3: Verify the token works outside Braintrust

Before re-running the scorer, confirm the token is valid against your gateway directly. 
export GENAI_API_BASE="https://<your-gateway-host>"
export GENAI_TOKEN="<raw-token-stored-as-provider-secret>"

curl -i "$GENAI_API_BASE/models" \
  -H "Authorization: Bearer $GENAI_TOKEN"

curl -i "$GENAI_API_BASE/v1/models" \
  -H "Authorization: Bearer $GENAI_TOKEN"
A successful response confirms the token and gateway are working correctly.

​
Step 4: Re-run the scorer or test in the Braintrust UI

After the secret is updated, re-run the affected custom code scorer or send a test prompt in the Braintrust UI to confirm LLM calls succeed.

​
Notes

  • This issue affects self-hosted or hybrid Braintrust deployments that migrated to data plane v2.
  • The Test key button in the UI may still show an error in some configurations even when the key is functional. Validate end-to-end by running the scorer or a prompt instead.
  • Environments not migrated to data plane v2 are unaffected.