Autoevals Scorer 401 Braintrust gateway error

Applies to:

Plan -
Deployment -

Summary

Issue: Scorer returns 401 Braintrust gateway error: auth failed [401 Unauthorized]: Invalid API Key when using LLMClassifierFromTemplate or other autoevals scorers. Cause: autoevals defaults to process.env.OPENAI_API_KEY || process.env.BRAINTRUST_API_KEY as the API key and process.env.OPENAI_BASE_URL || "https://api.braintrust.dev/v1/proxy" as the base URL. When OPENAI_API_KEY is set, it is sent to the Braintrust proxy instead of BRAINTRUST_API_KEY, causing auth to fail. Resolution: Initialize autoevals with a custom OpenAI client configured to use BRAINTRUST_API_KEY and the Braintrust gateway base URL.

Resolution steps

Initialize autoevals with a custom OpenAI client

Call init() before any evaluators run, passing a custom OpenAI client pointed at the Braintrust gateway with your BRAINTRUST_API_KEY.

import OpenAI from "openai";
import { init } from "autoevals";

init({
  client: new OpenAI({
    apiKey: process.env.BRAINTRUST_API_KEY,
    baseURL: "https://gateway.braintrust.dev",
  }),
});

All autoevals scorers (including LLMClassifierFromTemplate) will use this client.

⌘I

​Summary

​Resolution steps

​Initialize autoevals with a custom OpenAI client

Summary

Resolution steps

Initialize autoevals with a custom OpenAI client