Skip to main content
Applies to:
  • Plan -
  • Deployment -
Summary Goal: Configure pass/fail thresholds for scorers in Ruby using the Braintrust API. Features: Function endpoint, scorer metadata, pass threshold configuration.

Configuration Steps

Step 1: Create the scorer registration function

POST to the /v1/function endpoint with scorer metadata.

Step 2: Call the function with your scorer details

Pass the project ID, scorer name, and desired threshold value.

Step 3: Verify in the Braintrust UI

The pass threshold will appear in the eval results, marking scores as pass/fail based on the threshold.

Notes

  • The __pass_threshold feature is not directly supported in the Ruby SDK yet
  • This workaround uses the same API endpoint that Python and TypeScript SDKs use internally
  • The slug field must be lowercase alphanumeric with hyphens only
  • Registration errors won’t fail your eval; they’re logged as warnings