Configure a custom CA for Python scorers on self-hosted AWS

Applies to:

Plan -
Deployment -

Summary

Issue: A Python scorer fails with SSLCertVerificationError or CERTIFICATE_VERIFY_FAILED before its handler runs when the quarantine runtime cannot validate the certificate chain presented through a TLS-intercepting proxy. Cause: The Braintrust SDK login occurs before the scorer module is loaded. The required CA bundle and trust configuration must therefore be available before scorer code executes. Resolution: Package the required custom CA bundle in a small Python wheel, include that wheel in the scorer’s requirements, and point REQUESTS_CA_BUNDLE to its installed path. Configure the environment variable at organization scope in the Braintrust UI or at function scope through the API. Function scope limits the configuration to the selected scorer.

Resolution steps

Step 1: Obtain the required custom CA certificates

Ask your security or network team for the PEM-encoded CA certificates needed to validate the replacement certificate chain presented by the proxy. Depending on the proxy’s public key infrastructure (PKI) configuration, this can include a root CA, one or more intermediate CAs, or both.

Include only public CA certificates. Never include a CA private key in the package.

Step 2: Create the CA bundle package

REQUESTS_CA_BUNDLE must point to a file that already exists in the runtime filesystem; it cannot contain raw PEM data. Packaging the bundle in the scorer dependencies makes that file available before the Braintrust SDK login begins. Create a package with the following structure:

ca-package/
├── pyproject.toml
└── custom_proxy_ca/
    ├── __init__.py
    └── ca-bundle.crt

The package name is arbitrary, but it determines the installed filesystem path used by REQUESTS_CA_BUNDLE. Choose one of the following bundle options.

Custom CA bundle only

Use a custom-only bundle when all scorer HTTPS traffic is intercepted by the proxy and the bundle contains the CA certificates needed to trust the certificates presented by the proxy:

cp custom-proxy-ca.crt ca-package/custom_proxy_ca/ca-bundle.crt

Combined custom and public CA bundle

Use a combined bundle when any public destination bypasses the proxy, such as a hostname in NO_PROXY. Setting REQUESTS_CA_BUNDLE replaces the default Python Requests CA bundle rather than extending it, so add the certifi public roots before appending the custom CA certificates:

cp "$(python -m certifi)" ca-package/custom_proxy_ca/ca-bundle.crt
cat custom-proxy-ca.crt >> ca-package/custom_proxy_ca/ca-bundle.crt

In both examples, custom-proxy-ca.crt is the PEM file provided by your security or network team. If it contains multiple certificates, copy or append the entire file. Update certifi before creating a combined bundle so that it contains an up-to-date set of public roots. Use this pyproject.toml as a starting point:

pyproject.toml

[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

[project]
name = "custom-proxy-ca"
version = "0.1.0"
requires-python = ">=3.9"

[tool.setuptools]
packages = ["custom_proxy_ca"]

[tool.setuptools.package-data]
custom_proxy_ca = ["*.crt"]

Step 3: Build the wheel

Build the package from the scorer’s working directory:

uv build ca-package

Confirm that the generated wheel contains the CA file:

unzip -l ca-package/dist/custom_proxy_ca-0.1.0-py3-none-any.whl

The output must include:

custom_proxy_ca/ca-bundle.crt

If the custom CA certificates change, or if you update the certifi roots in a combined bundle, increment the package version and build a new wheel. If the bundle does not change, no version update or rebuild is needed.

Step 4: Add the wheel to the scorer requirements

Create a file named requirements-ca.txt in the scorer’s working directory. Add the wheel path as its only line:

./ca-package/dist/custom_proxy_ca-0.1.0-py3-none-any.whl

Push the scorer with the requirements file:

bt functions push scorer.py \
  --requirements requirements-ca.txt \
  --if-exists replace

Run this command from the scorer’s working directory so that the relative wheel path resolves correctly.

If the scorer already uses a requirements file, create a wrapper file named requirements-combined.txt that includes the existing dependencies and the CA wheel.

The wrapper file should contain:

-r requirements.txt
./ca-package/dist/custom_proxy_ca-0.1.0-py3-none-any.whl

Then push the scorer with the wrapper file:

bt functions push scorer.py \
  --requirements requirements-combined.txt \
  --if-exists replace

Step 5: Configure the environment variables in Braintrust

Add REQUESTS_CA_BUNDLE as an environment variable in Braintrust and set it to the installed CA bundle path. This tells Python Requests which CA bundle to trust. Also add the proxy environment variables required by your environment:

HTTP_PROXY=<proxy-url>
HTTPS_PROXY=<proxy-url>
REQUESTS_CA_BUNDLE=/var/task/custom_proxy_ca/ca-bundle.crt

The path after /var/task/ must match the package directory and certificate filename inside the wheel. Configure REQUESTS_CA_BUNDLE and any required proxy variables at either organization or function scope.

Organization scope through the UI

Go to Settings > Env variables and add the required variables. Organization-scoped variables are available to all Braintrust functions (including prompts, scorers, and tools) across every project in the organization.

Function scope through the API

To limit the variables to one scorer, create function-scoped environment variables through the Braintrust API. Braintrust stores a scorer as a function, so object_type: "function" with that scorer’s function ID applies the variable only to that scorer. On the scorer page, open the ... menu and copy the scorer ID. Then create REQUESTS_CA_BUNDLE:

curl -sS -X POST \
  "${BRAINTRUST_API_URL}/v1/env_var" \
  -H "Authorization: Bearer ${BRAINTRUST_API_KEY}" \
  -H "Content-Type: application/json" \
  --data '{
    "name": "REQUESTS_CA_BUNDLE",
    "value": "/var/task/custom_proxy_ca/ca-bundle.crt",
    "object_type": "function",
    "object_id": "<scorer-function-id>"
  }'

Repeat the request for HTTPS_PROXY and HTTP_PROXY when those variables should use the same function scope. Function-scoped variables are configured through the API. The environment-variable settings in the UI are organization-scoped. If some destinations must bypass the proxy, also configure NO_PROXY at the same scope with a comma-separated list of hostnames:

NO_PROXY=<direct-hostname>,<another-direct-hostname>

NO_PROXY changes routing only. Matching destinations are still validated against REQUESTS_CA_BUNDLE. Ensure the configured CA bundle trusts certificates presented by those direct destinations. Only include destinations that must connect directly, and do not add the Braintrust destination if the SDK login must pass through the intercepting proxy.

How to confirm it worked

Invoke the scorer through its normal workflow. A successful result confirms that the runtime found the packaged CA bundle and completed the Braintrust SDK login through the proxy. For end-to-end verification on AWS, confirm that the quarantine Lambda log contains a normal START, END, and REPORT with no TLS exception, and that the proxy access log contains a matching request or CONNECT entry for the Braintrust data-plane hostname. If the invocation still fails, use the error to narrow the cause:

Could not find a suitable TLS CA certificate bundle or invalid path means the wheel was not installed as expected or REQUESTS_CA_BUNDLE does not match the installed path.
self-signed certificate in certificate chain usually means the selected bundle does not validate the certificate chain presented by the proxy.
Certificate failures limited to destinations in NO_PROXY mean the configured CA bundle does not trust the certificates presented by those direct destinations.
CA cert does not include key usage extension means the CA certificate profile is not accepted by the runtime’s strict certificate validation. Ask the certificate owner to issue a CA certificate with appropriate CA basic constraints and certificate-signing key usage.

Notes

This workaround applies to Python scorers pushed with bt functions push on self-hosted AWS data planes using Lambda quarantine. It has not been validated for TypeScript scorers or for self-hosted Azure and GCP data planes.
REQUESTS_CA_BUNDLE affects all Python requests traffic in the scorer runtime, not only Braintrust requests.
NO_PROXY bypasses proxy routing for matching destinations. It does not restore the default certifi or system trust store.
This workaround packages public trust material for environments that require TLS interception. A platform-level custom CA configuration is preferable when one is available.
For local SDK or CLI certificate failures outside a scorer runtime, see SSL certificate verification failure behind a corporate proxy.
For scorer packaging and dependency options, see Custom code functions and bt functions push.

​Summary

​Resolution steps

​Step 1: Obtain the required custom CA certificates

​Step 2: Create the CA bundle package

​Custom CA bundle only

​Combined custom and public CA bundle

​Step 3: Build the wheel

​Step 4: Add the wheel to the scorer requirements

​Step 5: Configure the environment variables in Braintrust

​Organization scope through the UI

​Function scope through the API

​How to confirm it worked

​Notes

Summary

Resolution steps

Step 1: Obtain the required custom CA certificates

Step 2: Create the CA bundle package

Custom CA bundle only

Combined custom and public CA bundle

Step 3: Build the wheel

Step 4: Add the wheel to the scorer requirements

Step 5: Configure the environment variables in Braintrust

Organization scope through the UI

Function scope through the API

How to confirm it worked

Notes