Remote eval OpenTelemetry header propagation

Applies to:

Plan -
Deployment -

Summary

Remote eval requests from the Braintrust UI do not include OpenTelemetry trace context headers (for example, traceparent or baggage). Braintrust does include request context in the request body so results attach to the Playground run. Start an OTel span inside your remote eval /eval handler and add the Braintrust parent payload, project ID, or your own run identifier as span attributes to correlate traces across your services.

What is happening

When the Braintrust UI calls a remote eval service, it sends eval context in the request body, including fields such as parent, project_id, data, parameters, and scores, but does not propagate OTel trace headers. If your eval handler then makes downstream calls, those calls will not share an incoming OTel trace unless you create or propagate a span yourself. This means distributed traces from the eval pod will be disconnected unless you instrument the eval handler.

Fix or suggestion

Start an OTel span in the eval handler

Steps:

Instrument your remote eval service with OpenTelemetry.
In the /eval handler, start a span at the start of the request.
Add available Braintrust context from the request body as span attributes, such as parent, project_id, or another stable run identifier your service controls.
Make downstream calls within the span so they appear as child spans.

TypeScript (minimal):

import { context, trace } from "@opentelemetry/api";

const tracer = trace.getTracer("remote-eval");

export async function evalHandler(req, res) {
  const { parent, project_id: projectId } = req.body ?? {};

  const attributes: Record<string, string> = {};
  if (projectId) {
    attributes["braintrust.project_id"] = String(projectId);
  }
  if (parent) {
    attributes["braintrust.parent"] = JSON.stringify(parent);
  }

  const span = tracer.startSpan("remote-eval.request", {
    attributes,
  });

  try {
    await context.with(trace.setSpan(context.active(), span), async () => {
      // perform downstream calls
    });
    res.sendStatus(200);
  } finally {
    span.end();
  }
}

Python (minimal):

from opentelemetry import trace

tracer = trace.get_tracer(__name__)

def eval_handler(request):
    body = request.get_json() or {}
    parent = body.get("parent")
    project_id = body.get("project_id")

    attributes = {
        "braintrust.project_id": project_id,
    }
    if parent is not None:
        attributes["braintrust.parent"] = str(parent)

    with tracer.start_as_current_span(
        "remote-eval.request",
        attributes=attributes,
    ):
        # perform downstream calls
        pass

    return ("OK", 200)

How to confirm it worked

Check your tracing backend for a span named like remote-eval.request and verify it contains attributes: braintrust.project_id and, when present, braintrust.parent.
Verify downstream spans appear as children with the same trace ID (or that trace IDs match between services when propagating traceparent).

All troubleshooting guides

Remote eval OpenTelemetry header propagation

Summary

What is happening

Fix or suggestion

Start an OTel span in the eval handler

How to confirm it worked

Notes

​Summary

​What is happening

​Fix or suggestion

​Start an OTel span in the eval handler

​How to confirm it worked

​Notes

Summary

What is happening

Fix or suggestion

Start an OTel span in the eval handler

How to confirm it worked

Notes