Functions

Braintrust functions are atomic, reusable building blocks for executing AI-related logic. Functions are hosted and remotely executed in a performant serverless environment and are fully intended to be used in production. Functions can be invoked through the API, SDK, or the UI, and have built-in support for streaming and structured outputs.

There are currently three types of functions in Braintrust:

• Prompts
  Templated messages to send to an LLM.
• Tools
  General purpose code that can be invoked by LLMs.
• Scorers
  Functions for scoring the quality of LLM outputs (a number from 0 to 1).

Composability

Functions can be composed together to produce sophisticated applications that would otherwise require brittle orchestration logic.

In this diagram, a prompt is being invoked with an input and is calling two different tools and scorers to ultimately produce a streaming output. Out of the box, you also get automatic tracing, including the tool calls and scores.

Any function can be used as a tool, which can be called, and its output added to the chat history. For example, a RAG agent can be defined as just two components:

• A vector search tool, toolRAG,implemented in TypeScript or Python, which embeds a query, searches for relevant documents, and returns them

import { OpenAI } from "openai";
 
async ({ query, top_k }) => {
  const embedding = await openai.embeddings
    .create({
      input: query,
      model: "text-embedding-3-small",
    })
    .then((res) => res.data[0].embedding);
 
  const queryResponse = await pc.query({
    vector: embedding,
    topK: top_k,
    includeMetadata: true,
  });
 
  return queryResponse.matches.map((match) => ({
    title: match.metadata?.title,
    content: match.metadata?.content,
  }));
};

• A system prompt containing instructions for how to retrieve content and synthesize answers using the tool

import * as braintrust from "braintrust";
 
const project = braintrust.projects.create({ name: "Doc Search" });
 
project.prompts.create({
  name: "Doc Search",
  slug: "document-search",
  description:
    "Search through the Braintrust documentation to answer the user's question",
  model: "gpt-4o-mini",
  messages: [
    {
      role: "system",
      content:
        "You are a helpful assistant that can " +
        "answer questions about the Braintrust documentation.",
    },
    {
      role: "user",
      content: "{{{question}}}",
    },
  ],
  tools: [toolRAG],
});

Syncing functions via the SDK

You can sync functions between the Braintrust UI and your local codebase using the Braintrust SDK. Currently, this works for any prompts and tools written in TypeScript.

To push a change from your codebase to the UI, run npx braintrust push <filename> from the command line. You can push one or more files or directories to Braintrust. If you specify a directory, all .ts files under that directory are pushed.

To pull a change from the UI to your codebase, run npx braintrust pull. For example, you can use the pull command to:

• Download functions to public projects so others can use them
• Pin your production environment to a specific prompt version without running them through Braintrust on the request path
• Review changes to functions in pull requests

Code bundling

Braintrust uses esbuild to bundle your code for serverless execution. Bundling works by creating a single JavaScript file that contains all the necessary code, reducing the risk of version mismatches and dependency errors when deploying functions.

Limitations

• Read-only after bundling: Once code is bundled and uploaded to the Braintrust UI, you cannot edit it directly in the UI. Any changes must be made locally in your codebase and pushed via the SDK.
• No dynamic imports: Since esbuild statically analyzes your code, it cannot handle dynamic imports or runtime code modifications.

Runtimes

There are three runtimes available for functions:

• TypeScript (Node.js v18)
• Python (Python 3.11)
• Calling model providers with a prompt via the AI proxy