JS/TS SDK

If you are working with Node.js, Deno, or Edge functions, the langfuse library is the simplest way to integrate Langfuse into your application. The library queues calls to make them non-blocking.

Supported runtimes:

• Node.js
• Edge: Vercel, Cloudflare, ...
• Deno

Want to capture data (e.g. user feedback) from the browser? Use LangfuseWeb

Example

We integrated the Typescript SDK into the Vercel AI Chatbot project. Check out the blog post for screenshots and detailed explanations of the inner workings of the integration. The project includes:

• Streamed responses from OpenAI
• Conversations
• Collection of user feedback on individual messages using the Web SDK

Installation

npm i langfuse
# or
yarn add langfuse
 
# Node.js < 18
npm i langfuse-node
 
# Deno
import { Langfuse } from "https://esm.sh/langfuse"

In your application, set the api keys to create a client.

import { Langfuse } from "langfuse"; // or "langfuse-node"
 
const langfuse = new Langfuse({
  secretKey: process.env.LANGFUSE_SECRET_KEY, // sk-lf-...
  publicKey: process.env.LANGFUSE_PUBLIC_KEY, // pk-lf-...
  // options
});

Options

Making calls

• Each backend execution is logged with a single trace.
• Each trace can contain multiple observations to log the individual steps of the execution.
  • Observations can be of different types
    • Events are the basic building block. They are used to track discrete events in a trace.
    • Spans represent durations of units of work in a trace.
    • Generations are spans which are used to log generations of AI model. They contain additional attributes about the model and the prompt/completion and are specifically rendered in the Langfuse UI.
  • Observations can be nested.

Create a trace

Traces are the top-level entity in the Langfuse API. They represent an execution flow in a LLM application usually triggered by an external event.

// Example trace creation
const trace = langfuse.trace({
  name: "chat-app-session",
  userId: "user__935d7d1d-8625-4ef4-8651-544613e7bd22",
  metadata: { env: "production", user: "user@langfuse.com" },
});
 
// Example update, same params as create, cannot change id
trace.update({
  metadata: {
    tag: "long-running",
  },
});
 
// Properties
trace.id; // string
 
// Create observations
trace.event({});
trace.span({});
trace.generation({});
 
// Add scores
trace.score({});

langfuse.trace() takes the following parameters

Observations

• Events are the basic building block. They are used to track discrete events in a trace.
• Spans represent durations of units of work in a trace.
• Generations are spans which are used to log generations of AI models. They contain additional attributes about the model, the prompt/completion. For generations, token usage is automatically calculated.
• Observations can be nested.

Create an Event

Events are used to track discrete events in a trace.

// Example event
const event = trace.event({
  name: "get-user-profile",
  metadata: {
    attempt: 2,
    httpRoute: "/api/retrieve-person",
  },
  input: {
    userId: "user__935d7d1d-8625-4ef4-8651-544613e7bd22",
  },
  output: {
    firstName: "Maxine",
    lastName: "Simons",
    email: "maxine.simons@langfuse.com",
  },
});
 
// Properties
event.id; // string
event.traceId; // string
event.parentObservationId; // string | undefined
 
// Create children
event.event({});
event.span({});
event.generation({});
 
// Add scores
event.score({});

*.event() takes the following parameters

Create a Span

Spans represent durations of units of work in a trace. We generated convenient SDK functions for generic spans as well as LLM spans.

// Example span creation
const span = trace.span({
  name: "embedding-retrieval",
  input: {
    userInput: "How does Langfuse work?",
  },
});
 
// Example update
span.update({
  metadata: {
    httpRoute: "/api/retrieve-doc",
    embeddingModel: "bert-base-uncased",
  },
});
 
// Application code
const retrievedDocs = await retrieveDoc("How does Langfuse work?");
 
// Example end - sets endTime, optionally pass a body
span.end({
  output: {
    retrievedDocs,
  },
});
 
// Properties
span.id; // string
span.traceId; // string
span.parentObservationId; // string | undefined
 
// Create children
span.event({});
span.span({});
span.generation({});
 
// Add scores
span.score({});

*.span() takes the following parameters

Create a Generation

Generations are used to log generations of AI model. They contain additional attributes about the model and the prompt/completion and are specifically rendered in the Langfuse UI.

// Example generation creation
const generation = trace.generation({
  name: "chat-completion",
  model: "gpt-3.5-turbo",
  modelParameters: {
    temperature: 0.9,
    maxTokens: 2000,
  },
  prompt: messages,
});
 
// Application code
const chatCompletion = await llm.respond(prompt);
 
// Example update
generation.update({
  completionStartTime: new Date(),
});
 
// Example end - sets endTime, optionally pass a body
generation.end({
  completion: chatCompletion,
});
 
// Properties
generation.id; // string
generation.traceId; // string
generation.parentObservationId; // string | undefined
 
// Create children
generation.event({});
generation.span({});
generation.generation({});
 
// Add scores
generation.score({});

*.generation() takes the following parameters

Nesting of observations

Nesting of observations (spans, events, generations) is helpful to structure the trace in a hierarchical way.

# Simple example; there are no limits to how you nest observations
- trace: chat-app-session
  - span: chat-interaction
    - event: get-user-profile
    - generation: chat-completion

There are two options to nest observations:

const trace = langfuse.trace({ name: "chat-app-session" });
 
const span = trace.span({ name: "chat-interaction" });
 
span.event({ name: "get-user-profile" });
span.generation({ name: "chat-completion" });

Create score

Scores are used to evaluate executions/traces. They are attached to a single trace. If the score relates to a specific step of the trace, the score can optionally also be attached to the observation to enable evaluating it specifically.

Links

• Learn more about Scores in Langfuse
• Report scores from the browser (e.g. user feedback) using the Web SDK

await langfuse.score({
  traceId: message.traceId,
  observationId: message.generationId,
  name: "quality",
  value: 1,
  comment: "Factually correct",
});
 
// alternatively
trace.score({});
span.score({});
event.score({});
generation.score({});

Shutdown

The Langfuse SDKs sends events asynchronously to the Langfuse server. You should call shutdown to exit cleanly before your application exits.

langfuse.shutdown();
// or
await langfuse.shutdownAsync();

Debugging

Issues with the SDKs can be caused by various reasons ranging from incorrectly configured API keys to network issues.

The SDK does not throw errors to protect your application process. Instead, you can optionally listen to errors:

langfuse.on("error", (error) => {
  // Whatever you want to do with the error
  console.error(error);
});

Alternatively, you can enable debugging to get detailed logs of what's happening in the SDK.

langfuse.debug();

Short lived execution environments (lambda, serverless)

The SDK is optimize to run in the background to queue and flush all requests. When the process exits use await langfuse.shutdownAsync() to make sure all requests are flushed and pending requests are awaited.

Example:

const langfuse = new Langfuse({
  secretKey: process.env.LANGFUSE_SECRET_KEY, // sk-lf-...
  publicKey: process.env.LANGFUSE_PUBLIC_KEY, // pk-lf-...
})
 
export const handler() {
  const trace = langfuse.trace({ name: "chat-app-session" });
 
  trace.event({ name: "get-user-profile" });
  const span = trace.span({ name: "chat-interaction" });
  span.generation({ name: "chat-completion", model: "gpt-3.5-turbo", prompt, completion });
 
  // So far all requests are queued
 
  // Now we want to flush and await all pending requests before the process exits
  await langfuse.shutdownAsync();
}

Upgrading from v0.x to v1.x

Deprecation of externalTraceId

We deprecated the external trace id to simplify the API. Instead, you can now (optionally) directly set the trace id when creating the trace. Traces are still upserted in case a trace with this id already exists in your project.

// v0.x
const trace = langfuse.trace({ externalId: "123" });
// When manually linking observations and scores to the trace
const span = langfuse.span({ traceId: "123", traceIdType: "EXTERNAL" });
const score = langfuse.score({ traceId: "123", traceIdType: "EXTERNAL" });
 
// v1.x
const trace = langfuse.trace({ id: "123" });
// When manually linking observations and scores to the trace
const span = langfuse.span({ traceId: "123" });
const score = langfuse.score({ traceId: "123" });

Changes

• The traceIdType property is deprecated
• The externalId property on traces is deprecated

Ingestion of externalIds via older versions of the SDK or the API is still supported. However, we will remove support for this in the future and migrate all existing traces to the new format. We monitor the usage of deprecated properties on Langfuse Cloud and will reach out to you if we detect that you are still using them before a breaking change is introduced.

Introduction of shutdownAsync

With v1.0.0 we introduced the shutdownAsync method to make sure all requests are flushed and pending requests are awaited before the process exits. flush is still available but does not await pending requests that are already flushed.

This is especially important for short-lived execution environments such as lambdas and serverless functions.

export const handler() {
  // Lambda / serverless function
 
  // v0.x
  await langfuse.flush();
 
  // v1.x
  await langfuse.shutdownAsync();
}