Custom Scores via API/SDKs

Langfuse gives you full flexibility to ingest custom scores via the Langfuse SDKs or API. This allows you to run custom quality checks on the output of your workflows at runtime, or to run custom human evaluation workflows.

Exemplary use cases:

• Deterministic rules at runtime: e.g. check if output contains a certain keyword, adheres to a specified structure/format or if the output is longer than a certain length.
• Custom internal workflow tooling: build custom internal tooling that helps you manage human-in-the-loop workflows. Ingest scores back into Langfuse.

SDKs

Python

langfuse.score(InitialScore(
    trace_id=message.trace_id,
    observation_id=message.generation_id,
    name="quality",
    value=1,
    comment="Factually correct",
))

→ More details in Python SDK docs

JS/TS

await langfuse.score({
  traceId: message.traceId,
  observationId: message.generationId,
  name: "quality",
  value: 1,
  comment: "Factually correct",
});

→ More details in JS/TS SDK docs

API

Check out API reference for more details.