Skip to content
masterclass Buddy
Start learning

LangSmith · Module

Online evaluators

Online code evaluators run checks on selected production traces and add feedback. You configure them in LangSmith, filter the runs, and write the check in Python or JavaScript.

Last updated

Since
v0.14

Syntax

def evaluate(run):
    return {"key": "feedback_key", "score": value}

In-depth API docs are only for subscribers: parameters, examples, common mistakes and interview practice.

Subscribe
  • Online evaluations
  • Code evaluators
  • Filter traces (ClickHouse)
  • Chain evaluators
  • Adding metadata to your traces

Source: Set up online code evaluators. Checked 2026-11-03.