Skip to main content
Opik logo
LLM Ops

Opik

  • Freemium
  • Open source (Apache-2.0)
  • 22.3k

Verified Sep 2026

Open-source LLM evaluation and observability

Trace, evaluate and monitor LLM apps with LLM-as-judge metrics you can self-host.

Opik is the open-source LLM evaluation platform from Comet: tracing of every LLM call with inputs, outputs and costs, datasets and experiment tracking, LLM-as-judge evaluators, hallucination and moderation checks, and a comparison view for prompt and model iterations.

Why founders use it: the failure mode of AI features is silent quality decay, a prompt tweak that fixed one case and broke three, and Opik turns that into a measurable pipeline, every change scored against a dataset before it ships. The tracing works across frameworks with a few lines of integration, and self-hosting keeps the transcripts, your customers' conversations, on infrastructure you control.

A realistic setup: the SDK around model calls, traces per feature and user, a golden dataset of real cases, judge evaluators run on every candidate prompt, and a rule that nothing reaches production without beating the incumbent's scores.

Trade-offs: judge models need calibration against human labels, the self-hosted stack wants containers you babysit, and the space moves fast enough that workflows migrate. Compare with Langfuse for observability breadth, Braintrust for eval-first polish, and promptfoo for the lightweight CLI take.

Who it's for

Agent developers who need tracing plus evaluation in one loop and want the option to self-host.

Keep exploring

Concepts in the glossary

2 in catalog

Opik is an alternative to

5 in catalog

Opik features