Home › Best AI tools › LLM evaluation and observability tools
LLM evaluation and observability tools
16 tools for tracing, testing and monitoring LLM apps and agents. Platforms and open-source projects that record traces of model and tool calls, run evaluations and monitor quality in production. Compare self-hosting options, framework integrations and how usage is metered.
How to read this page. The tools are grouped by category and are not ranked. Pricing labels come from vendors' own pages when we could confirm them. Always verify plans and limits on the vendor's site.
AI Agents & Frameworks (1)
| Tool | What it does | Pricing | Link |
|---|---|---|---|
| AgentOps | Observability and monitoring for AI agents. | See site | Official site |
AI Developer Tools (15)
| Tool | What it does | Pricing | Link |
|---|---|---|---|
| Arize Phoenix | Open-source observability and evaluation tool for LLM apps. | See site | Official site |
| Braintrust | Evaluation and observability platform for AI products. | Freemium | Official site |
| Confident AI | LLM evaluation platform built on the open-source DeepEval framework. | Freemium | Official site |
| Evidently AI | Open-source and cloud tools for evaluating, testing and monitoring ML and LLM systems. | See site | Official site |
| Galileo | AI evaluation and observability platform for measuring, monitoring and protecting LLM apps and agents. | See site | Official site |
| Helicone | Open-source LLM observability and gateway platform. | Freemium | Official site |
| HoneyHive | Observability and evaluation platform for testing and monitoring AI agents. | Freemium | Official site |
| Laminar | Open-source observability and evaluation platform for AI agents. | Freemium | Official site |
| Langfuse | Open-source LLM engineering platform for tracing, evaluation and prompt management. | Freemium | Official site |
| LangSmith | Observability, evaluation and deployment platform for LLM applications. | See site | Official site |
| Opik | Open-source platform from Comet for evaluating and monitoring LLM applications. | See site | Official site |
| Patronus AI | AI research lab that studies where models fail and builds training data and simulated environments to improve them. | See site | Official site |
| Promptfoo | Open-source tool for testing and red-teaming prompts, models and agents. | See site | Official site |
| Ragas | Open-source toolkit for evaluating retrieval-augmented generation applications. | Free | Official site |
| Weave | Weights & Biases toolkit for tracking and evaluating LLM applications. | See site | Official site |