DeepEval

DeepEval is an open-source LLM evaluation framework that lets you write automated tests for your AI application in a way that feels similar to unit testing with pytest. It comes with over 50 built-in metrics covering accuracy, faithfulness, relevance, toxicity, and agent behavior, and it works across RAG pipelines, chatbots, and multi-step agents. Because it integrates directly into CI/CD pipelines, teams can run evals automatically on every code change rather than treating evaluation as a manual, one-off task.