Getting startedPython

deepevalOpen-Source Unit Testing Framework for LLM Apps

deepeval is an open-source Python framework for unit testing LLM applications, offering 50+ ready-made metrics for RAG pipelines, agents, and chatbots. Its style is modeled after Pytest, making it easy to integrate into existing test workflows. The project is released under the Apache-2.0 license and has 16,603 GitHub stars as of collection time.

16.6K Stars

1.6K Forks

340 Issues

193 Views

Python

Apache-2.0

IndexedJuly 3, 2026

GitHub repository Online Demo

Project Overview

deepeval is an open-source Python framework designed for unit testing large language model (LLM) applications. It provides over 50 ready-made metrics for scenarios such as RAG pipelines, agents, and chatbots, helping developers validate output quality.

Core Features

Provides 50+ pre-built test metrics to meet diverse evaluation needs.
Test style mimics Pytest, ensuring ease of adoption and integration.
Supports LLM application types including RAG pipelines, agents, and chatbots.

Tech Stack

The framework is primarily developed in Python.

License

The project is licensed under Apache-2.0, permitting broad usage and modification.

Getting Started

As the README summary does not include specific installation or usage steps, please refer to the project GitHub repository for detailed documentation and examples.

LLM evaluationopen-source frameworkautomated testingmodel qualityPythonunit testingCI integrationhallucination detectionRAG evaluation

Project Rating

0.0 (0 Reviews)

Frequently Asked Questions

What is deepeval: Open-Source Unit Testing Framework for LLM Apps?

What language is deepeval: Open-Source Unit Testing Framework for LLM Apps written in?

deepeval: Open-Source Unit Testing Framework for LLM Apps is primarily written in Python.

What license is deepeval: Open-Source Unit Testing Framework for LLM Apps under?

deepeval: Open-Source Unit Testing Framework for LLM Apps is released under the Apache-2.0 license.

Related Projects

No results yet

Explore More

Similar Tools

CmdRunner

CmdRunner uses AI agents to convert existing test cases into automated browser tests. Upload an Excel sheet or document, and the agent understands the test intent, executes steps in a real browser, validates outcomes, and generates reports automatically. No Selenium, Playwright, or Cypress scripting required. The tool helps reuse existing test cases, reduce automation effort, and accelerate software releases.

Testron

Testron.ai is an AI-driven software testing platform that helps teams automate test case generation, regression test selection, UI automation, visual testing, test data creation, and self-healing test maintenance. Built for modern QA teams, it accelerates testing cycles, reduces manual effort, and improves software quality with intelligent testing agents and enterprise-ready deployment options.