Evals

DeepEval (Confident AI)

DeepEval is an open-source LLM evaluation package, paired with Confident AI as the hosted observability/evals/monitoring tier. The docs call DeepEval "an open-source LLM eval package" and Confident AI "an AI quality platform with observability, evals, and monitoring." Metrics include GEval (research-backed custom metric), AnswerRelevancyMetric, TaskCompletionMetric, and ConversationalGEval. Test cases use LLMTestCase and ConversationalTestCase shapes; datasets organize Golden test cases for sync or async runs.

GitHub

Documentation

📖
Documentation
https://www.deepeval.com/docs/getting-started

SDKs

📦
GitHubRepository
https://github.com/confident-ai/deepeval

Other Resources

🔗
Portal
https://app.confident-ai.com

API entry from apis.yml

apis.yml Raw ↑ 
name: DeepEval (Confident AI)
description: DeepEval is an open-source LLM evaluation package, paired with Confident AI as the hosted
  observability/evals/monitoring tier. The docs call DeepEval "an open-source LLM eval package" and Confident
  AI "an AI quality platform with observability, evals, and monitoring." Metrics include GEval (research-backed
  custom metric), AnswerRelevancyMetric, TaskCompletionMetric, and ConversationalGEval. Test cases use
  LLMTestCase and ConversationalTestCase shapes; datasets organize Golden test cases for sync or async
  runs.
humanURL: https://www.deepeval.com/
baseURL: https://api.confident-ai.com
tags:
- Open Source
- GEval
- RAG
- Conversational
- Python
properties:
- type: Documentation
  url: https://www.deepeval.com/docs/getting-started
- type: GitHubRepository
  url: https://github.com/confident-ai/deepeval
- type: Portal
  url: https://app.confident-ai.com