Mastra

Mastra Evals

Built-in evaluation library with model-graded (LLM-as-judge), rule-based, and statistical metrics for measuring agent output quality, hallucination, faithfulness, relevance, bias, toxicity, and answer correctness. Evals run locally or against Mastra Cloud and emit traces alongside production runs.

Mastra Evals is one of 12 APIs that Mastra publishes on the APIs.io network.

Tagged areas include Evaluation, Testing, and LLM-as-Judge. The published artifact set on APIs.io includes API documentation.

Documentation GitHub

Documentation

https://mastra.ai/docs/evals/overview

Other Resources

https://github.com/mastra-ai/mastra/tree/main/packages/evals

https://www.npmjs.com/package/@mastra/evals

API entry from apis.yml

aid: mastra-ai:mastra-evals
name: Mastra Evals
description: Built-in evaluation library with model-graded (LLM-as-judge), rule-based, and statistical
  metrics for measuring agent output quality, hallucination, faithfulness, relevance, bias, toxicity,
  and answer correctness. Evals run locally or against Mastra Cloud and emit traces alongside production
  runs.
humanURL: https://mastra.ai/docs/evals/overview
tags:
- Evaluation
- Testing
- LLM-as-Judge
properties:
- type: Documentation
  url: https://mastra.ai/docs/evals/overview
- type: SourceCode
  url: https://github.com/mastra-ai/mastra/tree/main/packages/evals
- type: PackageManager
  url: https://www.npmjs.com/package/@mastra/evals