Eval Suites

An eval suite is a collection of test cases that evaluate your AI output. Think of it as a “launch checklist” — a set of quality checks that must pass before deployment.

Key properties

Pass threshold

The pass_threshold determines whether a run is cleared or aborted:

• If the pass rate ≥ threshold → "cleared" (Cleared for launch ✓)
• If the pass rate < threshold → "aborted" (Launch aborted ✗)

For example, with a threshold of 0.8 and 10 cases, at least 8 must pass for the run to be cleared.

Creating a suite

From scratch

curl -X POST https://api.launchgate.ai/v1/projects/my-project/suites \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "RAG Faithfulness",
    "pass_threshold": 0.8
  }'

From a template

LaunchGate includes 6 pre-built templates for common use cases:

curl -X POST https://api.launchgate.ai/v1/projects/my-project/suites/from-template \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "template_slug": "rag-faithfulness"
  }'

See Using Templates for the full list.

Suite operations