Skip to Content
API ReferenceRuns

Runs API

Execute a run

POST /v1/run

The core endpoint — triggers an evaluation of your AI output against an eval suite.

Request body

FieldTypeRequiredDefaultDescription
suitestringYes—Suite slug
outputstringYes—AI-generated output to evaluate
inputobjectNo{}Input data for scorers
triggerstringNo"api"Source: sdk, cli, github_action, dashboard, api
trigger_metaobjectNo{}Metadata (e.g., git SHA, PR number)
asyncbooleanNofalseReturn immediately with pending run ID
curl -X POST https://api.launchgate.ai/v1/run \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "suite": "rag-faithfulness",
    "output": "The Eiffel Tower was built in 1889.",
    "input": {
      "context": "The Eiffel Tower was completed in 1889 for the World'\''s Fair.",
      "query": "When was the Eiffel Tower built?"
    }
  }'

Synchronous response 200 OK

{
  "id": "run_abc123",
  "status": "cleared",
  "passed": true,
  "total_cases": 3,
  "passed_cases": 3,
  "failed_cases": 0,
  "pass_rate": 1.0,
  "duration_ms": 842,
  "scores": [
    {
      "case": "Contains source citation",
      "passed": true,
      "score": 1.0,
      "threshold": 1.0,
      "reason": "All required values found in output"
    }
  ],
  "failures": []
}

Asynchronous response 202 Accepted

When async: true:

{
  "id": "run_abc123",
  "status": "pending",
  "message": "Eval run enqueued. Poll GET /v1/runs/:runId for results."
}

Each completed run counts as one usage unit against your plan’s monthly limit. See Billing & Usage.

Get run

GET /v1/runs/:runId

Returns the status and summary of a run. Use this to poll async runs.

Response

{
  "data": {
    "id": "run_abc123",
    "suite_id": "uuid",
    "status": "cleared",
    "pass_rate": 1.0,
    "total_cases": 3,
    "passed_cases": 3,
    "failed_cases": 0,
    "duration_ms": 842,
    "trigger": "sdk",
    "trigger_meta": { "commitSha": "abc123" },
    "created_at": "2026-04-07T12:00:00.000Z",
    "completed_at": "2026-04-07T12:00:01.000Z"
  }
}

List runs for a suite

GET /v1/suites/:suiteId/runs?page=1&pageSize=20

Returns paginated list of runs for a suite, ordered by most recent first.

Get run results

GET /v1/runs/:runId/results

Returns detailed per-case results for a run.

Response

{
  "data": [
    {
      "id": "uuid",
      "run_id": "run_abc123",
      "case_id": "uuid",
      "case_name": "Contains source citation",
      "passed": true,
      "score": 1.0,
      "threshold": 1.0,
      "reason": "All required values found in output",
      "scorer_output": {
        "mode": "all",
        "matches": [
          { "value": "source:", "found": true },
          { "value": "reference", "found": true }
        ],
        "found_count": 2,
        "total": 2
      },
      "duration_ms": 12
    }
  ]
}

scorer_output is scorer-specific debug data. Common shapes by type:

Scorerscorer_output fields
exact_matchexpected, actual, case_sensitive
containsmode, matches (per-value found/not-found), found_count, total
regexpattern, flags, should_match, matched, match_groups
json_schemavalid, errors (on failure), parsed_output (on success)
llm_judgeraw_response (full LLM API response)

On scorer configuration errors (e.g. missing BYOK key) the field is { "error": "<error_code>" }.

Last updated on
ScorersTemplates