Runs

A run is a single execution of an eval suite. Runs are immutable once complete — they serve as an audit trail of every evaluation.

Run statuses

Status	Brand label	Meaning
`pending`	—	Run is queued (async mode)
`running`	Running pre-launch checks	Cases are being evaluated
`cleared`	Cleared for launch ✓	Pass rate met the suite threshold
`aborted`	Launch aborted ✗	Pass rate fell below the suite threshold
`error`	—	An unexpected error occurred during evaluation
`skipped`	—	SDK could not reach the API after retries; evaluation was bypassed so your application continues running

Triggering a run

Runs can be triggered from any integration:

Trigger	Method
SDK	`lg.run("suite-slug", { output: "..." })`
CLI	`launchgate run suite-slug --output "..."`
GitHub Action	`uses: launchgate/eval-action@v1`
Dashboard	Manual run from the suite page
REST API	`POST /v1/run`

Sync vs async

Synchronous (default)

The API blocks until all cases are evaluated and returns the full result:


const result = await lg.run("my-suite", {
  output: "...",
});
// result.status is "cleared" or "aborted"

Asynchronous

The API returns immediately with a pending status and a run ID. Poll for results:


const result = await lg.run("my-suite", {
  output: "...",
  async: true,
});
// result.status is "pending"
// result.id can be used to poll: GET /v1/runs/:runId

Polling example — check every 2 seconds until the run completes:


async function pollRun(runId: string, intervalMs = 2000, maxAttempts = 30) {
  for (let i = 0; i < maxAttempts; i++) {
    const run = await lg.getRun(runId);
    if (run.data.status !== "pending" && run.data.status !== "running") {
      return run.data;
    }
    await new Promise((resolve) => setTimeout(resolve, intervalMs));
  }
  throw new Error(`Run ${runId} did not complete within the polling window`);
}
 
const completed = await pollRun(result.id);
console.log(completed.status); // "cleared" or "aborted"

Async mode is useful for suites with many LLM judge cases that may take longer to evaluate. The run is processed by a background worker.

Run result shape


{
  id: string;
  status: "cleared" | "aborted" | "error" | "skipped";
  passed: boolean;
  passRate: number;       // 0.0 - 1.0
  totalCases: number;
  passedCases: number;
  failedCases: number;
  durationMs: number;
  scores: Score[];        // All case scores
  failures: Score[];      // Only failed cases
}

Each Score contains:


{
  case: string;       // Case name
  passed: boolean;
  score: number;      // 0.0 - 1.0
  threshold: number;  // The case threshold
  reason: string;     // Human-readable explanation
}

Immutability

Runs and their results are immutable — they cannot be updated or deleted after creation. This guarantees a reliable audit trail for compliance and debugging.

Usage metering

Each completed POST /v1/run call counts as one run against your plan’s monthly limit.