Analytics API

Analytics endpoints provide trend data and regression detection for eval suites.

Analytics endpoints require a Pro plan or higher. Free plan users receive HTTP 403 with code PLAN_REQUIRED.

Pass rate trends


GET /v1/suites/:suiteId/trends?days=30

Returns daily pass rate averages for a suite.

Query parameters

Param	Type	Default	Max	Description
`days`	number	`30`	`90`	Number of days to look back

Response


{
  "data": [
    {
      "date": "2026-04-07",
      "avg_pass_rate": 0.92,
      "cleared_count": 18,
      "aborted_count": 2,
      "total_runs": 20
    },
    {
      "date": "2026-04-06",
      "avg_pass_rate": 0.85,
      "cleared_count": 12,
      "aborted_count": 3,
      "total_runs": 15
    }
  ]
}

Regression detection


GET /v1/suites/:suiteId/regressions

Identifies eval cases that previously passed but have recently started failing.

Response


{
  "data": [
    {
      "case_name": "Factual accuracy",
      "case_id": "uuid",
      "current_score": 0.3,
      "previous_score": 0.9,
      "threshold": 0.7,
      "regression": true,
      "reason": "Output contains inaccurate claims about dates"
    }
  ]
}

A case is flagged as a regression when:

It passed in the previous run
It failed in the most recent run
The score dropped by a significant margin