BYOK Keys API

Manage encrypted LLM provider API keys for AI-powered scoring.

API keys are encrypted at rest and only returned in plaintext at creation time. Subsequent reads only show the key_prefix.

List BYOK keys


GET /v1/byok-keys?page=1&pageSize=20

Response


{
  "data": [
    {
      "id": "uuid",
      "org_id": "uuid",
      "provider": "openai",
      "name": "Production OpenAI Key",
      "key_prefix": "sk-abc12",
      "is_default": true,
      "last_used_at": "2026-04-07T12:00:00.000Z",
      "created_at": "2026-04-07T12:00:00.000Z"
    }
  ],
  "pagination": { "page": 1, "pageSize": 20, "total": 1, "hasMore": false }
}

Add a BYOK key


POST /v1/byok-keys

Request body

Field	Type	Required	Default	Description
`provider`	string	Yes	—	`openai`, `anthropic`, `google`, `azure_openai`
`key`	string	Yes	—	The raw API key (encrypted before storage)
`name`	string	No	`null`	Display name for the key
`is_default`	boolean	No	`false`	Set as default for this provider


{
  "provider": "openai",
  "name": "Production Key",
  "key": "sk-...",
  "is_default": true
}

Setting is_default: true automatically unsets the previous default for that provider.

Azure OpenAI keys — the key field is your Azure resource API key (found under Keys and Endpoint in the Azure portal). The resource endpoint and deployment name are set separately in the llm_judge scorer config, not in the BYOK key. In the scorer’s model field, use the format "https://my-resource.openai.azure.com/|my-deployment". The endpoint and deployment are split on | at eval time.

Test a BYOK key

This endpoint is not yet available. It is planned for a future release.


POST /v1/byok-keys/:keyId/test

Will validate that the key is accepted by the provider and return:


{
  "valid": true,
  "provider": "openai"
}

Revoke a BYOK key


DELETE /v1/byok-keys/:keyId

Removes the key. Any llm_judge scorers using this provider will fail until a replacement key is added. Returns 204 No Content.