BYOK Setup

Set up your LLM provider API keys for AI-powered llm_judge scoring.

Why BYOK?

LaunchGate’s llm_judge scorer calls external LLMs to evaluate your outputs. Rather than routing through a LaunchGate-managed key, you provide your own API keys. This means:

• Your rate limits — you control throughput
• Your billing — LLM costs are on your provider account
• Your choice of model — use any model your provider offers
• Maximum security — keys are AES-256-GCM encrypted and never logged

Supported providers

Setup

curl -X POST https://api.launchgate.ai/v1/byok-keys \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "openai",
    "name": "Production OpenAI",
    "key": "sk-...",
    "is_default": true
  }'

curl -X POST https://api.launchgate.ai/v1/byok-keys/{keyId}/test \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY"

curl -X POST https://api.launchgate.ai/v1/projects/my-project/scorers \
  -H "Authorization: Bearer $LAUNCHGATE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Faithfulness judge",
    "type": "llm_judge",
    "config": {
      "rubric": "Evaluate faithfulness to the source context...",
      "model": "gpt-4o",
      "_provider": "openai"
    }
  }'

Rate limiting

LLM judge calls are rate-limited to 20 concurrent requests per org per provider. If the limit is hit, requests are queued with exponential backoff (max 3 retries).

Multiple keys per provider

You can add multiple keys per provider (e.g., different keys for different environments). Mark one as is_default — this is the key used when no specific key is referenced.

Security details

• Keys are encrypted with AES-256-GCM using a dedicated BYOK_ENCRYPTION_KEY
• The encryption key never touches the database
• Keys are decrypted only in-memory inside the eval worker at point of use
• Keys are never logged or included in API responses after creation
• The dashboard shows only the key_prefix (first 8 characters)