GitHub Action

The launchgate/eval-action GitHub Action runs LaunchGate evaluations as part of your CI/CD pipeline, with automatic PR comment reporting.

Quick start

.github/workflows/eval.yml


name: LaunchGate Eval
 
on:
  pull_request:
    branches: [main]
 
jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
 
      - name: Generate AI output
        id: generate
        run: |
          # Your AI generation step here
          echo "text=The Eiffel Tower was built in 1889." >> $GITHUB_OUTPUT
 
      - name: Run LaunchGate Eval
        uses: launchgate/eval-action@v1
        with:
          api-key: ${{ secrets.LAUNCHGATE_API_KEY }}
          suite: "rag-faithfulness"
          output: ${{ steps.generate.outputs.text }}
          input: '{"context": "The Eiffel Tower was completed in 1889."}'

What it does

Sends your AI output to LaunchGate for evaluation
Waits for the eval run to complete
Posts a formatted comment on the PR with results
Fails the workflow step if the eval is “aborted” (configurable)

The action automatically captures GitHub context (repo, SHA, ref, branch, PR number) and attaches it as trigger metadata for traceability in the dashboard.