PR Comments

The GitHub Action automatically posts formatted eval results as a PR comment.

Comment format

When a run completes, the action posts (or updates) a comment like this:


## 🟢 LaunchGate: Cleared for launch
 
**Suite:** rag-faithfulness
**Pass rate:** 100% (3/3)
**Duration:** 842ms
**Run ID:** run_abc123
 
| Case | Score | Threshold | Status |
|---|---|---|---|
| Contains source citation | 1.0 | 1.0 | ✅ |
| Factual accuracy | 0.9 | 0.7 | ✅ |
| No hallucination | 1.0 | 0.8 | ✅ |

For failed runs:


## 🔴 LaunchGate: Launch aborted
 
**Suite:** rag-faithfulness
**Pass rate:** 33% (1/3)
**Duration:** 1234ms
**Run ID:** run_def456
 
| Case | Score | Threshold | Status | Reason |
|---|---|---|---|---|
| Contains source citation | 1.0 | 1.0 | ✅ | — |
| Factual accuracy | 0.3 | 0.7 | ❌ | Output contains inaccurate claims |
| No hallucination | 0.1 | 0.8 | ❌ | Hallucinated information detected |

Idempotent updates

The action uses a hidden marker in the comment to identify it. On subsequent runs for the same suite, the existing comment is updated rather than creating a new one. This keeps PR threads clean.

Each suite gets its own comment. If you run multiple suites in the same workflow, each will have a separate comment.

Disabling comments

If you don’t want PR comments, set comment: "false":


- uses: launchgate/eval-action@v1
  with:
    api-key: ${{ secrets.LAUNCHGATE_API_KEY }}
    suite: "my-suite"
    output: "..."
    comment: "false"

Permissions

The action uses GITHUB_TOKEN to post comments. If your workflow has restricted permissions, ensure it has pull-requests: write:


permissions:
  pull-requests: write