Setting Up AI Code Review in Your CI Pipeline

Automated code review in CI gives every PR a structured quality check before a human even looks at it. Here's how to add it using the Runcept API.

What You Get

• A quality score (0-100) for every push
• A list of issues with severity (error/warning/info)
• Optional: block the merge if score is below threshold
• Optional: post the review as a PR comment

Setup

1. Get a Runcept API key from runcept.com/dashboard
2. Add it as a GitHub repo secret: RUNCEPT_API_KEY

The Workflow

.github/workflows/ai-code-review.yml

name: AI Code Review
on:
  pull_request:
    branches: [main, develop]

jobs:
  review:
    name: AI Review
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get PR diff
        id: diff
        run: |
          git diff origin/${{ github.base_ref }}...HEAD \
            -- . ':(exclude)*.lock' ':(exclude)*.min.*' ':(exclude)dist/' \
            | head -c 12000 > /tmp/review.diff
          echo "Diff size: $(wc -c < /tmp/review.diff) bytes"

      - name: Run AI code review
        id: review
        env:
          RUNCEPT_API_KEY: ${{ secrets.RUNCEPT_API_KEY }}
        run: |
          DIFF_CONTENT=$(cat /tmp/review.diff)
          INPUT=$(jq -n --arg diff "$DIFF_CONTENT" '{"diff": $diff}')

          RESPONSE=$(curl -sS -X POST https://www.runcept.com/api/v1/run \
            -H "Authorization: Bearer $RUNCEPT_API_KEY" \
            -H "Content-Type: application/json" \
            -d "{\"agent\": \"code-reviewer\", \"input\": $INPUT}")

          JOB_ID=$(echo $RESPONSE | jq -r '.job_id')
          echo "Job ID: $JOB_ID"

          SCORE=0
          for i in {1..25}; do
            sleep 5
            JOB=$(curl -sS "https://www.runcept.com/api/v1/jobs/$JOB_ID" \
              -H "Authorization: Bearer $RUNCEPT_API_KEY")
            STATUS=$(echo $JOB | jq -r '.status')

            if [ "$STATUS" = "complete" ]; then
              SCORE=$(echo $JOB | jq -r '.output.score')
              ISSUES=$(echo $JOB | jq -c '.output.issues')
              echo "score=$SCORE" >> $GITHUB_OUTPUT
              echo "issues=$ISSUES" >> $GITHUB_OUTPUT
              echo "Review complete. Score: $SCORE"
              break
            fi

            if [ "$STATUS" = "failed" ]; then
              echo "Review agent failed" && exit 1
            fi
          done

      - name: Post review comment
        uses: actions/github-script@v7
        with:
          script: |
            const score = ${{ steps.review.outputs.score }}
            const issues = ${{ steps.review.outputs.issues }}

            const emoji = score >= 80 ? '✅' : score >= 60 ? '⚠️' : '❌'

            let body = `## AI Code Review ${emoji}\n\n**Score: ${score}/100**\n`

            if (issues && issues.length > 0) {
              body += '\n### Issues\n'
              for (const issue of issues.slice(0, 10)) {
                const icon = issue.severity === 'error' ? '🔴' : '🟡'
                body += `- ${icon} **${issue.severity}**: ${issue.message}\n`
              }
            }

            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body,
            })

      - name: Enforce quality gate
        env:
          SCORE: ${{ steps.review.outputs.score }}
        run: |
          if [ "$SCORE" -lt 65 ]; then
            echo "Code quality score ($SCORE) is below threshold (65). Blocking merge."
            exit 1
          fi
          echo "Quality gate passed: $SCORE/100"

Tuning the Quality Gate

65 is a conservative threshold. You can adjust based on your team's standards:

• Strict teams: 75-80
• Typical teams: 60-70
• Just informational: comment only, no gate

Start without a gate (comment only) for the first week so developers can calibrate expectations.

Handling Large PRs

Large diffs can exceed the agent's input limit. Trim to the most important files:

git diff origin/${{ github.base_ref }}...HEAD \
  -- '*.ts' '*.tsx' '*.py' '*.go' \
  -- ':!*.test.*' ':!*.spec.*' ':!dist/' \
  | head -c 12000

Next Steps

• View all code agents
• Add PR description generation to the same workflow
• Complete GitHub Actions guide