LLM CI/CD Automated Code Review Guide [2026]

In the fast-paced environment of Agile development and Continuous Integration/Continuous Delivery (CI/CD), Code Review (CR) often becomes the bottleneck for R&D teams.

Manual CR is time-consuming and laborious, and due to reviewers' limited energy, it often becomes a formality (like only looking at code style while ignoring potential concurrency or security vulnerabilities). Traditional static scanning tools (such as SonarQube, ESLint) are fast, but they can only act based on preset rules. They cannot understand business logic, let alone propose architectural-level refactoring suggestions.

The emergence of Large Language Models (LLMs) provides a revolutionary solution to this pain point. This article will teach you step-by-step how to deeply integrate LLMs into your CI/CD pipeline to build a tireless, eagle-eyed AI Code Reviewer.

1. Why Do We Need AI Code Review?

Compared to traditional static scanning, the core advantage of LLMs in code review lies in their powerful contextual understanding capabilities and semantic reasoning capabilities:

• Discover Deep Logical Vulnerabilities: AI can identify performance anti-patterns like "frequently querying the database in a loop," whereas static tools usually struggle to trace data flow across files.
• Business Semantic Checks: AI can understand the intent of variable naming and comments, pointing out business-level defects like "Although the code has no syntax errors, it doesn't verify the order status when processing refund logic."
• Constructive Refactoring Suggestions: Not only does AI point out problems, but it can also directly provide modified code snippets.

2. Core Architecture Design of AI Code Review

To build an automated AI review bot, its workflow typically looks like this:

Key Challenge: Context Assembly

Although large models are smart, their input window (Context Window) is limited and billed per Token. If you throw the entire repository's code at it, not only is it extremely expensive, but it will also cause it to "get lost" in the long context.

Best Practice: Precise Slicing Based on Git Diff

We should not submit the entire file, but only extract the parts modified in this Pull Request (Git Diff) and attach a few lines of Context Lines appropriately.

3. Practical Guide: Building an Automated Review Bot Using GitHub Actions

Below, we will demonstrate how to intercept a PR and call an LLM for review through a real GitHub Actions script.

3.1 Write the GitHub Actions Workflow File

Create .github/workflows/ai-code-review.yml in your project's root directory:

name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize] # Triggered when a PR is created or updated

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0 # Fetch complete git history to generate accurate diffs

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm install axios @actions/github @actions/core

      - name: Run AI Review Script
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: node scripts/ai-reviewer.js

3.2 Write the Core Review Script (ai-reviewer.js)

In this script, we will get the PR's changed files, send the Diff content to the LLM, and write the review comments back to the GitHub comments section. Note that the GitHub API already returns each file's changes as a ready-to-use unified diff (file.patch), so no separate diff-formatting step is needed.

const github = require('@actions/github');
const core = require('@actions/core');
const axios = require('axios');

async function run() {
  try {
    const token = process.env.GITHUB_TOKEN;
    const openaiKey = process.env.OPENAI_API_KEY;
    const octokit = github.getOctokit(token);
    const context = github.context;

    if (context.eventName !== 'pull_request') return;

    const prNumber = context.payload.pull_request.number;
    const owner = context.repo.owner;
    const repo = context.repo.repo;

    // 1. Get all file changes (Diff) for the PR
    const { data: files } = await octokit.rest.pulls.listFiles({
      owner,
      repo,
      pull_number: prNumber,
    });

    for (const file of files) {
      // Ignore lock files, images, or overly large changes
      if (file.filename.endsWith('.lock') || file.changes > 500) continue;

      const diffContent = file.patch;
      if (!diffContent) continue;

      // 2. Build Prompt, asking LLM to review
      const prompt = `
You are a senior software architect. Please review the following code changes (in Git Diff format) and point out potential bugs, performance issues, or security risks.
If the code has no obvious problems, please reply "LGTM". Otherwise, please provide specific modification suggestions.

File Name: ${file.filename}
Changes:
\`\`\`diff
${diffContent}
\`\`\`
      `;

      // 3. Call the OpenAI API. Model names change quickly — check the
      // provider's current model list and pick a capable, cost-appropriate
      // reasoning model rather than hard-coding an old snapshot.
      const response = await axios.post(
        'https://api.openai.com/v1/chat/completions',
        {
          model: process.env.OPENAI_MODEL || 'gpt-4o',
          messages: [{ role: 'user', content: prompt }],
          temperature: 0.2,
        },
        {
          headers: {
            'Authorization': `Bearer ${openaiKey}`,
            'Content-Type': 'application/json',
          },
        }
      );

      const reviewComment = response.data.choices[0].message.content;

      // 4. Post review comments as a PR comment
      if (reviewComment && reviewComment.trim() !== "LGTM") {
        await octokit.rest.pulls.createReviewComment({
          owner,
          repo,
          pull_number: prNumber,
          body: `🤖 **AI Reviewer**:\n\n${reviewComment}`,
          commit_id: context.payload.pull_request.head.sha,
          path: file.filename,
          position: 1, // Simplified example; actually calculate the specific line number based on the diff
        });
      }
    }
  } catch (error) {
    core.setFailed(error.message);
  }
}

run();

4. Advanced: Automated Generation of Missing Unit Tests

Besides pointing out problems, LLMs can also directly generate productivity. We can extend the above CI script so that when it detects that core business logic has been modified but there are no corresponding .test.js or _spec.py file updates, it automatically triggers another process:

Test Generation Prompt Example:

As a QA engineer, please write complete unit tests for the following newly added function (using the Jest framework).
Requirements:
1. You must cover all boundary conditions (like empty inputs, maximum values).
2. You must Mock external dependencies.
3. Please only output the test code itself.

[Newly added function code]

In this way, AI acts as an extra pair of eyes on every PR and can proactively improve the project's test coverage. Keep in mind that posting a review comment does not by itself block a merge — if you want a hard gate, wire the review job into a required status check (and remember that LLM feedback is advisory, so a human still owns the final decision).

FAQ

Q: AI often makes "nitpicky" suggestions (like suggesting changing var to let), making the comments section extremely noisy. How can I solve this? A: This is the most common problem when initially integrating AI Review. The solution is to set strong constraints in the System Prompt, for example: "Only report severe issues that could cause runtime errors, significant performance degradation, or security vulnerabilities. Ignore minor details like code style (indentation, variable declaration methods)."

Q: The company's code contains sensitive business logic and cannot be sent to the public OpenAI network. What should I do? A: You can combine this with the previous article in this series, "Ollama Advanced Practical Guide," deploy an open-source model (like Qwen2.5-Coder) on the company intranet, and point the API endpoint in the CI script to the intranet Ollama.

Q: How do I efficiently test the prompts for my AI Code Review bot? A: Iterate locally before wiring anything into CI. Save a few representative diffs to disk, run your prompt against them from a small script (or the provider's playground), and eyeball whether the feedback is useful and low-noise. If you later switch to a structured output format (for example, asking the model to return JSON so you can render inline comments precisely), add a schema-validation step in the script and fail fast when the response doesn't parse — that keeps a malformed model response from crashing the whole pipeline.

Conclusion

Deeply integrating LLMs into the CI/CD pipeline is a practical step for R&D teams moving toward "AI-Native Engineering." A well-configured AI reviewer works like a tireless second reviewer that is familiar with common best practices, helping surface issues earlier and keep technical debt in check. Treat it as an assistant to your human review process rather than a replacement for it: the model catches things people miss, and people catch the things the model gets wrong.