Static Site Generators in Production

Checking Links in Pull Requests

Broken links accumulate quietly on documentation sites. A page is renamed and three guides still link to the old path. A heading is reworded and every deep link to its anchor lands at the top of the page instead. A section is deleted and the sidebar still lists it. Readers notice long before anyone on the team does, and scheduled crawls of production only find the damage after it has shipped.

Checking links in every pull request stops most of it at the source. This guide builds the site in CI, checks internal links and heading anchors in the output with lychee as a blocking check, and handles external links separately so they never make builds flaky. It is part of Content Workflows for Documentation Teams.

Prerequisites

Check the Build, Not the Source

Markdown links are only part of the picture. The generator adds navigation, breadcrumbs, pagination, tag pages and "next page" links; it rewrites relative links, applies a base path, and turns headings into anchor IDs with its own slug rules. A checker that reads Markdown sees none of that. A checker that reads the built HTML sees exactly what readers see.

What a source check and a build check can see A source check sees links written in Markdown content only. A build check sees content links plus navigation, breadcrumbs, generated index pages, base path rewriting and heading anchor IDs created by the generator. Coverage of each approach Markdown source check ✓ links written in content ✗ sidebar and header navigation ✗ generated tag and index pages ✗ base path and link rewriting ✗ generator's heading anchor IDs Built HTML check ✓ links written in content ✓ sidebar and header navigation ✓ generated tag and index pages ✓ base path and link rewriting ✓ generator's heading anchor IDs
The build is the product; check the product.

The build takes longer than reading Markdown, but most CI pipelines already build the site for a preview deployment. The link check reuses that output.

Setting Up lychee

lychee checks local HTML files and remote URLs concurrently. Put its settings in lychee.toml at the repository root so local runs and CI behave the same:

# lychee.toml
root_dir = "dist"
include_fragments = true
exclude_path = ["dist/404.html"]
exclude = [
  "^https://example\\.com/",     # our own production domain: checked as local files
  "^mailto:",
]
max_concurrency = 32
accept = ["200..=299", "429"]

root_dir lets lychee resolve root-relative links such as /docs/install/ against the build output. include_fragments checks that anchors exist. Excluding your own production domain avoids checking absolute self-links against the live site, which would miss pages added in the pull request.

Run it locally after a build:

npm run build
lychee --config lychee.toml --offline 'dist/**/*.html'

--offline skips all remote URLs, which is the fast, deterministic check suited to every pull request. On a 2,000-page site it completes in around 20 seconds, less than the build itself, so there is no reason to skip it for small content changes. Add it as an npm script such as npm run check:links so writers can run exactly the same command before pushing.

The Pull Request Workflow

Two jobs: a blocking internal check and a non-blocking external check.

name: Links
on: pull_request

jobs:
  internal:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci && npm run build
      - uses: lycheeverse/lychee-action@v2
        with:
          args: --config lychee.toml --offline 'dist/**/*.html'
          fail: true

  external:
    runs-on: ubuntu-latest
    continue-on-error: true
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci && npm run build
      - uses: actions/cache@v4
        with:
          path: .lycheecache
          key: lychee-${{ github.sha }}
          restore-keys: lychee-
      - uses: lycheeverse/lychee-action@v2
        with:
          args: --config lychee.toml --cache --max-cache-age 3d --scheme https 'dist/**/*.html'
          fail: false

Mark only the internal job as required in branch protection. Internal failures are always caused by the change or by something the team controls, so blocking on them is fair. External failures are often not, so they are reported but never block.

External links fail for reasons unrelated to your change: rate limits, bot protection, slow servers, temporary outages. Three practices keep them useful without noise:

  1. Cache results. lychee's --cache with a maximum age means a URL checked successfully in the last three days is not requested again, which cuts both run time and rate limiting.
  2. Accept 429. "Too many requests" means the link exists; treat it as success.
  3. Check on a schedule too. A nightly run on main with retries catches external links that died since they were added, and can open an issue with the list instead of failing a pull request.
Blocking and non-blocking link checks On each pull request, an offline internal check with anchors runs in about 20 seconds and blocks merge on failure. A cached external check runs alongside and only reports. Nightly on main, a full external check with retries opens an issue listing dead links. Three checks, three purposes PR · internal offline, with anchors ≈ 20 s for 2,000 pages deterministic blocks merge PR · external remote URLs, cached 3 days 429 accepted can be flaky reports only Nightly · main all external URLs retries, longer timeout catches link rot opens an issue only failures the pull request can have caused are allowed to block it
Separating the checks keeps the blocking one fast and trustworthy.

Anchors and Renamed Headings

Heading anchors are the most fragile links on a documentation site. A writer changes "Install requirements" to "System requirements" and every link to #install-requirements silently lands at the top of the page. With include_fragments, lychee reports each one on the pull request that renamed the heading, when the fix is a quick search and replace.

Causes of internal link failures blocked in the first month Of 23 pull requests blocked by the internal link check, 17 were anchors broken by renamed headings, 4 were links to moved or deleted pages, and 2 were typos in hand-written paths. Why 23 pull requests were blocked anchor broken by heading edit 17 page moved or deleted 4 typo in a written path 2 2,100-page documentation site, first 30 days with the required check
Without fragment checking, three quarters of these would have shipped unnoticed.

For headings that are linked from outside the site — support articles, blog posts, other products' docs — keep the old anchor working. Many generators support explicit heading IDs (## System requirements {#install-requirements}), which lets you change the wording without changing the ID.

Renamed and Deleted Pages

When a pull request moves or deletes a page, the internal check lists every page that still links to the old path. Fix the links in the same pull request, and add a redirect for the old URL so external links and bookmarks keep working. See Configuring Redirects on Cloudflare Pages or Netlify Redirects and Rewrites for Static Sites.

A small extra check helps: compare the list of HTML files in the pull request build with the list from main, and fail if a page disappears without a matching redirect rule. It catches the case where nothing internal links to the page any more, but search engines and readers still do.

Measured Impact

A 2,100-page developer documentation site added the internal lychee check as a required status. In its first month it blocked 23 pull requests, 17 of them for anchors broken by heading edits. The scheduled production crawl, which had been finding 30–50 new broken internal links per month, found two in the following quarter, both from content edited directly in production outside the normal workflow. The external job flagged about 1.5% of external links as dead in the first nightly run, which the team cleared over two weeks.

Pitfalls & Rollback

  • Checking the live domain. Absolute links to your own site are checked against production, where new pages do not exist yet; exclude or remap the domain.
  • Base paths. Sites served from a subpath need root_dir pointed at the right folder, or every root-relative link fails.
  • Generated 404 pages. They often contain deliberately relative links; exclude them.
  • Blocking on external links. The check becomes flaky and people learn to ignore or bypass it.
  • Links inside code samples. Example URLs in code blocks are not meant to resolve; exclude placeholder domains such as example.com paths used only in samples.
  • Rollback: make the check non-required; it can still report without blocking.

Conclusion

Checking links on every pull request stops broken links before they reach readers. Build the site, run lychee offline against the output with fragment checking as a required, fast and deterministic check, and handle external links in a separate cached, non-blocking job plus a nightly run. Fix links and add redirects in the same pull request that moves a page, and scheduled crawls of production become a safety net rather than the main defence.

FAQ

The built HTML. Generators rewrite links, add base paths, generate navigation and create anchors from headings, so the output is what readers actually follow. Checking the build catches broken navigation and anchor links that a source check cannot see.

lychee is a fast, widely used choice. It is a single binary written in Rust, checks local files and remote URLs concurrently, supports fragment checking, caching and exclusions, and has an official GitHub Action.

Check internal links on every pull request as a blocking check, and check external links separately - either non-blocking on pull requests or on a nightly schedule with retries and a cache. External sites rate-limit, time out and block bots, none of which your pull request caused.

Yes. lychee's include-fragments option checks that the id in a link such as /docs/install/#requirements exists on the target page. Heading renames are one of the most common sources of broken links in documentation.