Testing Static Sites in CI

"It's just a static site" is the reason most static sites have no tests, and the reason they break in the same ways over and over. There is no server to fail, but there is a build that turns hundreds of templates and thousands of Markdown files into HTML, a host that applies redirects and headers, and a handful of scripts that make search, theme toggles and menus work. A one-line template change can remove the canonical tag from every page, drop the lang attribute from a whole locale, or break every redirect from the last migration — and nothing in the browser looks wrong.

This guide builds a CI test suite aimed at the ways static sites actually fail, organised by what each test needs: the build directory, a running preview, or a real browser. It is written for teams running Astro, Eleventy, Hugo or Next.js static export on GitHub Actions with preview deploys, and it sits within Production-Ready Deployment & CI/CD Workflows. It complements Checking Links in Pull Requests and Visual Regression Testing on Preview Deploys, which already exist as separate practices.

The numbers come from two sites over twelve months: a 2,300-page Hugo product documentation site and a 410-page Astro marketing and docs site. Both ran every test in this guide on every pull request.

What Breaks on Static Sites

Before choosing tools, it helps to know what actually went wrong. Over twelve months the two sites recorded every production defect and every defect caught in CI, and classified them by cause:

Almost nothing on the list involves logic that a unit test would cover. The suite is therefore mostly output testing: assert things about the HTML and HTTP responses the site produces.

Layer 1: Tests Against the Build Output

The cheapest and most valuable tests read files from dist/ and need no server.

HTML validation. Templates that produce unclosed elements, duplicate IDs or invalid nesting render fine in browsers, which repair them silently, but they break assistive technology, search indexers and anything else that parses the HTML. html-validate checks thousands of pages in seconds with rules you choose; see Validating HTML Output in CI.

Internal links and anchors. A crawler over the build directory finds links to missing pages and #fragment links to headings that were renamed. This was the single largest source of defects caught on the documentation site.

Snapshots of built output. Store a normalized snapshot of key pages' HTML (or a structural summary of every page) and diff it on each pull request. A template refactor that should change nothing then proves it changed nothing; see Snapshot Testing Build Output.

Metadata and structured data. Every page has one title, one description, one canonical, valid JSON-LD — checked by a script, as in Testing Structured Data and Meta Tags.

# .github/workflows/test.yml (layer 1)
- run: npm ci && npm run build
- run: npx html-validate "dist/**/*.html"
- run: npx linkinator dist --recurse --skip "^(?!http://localhost)" --silent
- run: node tests/metadata.mjs dist
- run: npx vitest run tests/snapshots

Layer 2: Tests Against the Preview Deploy

Some behaviour exists only on the host: redirect rules in _redirects or vercel.json, headers from _headers, trailing-slash handling, compression, and edge functions. These cannot be tested from files. Run them against the preview URL once the preview deploy is ready, as described in Automating Preview Deploy Pipelines with GitHub Actions.

A table-driven test suite covers most of it: a list of URLs with expected status, Location, and headers. The documentation site keeps 1,400 redirect expectations generated from its migration map and 30 header expectations. The details are in Testing Redirects and Headers Before Deploy.

// tests/http.spec.mjs — excerpt
const cases = [
  { path: '/docs/old-install', status: 301, location: '/guide/install/' },
  { path: '/', headers: { 'strict-transport-security': /max-age=\d{8}/ } },
  { path: '/assets/app.3f9a1c2e.js', headers: { 'cache-control': /immutable/ } }
];

Layer 3: Tests in a Real Browser

Interactive features and accessibility need a browser. Keep this layer small and sampled, because it is the slowest and the most prone to flakiness.

Accessibility. axe-core catches roughly a third to a half of WCAG issues automatically — contrast, missing labels, heading order, lang, ARIA misuse — which is most of the regressions templates introduce. Run it on one or two representative pages per template, in light and dark themes, plus any page changed in the pull request. Accessibility Testing with axe and pa11y in CI covers the setup.

End-to-end flows. A static site has a handful of flows worth protecting: search returns results and navigates, the theme toggle persists, the mobile menu opens and traps focus, code copy buttons copy, forms submit to their function. Each is a short Playwright test against the preview; see End-to-End Tests for Static Sites with Playwright.

Visual regression. Screenshots of key templates at two widths, compared against approved baselines, catch the CSS changes nothing else notices.

Sampling is what keeps this layer affordable. The documentation site has 2,300 pages but only 14 templates; running axe on 28 pages (two per template, both themes) plus changed pages took 70 seconds, and a full-site axe run done monthly found only two issues the sample had missed in a year.

Running It in GitHub Actions

Three jobs map onto the three layers. The build job uploads dist as an artifact and runs layer 1; the deploy job publishes the preview and outputs its URL; the test job runs layers 2 and 3 against that URL.

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: npm }
      - run: npm ci && npm run build
      - run: npm run test:output          # html-validate, links, metadata, snapshots
      - uses: actions/upload-artifact@v4
        with: { name: dist, path: dist }
  preview:
    needs: build
    outputs: { url: ${{ steps.deploy.outputs.url }} }
    steps: [ ... deploy dist, set url ... ]
  e2e:
    needs: preview
    runs-on: ubuntu-latest
    container: mcr.microsoft.com/playwright:v1.48.0-noble
    env: { BASE_URL: ${{ needs.preview.outputs.url }} }
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:http && npx playwright test

Using the official Playwright container avoids downloading browsers on every run, which saved about 50 seconds per run on both sites. Mark layer 1 and 2 as required checks; keep flaky-prone browser tests required too, but with retries: 2 in Playwright and tolerances in visual comparisons.

Choosing Tools

The tooling landscape is wide, and most of it overlaps. The combination that worked on both sites, chosen for speed, local reproducibility and readable failures:

Two things are deliberately absent. Unit-test frameworks for template code rarely paid off: templates are best tested through their output. And a single all-in-one "site quality" SaaS was tried on the Astro site and dropped, because it ran on its own schedule against production rather than on pull requests, which is too late.

Whatever the tools, standardise the interface: every check is an npm script (test:output, test:http, test:e2e), runs locally against dist/ or a BASE_URL, and prints failures with a page path. Writers and developers then run the same commands CI runs.

Content Is Code Too

On documentation sites, most pull requests are content changes by writers, and the suite should serve them rather than slow them down. Three adjustments helped.

Run the cheap layer on every pull request and the browser layer only when templates, components, styles or scripts change, detected with a paths filter in the workflow. A typo fix then gets a two-minute check instead of six.

Surface failures in writer terms. "Broken link on /guide/install/ to /guide/configure/#proxy — the heading 'Proxy' no longer exists on that page" is actionable for a writer; a stack trace from a link checker is not.

Add editorial checks to the same pipeline: Vale for style rules, a spell checker with a project dictionary, and a rule that every new page has a description and at least one internal link in. Those are covered in Editorial Checks with Vale in CI and Content Linting with markdownlint.

What the Suite Costs

The suite is not free, and it is worth being honest about its costs. CI minutes rose from about 40 seconds per pull request (link check only) to 5.7 minutes on the Hugo site, roughly 1,900 extra runner minutes a month across 330 pull requests — within the free allowance for public repositories, and about the cost of one hour of engineering time a month for private ones. Maintenance took around two hours a month: updating baselines, adjusting a selector after a redesign, adding redirect cases after a restructure. Against 108 fewer production defects a year, most of which had each cost an hour or more to diagnose after readers reported them, the trade was clearly positive on both sites.

Keeping the Suite Trustworthy

A test suite nobody trusts is worse than none, because failures get re-run until green and real regressions slip through. Three practices kept both suites trusted.

Deterministic by default. Tests that read files or compare HTTP responses from a static preview do not flake. The flaky tests were all in the browser layer, and nearly all were caused by waiting for fixed timeouts instead of for state (await page.waitForTimeout(500) instead of await expect(dialog).toBeVisible()).

External dependencies as reports. External link checks, Lighthouse scores and third-party embeds depend on the network. They run on a schedule and post reports rather than blocking pull requests — see Crawling for Broken Links on a Schedule and Reducing Lighthouse Score Variance in CI.

Failures that explain themselves. Every assertion message names the page, the rule and the expected value. "canonical on /guide/install/ is https://docs.example.com/guide/install (missing trailing slash), expected …/install/" gets fixed in minutes; "expected true, received false" gets re-run.

Adding the Suite to an Existing Site

Introducing all of this at once to a mature site produces a first run with thousands of failures and a team that disables the checks by Friday. Both sites rolled the suite out in the same order over about six weeks, and the order mattered.

Week one added the internal link check and HTML validation as reports only, then fixed the handful of template bugs responsible for most findings. Week two made both required and added the metadata script as a report. Week three added redirect and header tests against previews, generated from the existing redirect map, and made them required immediately, since they had no backlog — either a redirect worked or it did not. Weeks four and five introduced accessibility checks with a committed baseline, and end-to-end tests for the five most-used interactive features. Week six added snapshots, once the team had seen enough template-driven surprises to want them.

The principle behind the order: start with checks whose failures are unambiguous and cheap to fix, make each one required as soon as its backlog is clear, and use baselines for anything with a large backlog. Every check that was made required early stayed required; the one check that was made required before its backlog was cleared (description length, on the Astro site) was switched back to a warning within a week.

Measured Impact

Common Pitfalls

• Testing the source instead of the output. Linting Markdown does not catch template bugs; test dist/.
• Full-site browser runs on every pull request. Slow and rarely more useful than a sample per template.
• Redirect tests against the build directory. Redirects live on the host; test them on the preview.
• Ignoring dark mode. Contrast regressions often exist in one theme only; run axe in both.
• Fixed timeouts in Playwright. The main source of flakiness; wait for state.
• External links as a blocking check. Third-party outages then block your merges.

Key Takeaways

• Static sites fail through their output; test the HTML and HTTP responses, not the code.
• Organise tests by what they need: build directory, preview deploy, or browser.
• Put the cheap deterministic checks first and make them required.
• Sample browser tests by template, and include both themes for accessibility.
• Keep network-dependent checks as scheduled reports.
• Write assertion messages that name the page and the expected value.

FAQ

What is worth testing on a site with no backend?

The built output itself. Static sites fail through broken templates, missing pages, invalid HTML, accessibility regressions, redirects that stopped working, headers that disappeared and metadata that went missing. All of those can be checked deterministically against the build before it deploys.

Should tests run against the build directory or a preview deploy?

Both, for different things. Checks that only need files, such as HTML validation, snapshot diffs and metadata checks, run fastest against the build directory. Anything that depends on the host, such as redirects, headers, compression and edge functions, has to run against a preview deploy.

How long should a static site test suite take?

Under five minutes for the pull request gate is a practical target, because slower suites get skipped or ignored. Sampling helps, since running axe or Playwright on every page of a 3,000-page site is rarely necessary when pages share templates.

Which tool should I start with?

An internal link checker and an automated accessibility check on one page per template give the most value for the least setup. Add header and redirect tests next if the site has a history of migrations, then snapshot and metadata tests.

How do I stop flaky tests from blocking deploys?

Keep deterministic checks such as HTML validation, links in the build output and snapshots as hard gates, and treat checks that depend on the network or timing, such as external links and Lighthouse scores, as reports or with retries and tolerances.

Related

• Up: Production-Ready Deployment & CI/CD Workflows — where testing fits in the pipeline.
• Accessibility Testing with axe and pa11y in CI — automated WCAG checks.
• End-to-End Tests for Static Sites with Playwright — protecting interactive flows.
• Validating HTML Output in CI — catching template markup errors.
• Testing Redirects and Headers Before Deploy — host behaviour on the preview.
• Snapshot Testing Build Output — proving refactors change nothing.
• Testing Structured Data and Meta Tags — SEO output checks.
• Checking Links in Pull Requests — link checking in depth.
• Running Smoke Tests Against a Preview URL — the minimal post-deploy check.
• Fixing Flaky Playwright Tests Against Preview Deploys — stabilise Playwright tests that run against preview URLs: wait for the right deploy, warm caches, drop timing assumptions and isolate third parties.
• Testing Search and Navigation Flows With Playwright — test the interactive parts of a static docs site with Playwright: search results and keyboard use, mobile menus, sidebar state, versions and 404s.