Running Smoke Tests Against a Preview URL

Build gates check the artifact. Smoke tests check the deployment — and there is a whole category of failure that only exists between the two: a redirect rule that swallows a path, a base URL pointing at the wrong environment, a deploy-ignore pattern that excluded the asset directory, a host that serves /guides but not /guides/.

These failures produce a build that passes every check and a site that is broken for readers. Ten seconds of HTTP requests against the deployed preview catches nearly all of them. This guide is the check set, the wiring, and what to do when one fails. It is part of Rollbacks and Deploy Safety for Static Sites.

Prerequisites

• A per-pull-request preview URL — see Preview Environments for Pull Requests for the deployment side.
• A CI step that can run Node or curl after the preview is live and knows its URL.
• A short list of representative URLs: the homepage plus one page per template.

What Belongs in a Smoke Test

The Check Set

Five checks cover the failure modes that reach production:

1. Status codes. Every representative URL returns 200 — including the trailing-slash form your site actually links to.

2. Template markers. Each page contains the markup that proves its layout rendered: a header class, a footer class, an <h1>. A missing partial produces a page that returns 200 and is unusable.

3. Referenced assets exist. Parse each page's href/src attributes and HEAD every same-origin asset. This is the check that catches deploy-ignore mistakes and asset-pipeline failures.

4. Sitemap parses and matches. The sitemap is XML, it contains entries, and a sample of those entries return 200.

5. One redirect resolves. Pick a known redirect and assert it returns 301 to a URL that returns 200 in one hop.

// scripts/smoke.mjs <base-url>
const base = process.argv[2].replace(/\/$/, '');
if (!base) { console.error('usage: smoke.mjs <base-url>'); process.exit(2); }

const PAGES = [
  { path: '/', must: ['site-header', 'site-footer', '<h1'] },
  { path: '/guides/', must: ['site-header', '<h1', 'href="/guides/'] },
  { path: '/guides/deploying/', must: ['site-header', 'site-footer', '<h1', '</article>'] },
  { path: '/about/', must: ['site-header', '<h1'] },
];
const REDIRECT = { from: '/old-docs/deploying/', to: '/guides/deploying/' };

let failures = 0;
const fail = (msg) => { console.error(`FAIL ${msg}`); failures++; };

for (const { path, must } of PAGES) {
  const res = await fetch(base + path, { redirect: 'manual' });
  if (res.status !== 200) { fail(`${path} → HTTP ${res.status}`); continue; }
  const html = await res.text();
  for (const marker of must) if (!html.includes(marker)) fail(`${path} missing marker ${marker}`);

  const refs = [...html.matchAll(/(?:href|src)="(\/[^"#?]+\.(?:css|js|svg|png|avif|webp|woff2))"/g)]
    .map((m) => m[1]);
  for (const ref of [...new Set(refs)]) {
    const head = await fetch(base + ref, { method: 'HEAD' });
    if (!head.ok) fail(`${path} references missing asset ${ref} (${head.status})`);
  }
}

const sm = await fetch(base + '/sitemap.xml');
const smText = sm.ok ? await sm.text() : '';
if (!sm.ok || !smText.includes('<urlset')) fail(`sitemap.xml → ${sm.status}`);
const locs = [...smText.matchAll(/<loc>([^<]+)<\/loc>/g)].map((m) => m[1]);
if (locs.length < 5) fail(`sitemap has only ${locs.length} entries`);

const r = await fetch(base + REDIRECT.from, { redirect: 'manual' });
if (r.status !== 301) fail(`${REDIRECT.from} → ${r.status}, expected 301`);
else {
  const dest = new URL(r.headers.get('location'), base);
  const destRes = await fetch(dest, { redirect: 'manual' });
  if (destRes.status !== 200) fail(`${REDIRECT.from} → ${dest.pathname} → ${destRes.status}`);
}

console.log(failures ? `smoke: ${failures} failure(s)` : `smoke: OK (${PAGES.length} pages checked)`);
process.exit(failures ? 1 : 0);

Wire It Into the Pipeline

# .github/workflows/preview.yml (excerpt)
- name: Deploy preview
  id: preview
  run: |
    URL=$(npx wrangler versions upload --json | jq -r '.preview_url')
    echo "url=$URL" >> "$GITHUB_OUTPUT"

- name: Smoke test the preview
  run: node scripts/smoke.mjs "${{ steps.preview.outputs.url }}"

- name: Comment the preview URL
  uses: actions/github-script@v7
  with:
    script: |
      github.rest.issues.createComment({
        issue_number: context.issue.number,
        owner: context.repo.owner, repo: context.repo.repo,
        body: `Preview: ${{ steps.preview.outputs.url }} — smoke tests passed.`
      })

Run the same script against production immediately after promotion. The two runs use identical code, which means a production failure is unambiguous: the artifact was fine on the preview host, so the difference is environmental.

Keeping the Suite Honest

A smoke suite decays in two directions, and both are worth guarding against.

It grows, because every incident produces a suggestion to add a check. Most of those belong in build gates, where they run on every page rather than on four. Before adding a check, ask whether it could have been caught pre-deploy; if it could, add it there instead. Cap the suite at a fixed wall-clock budget — fifteen seconds — and treat exceeding it as a signal to move something out.

It also rots, because markers drift. A CSS refactor renames site-header and the check silently starts asserting on a class that never appears, which is a test that can only pass by accident. Guard against this by asserting the markers exist on a known-good build in CI: if a marker matches zero pages of the current build, fail the test suite rather than the deploy, so the failure is understood as maintenance rather than an incident.

Finally, keep the URL list representative rather than historical. When a new template ships, add a page of it and remove one of the two article pages that now test the same code path. The suite should always be one page per template, not one page per template you have ever had.

Measured Impact

Six months of pipeline data from a documentation site deploying roughly forty times a month:

Of the nine preview catches, five were missing assets after a build-tool upgrade changed the output directory, three were redirect rules that shadowed real pages, and one was a base URL left pointing at staging. None is detectable from the build directory alone.

Reading a Failure

A smoke failure names a URL and a reason, which is usually enough to classify it in seconds.

A non-200 status on a page that exists in the build points at routing: a redirect rule that matched too broadly, a trailing-slash mismatch, or a host that has not finished propagating the deployment. Check the redirect file first; it is the most common cause by a wide margin.

A missing marker on a page that returns 200 points at a template: a partial that failed to render, a layout that resolved to the wrong file, or a component that threw and was swallowed. The build log usually has a warning that nobody read.

A missing asset points at the pipeline or the deploy configuration: an output directory that changed, an ignore pattern that now matches, or a hashed filename that the HTML references from a different build. This is the failure that most often reaches production, because nothing about the artifact looks wrong.

Whatever the category, re-run the suite once before acting. Preview hosts occasionally serve a 404 for a few seconds while a deployment propagates, and a single retry distinguishes propagation from breakage without adding a sleep to every run.

Pitfalls & Rollback

• Testing the build directory. A local file server has no redirect rules, no headers and no deploy-ignore behaviour. Test the deployed URL.
• Too many URLs. A smoke suite that takes two minutes gets skipped. Keep it under fifteen seconds and let build gates own coverage.
• Asserting on prose. Markers should be structural (site-header, </article>); asserting on wording makes the suite fail on every copy edit.
• Ignoring the trailing-slash variant. Test the exact form your internal links use, since hosts differ in how they normalise.
• Silent continue-on-error. A smoke step that never fails the job is decoration. Fail loudly, then decide what the failure triggers.
• Rollback: the suite is one script and one pipeline step. Removing the step restores the previous pipeline; a failing check that turns out to be wrong should be fixed or deleted the same day, never muted indefinitely.

Conclusion

Smoke tests occupy a narrow and unglamorous slot: a handful of URLs, no browser, ten seconds, run against the thing readers will actually hit. That narrowness is why they stay fast enough to run on every deploy, and running them against the preview and production is what turns "the site is broken" into a specific, environmental difference. Pair them with a rehearsed rollback from Rolling Back a Bad Static Deploy in Under a Minute.

FAQ

Why test the preview URL instead of the build directory?

Because a build directory does not exercise the host. Redirect rules, header files, base URL configuration, deploy-ignore patterns and asset routing only exist once the artifact is deployed, and each of them can produce a site that serves incorrectly from output that built perfectly.

How many pages should a smoke test cover?

One per template plus the homepage, which is usually four to six URLs. Smoke tests answer "is the site fundamentally usable", not "is every page correct" — coverage belongs to the build gates, which are far cheaper per page.

Should a smoke test failure block the deploy?

On a preview, yes: fail the pull request. On production, fail the pipeline and trigger a rollback only for unambiguous failures such as a non-200 status or a missing referenced asset. Threshold-based checks should alert rather than act.

How long should the whole suite take?

Under fifteen seconds. Anything slower gets skipped under deadline pressure, and speed here comes free because the checks are a handful of HTTP requests with no browser involved.

Do I need a headless browser?

Not for smoke tests. Fetching HTML and asserting on markers and asset availability catches the catastrophic failures. Add a browser only for checks that genuinely require rendering, and keep those in a separate, slower job.

Related

• Parent: Rollbacks and Deploy Safety for Static Sites — the layer this sits in.
• Rolling Back a Bad Static Deploy in Under a Minute — what a production failure triggers.
• Preview Environments for Pull Requests — the URL these tests run against.
• Setting Up Deploy Previews on Netlify for Every Pull Request — the same wiring on another host.
• GitHub Actions for Automated SSG Builds — where the step lives.