Static Site Generators in Production

Fixing Soft 404s on Static Hosts

A static site cannot have a server bug that returns the wrong status code — except that the host is a server, and its configuration decides what status every missing path gets. Soft 404s on static sites almost always come from that configuration: a catch-all rewrite to index.html copied from a single-page-app tutorial, an error page configured to return 200, or a CDN rule that "fixes" 404s by serving the home page. The result is that every mistyped, deleted or guessed URL returns a 200 with some page attached. Search engines notice and flag them as soft 404s. Monitoring that counts 404 responses sees nothing at all.

This guide finds and fixes soft 404s on a documentation site that over three years was hosted on Netlify, then S3 with CloudFront, then Cloudflare, and accumulated a different soft-404 cause on each. It belongs to Technical SEO for Static Sites.

Step 1: Test What a Missing Page Returns

One request tells you whether the host is producing soft 404s:

curl -s -o /dev/null -w '%{http_code} %{size_download}\n' https://docs.example.com/this-page-does-not-exist-7f3a/
# 200 48211

A 200 with a large body means a catch-all. A correct setup returns 404 with the custom error page's size. Search Console's page indexing report had 1,940 URLs under "Soft 404".

Step 2: Group the Reported URLs

GroupURLsExampleCause
Old URLs from before a restructure1,210/v1/guide/setupCatch-all rewrite served home page
Mistyped inbound links380/guides/instal/Same catch-all
Empty category pages190/category/legacy/Generator produced empty lists
Search result pages160/search/?q=xyzIndexed via external links, empty results
Number of soft 404 URLs per cause Vertical bars. Old URLs served by a catch-all rewrite 1,210. Mistyped inbound links served by the same rewrite 380. Empty category pages 190. Search result pages with no results 160. Soft 404s by cause 1210 Old URLs (catch-all) 380 Typos (catch-all) 190 Empty categories 160 Empty search pages
Search Console soft 404 report, 1,940 URLs grouped by pattern; 82 percent came from one host rule.

Cause 1: A Catch-All Rewrite

On Cloudflare, the project's _redirects file ended with:

/*  /index.html  200

That line came from a period when part of the docs was a single-page app. It turned every unknown path into the home page with status 200. Removing it, and making sure 404.html exists at the root of the output, makes Cloudflare serve the custom page with status 404. The equivalent fixes on other hosts:

  • Netlify: remove /* /index.html 200 from _redirects or netlify.toml; Netlify serves 404.html with 404 automatically.
  • S3 + CloudFront: in the distribution's custom error responses, map 403 and 404 to /404.html with response code 404, not 200. SPA guides often set 200 here.
  • Nginx: try_files $uri $uri/ =404; with error_page 404 /404.html; — not try_files $uri /index.html;.
Configuration that causes soft 404s and the correct setting per host A matrix of four hosts. On Cloudflare the cause is a catch-all line in the redirects file and the fix is to remove it and ship 404.html. On Netlify the cause is the same catch-all rule and the fix is the same. On S3 with CloudFront the cause is a custom error response with status 200 and the fix is to use status 404. On Nginx the cause is try_files falling back to index.html and the fix is to end with equals 404 and set error_page. The soft-404 setting on each host Host Soft-404 cause Correct setting Cloudflare /* /index.html 200 404.html at root Netlify /* /index.html 200 404.html at root S3 + CloudFront error response code 200 error response code 404 Nginx try_files … /index.html try_files … =404
All four hosts serve a custom 404 page with the correct status once the SPA-style fallback is removed.

Cause 2: Old URLs Without Redirects

Once the catch-all was gone, the 1,210 old URLs correctly returned 404 — which removed the soft-404 problem but turned them into real 404s. Many had inbound links. Server logs and Search Console's links report showed which ones still received traffic or links; 340 of them did. Those got 301 redirects to their current equivalents, generated from the old site's URL list as in Keeping Redirects Working After an SSG Migration. The rest were left as 404s.

Mistyped inbound links (/guides/instal/) were handled the same way where they had meaningful traffic: eleven typo URLs from popular external posts got redirects. The 404 page itself gained a search box prefilled from the URL's last segment, which turned many remaining 404s into successful visits.

Cause 3: Empty List Pages

The generator produced category pages for every category in the taxonomy, including 14 categories whose posts had all been moved or deleted. Each rendered as a heading and "No posts found" — a textbook soft 404, multiplied by pagination and feed variants to 190 URLs. The fix was to not generate empty term pages. In Hugo, a check in the term template is not enough (the page still exists), so the categories were removed from the taxonomy data and their URLs redirected to the parent section. In Astro and Eleventy the equivalent is filtering empty groups out of getStaticPaths or the pagination data.

Cause 4: Search Result Pages

/search/?q=… pages are rendered client-side from a static shell. External sites had linked to some search URLs, and search engines crawled them; with an empty or odd query they showed "No results", and got flagged. Two fixes: <meta name="robots" content="noindex"> on the search page, and Disallow: /search/ in robots.txt so crawlers stop requesting it. Because the page is a static shell, the meta tag is in the HTML and does not depend on the query.

Step 3: Monitor Real 404s

With soft 404s gone, real 404s are visible again — which is the point. Edge logs now feed a weekly report of the top 404 paths and their referrers, as in Logging 404s at the Edge, and a CI check requests a random nonexistent path on every preview deploy and fails unless it returns 404:

code=$(curl -s -o /dev/null -w '%{http_code}' "$PREVIEW_URL/ci-missing-$(date +%s)/")
test "$code" = "404" || { echo "Missing page returned $code"; exit 1; }

That one check would have caught the original catch-all on the day it was added.

Checks that prevent soft 404s from returning Four steps. Every preview deploy requests a random nonexistent path and fails unless the status is 404. The custom 404 page offers search prefilled from the URL. Edge logs report the top 404 paths and referrers weekly. Paths with traffic or links get redirects; the rest stay 404. Keeping 404s honest CI missing-path check must return 404 Helpful 404 page search prefilled Weekly 404 report from edge logs Redirect what matters traffic or links
The CI check takes under a second; the weekly report takes ten minutes to review.

Making the 404 Page Useful

A correct 404 status is half the job; the other half is what the reader sees. The site's 404 page had been a single sentence. It now does three things, all static: it shows a search box prefilled with words from the requested path (/guides/instal/ becomes "instal", which Pagefind's prefix matching still finds), it lists the site's five most visited sections, and it includes a short "report a broken link" link that opens a prefilled issue with the requested URL and the referrer. The page is the same 404.html on every host, built by the generator like any other page, so it gets the site's layout, theme and navigation.

Analytics events on the 404 page showed what readers did next: 41% used the search, 21% clicked a section link, and 38% left. Before the change, almost everyone left. The broken-link reports, about four a week, turned out to be the best source of redirect candidates — each one names a real external page linking to a wrong URL — and are now reviewed alongside the weekly 404 report.

Measured Impact

MeasureBefore6 weeks afterTool
Soft 404s in Search Console1,94012Page indexing report
Missing path status200 (home page)404curl
Real 404s visible in monitoring / week0 (hidden)2,300 → 410 after redirectsEdge logs
Visitors leaving from the 404 page—38% (others searched or clicked)Analytics
Indexed pages2,8802,910Search Console

Pitfalls & Rollback

  • SPA fallback rules on a multi-page site. They turn every missing URL into a soft 404.
  • Error responses configured with status 200. Use 404.
  • Removing the catch-all without redirects. Old URLs with links need 301s.
  • Generating empty list pages. Filter them out of generation.
  • Rollback: each host rule is one line; reverting it restores the old behaviour, and the CI check will fail the next preview.

Conclusion

Soft 404s on static sites are a host configuration problem far more often than a content one. Removing SPA-style catch-all rewrites and 200-status error responses, redirecting the old URLs that still have traffic, not generating empty list pages and keeping search result pages out of the index took this site from 1,940 soft 404s to 12 — and made real 404s visible to monitoring for the first time in three years.

FAQ

What is a soft 404?

A page that returns HTTP status 200 but looks like an error or has no meaningful content, such as a not found message, an empty search result or an empty category page. Search engines treat it as a 404 for indexing but waste crawl time on it, and it can hide real broken links from monitoring that only counts 404 status codes.

Why do static sites produce soft 404s?

Mostly through host configuration. A rewrite that sends every unknown path to index.html, copied from single-page application setups, makes every mistyped or deleted URL return the home page with status 200. Custom error pages configured with status 200 do the same. Generators also produce empty list pages, such as a category with no posts, that look like errors.

How do I find soft 404s?

Search Console lists them under Soft 404 in the page indexing report. To find them before Google does, request a deliberately nonexistent URL on the site and check the status code, and crawl the site looking for pages with status 200 whose title or body matches the 404 page or that have almost no content.

Should deleted pages return 404 or 410?

Either works for search engines. 410 states the removal is permanent and is sometimes dropped from the index slightly faster. Use 301 redirects instead whenever a page has a reasonable replacement, so readers and links land somewhere useful.

Filed underFixes