Static Site Generators in Production

Running WebPageTest Scripts Against Preview Deploys

Lighthouse CI answers one question well: how does this page load, cold, on an emulated mid-range phone? Many regressions on static sites do not show up in that question. A search dialog that freezes when opened, a client-side route change that takes a second, a repeat visit that re-downloads fonts because a cache header broke, a page that is fast in Virginia and slow in Singapore — each needs a multi-step journey, a real browser on a real network, or a specific location.

WebPageTest runs scripted journeys on real browsers in dozens of locations and produces filmstrips and waterfalls that make a regression easy to explain. This guide scripts three journeys for a documentation site, runs them against each preview deploy, and reports what they caught that Lighthouse CI did not. It complements Performance Budgets and Lighthouse CI.

Prerequisites

  • A preview deploy per pull request — see Preview Environments for Pull Requests.
  • A WebPageTest API key (hosted plan with API access) or a private instance.
  • The webpagetest CLI: npm i -D webpagetest.
  • Two or three journeys that matter to your readers, written down in plain language first.

Step 1: Choose Journeys, Not Pages

Pick journeys that exercise what Lighthouse cannot. For this docs site:

  1. Search to answer: load the homepage, open search, type a query, click the first result. Measures the search dialog's load and the navigation after it.
  2. Browse three pages: land on a guide, click to the next guide, click to a reference page. Measures repeat-view caching and navigation.
  3. Far-away reader: load a long guide from a location far from the origin (Singapore and São Paulo here). Measures CDN reach and asset cacheability.

Each journey should map to a real behaviour visible in your analytics. Search journeys are worth scripting only if a meaningful share of sessions use search.

Step 2: Write the Scripts

WebPageTest scripts are tab-separated commands. logData 0 turns off measurement for setup steps; logData 1 turns it back on for the step you care about.

// search-journey.wpt
logData 0
navigate    %PREVIEW%/
logData 1
execAndWait document.querySelector('#search-open').click()
execAndWait (async()=>{const i=document.querySelector('#search input');i.value='cache headers';i.dispatchEvent(new Event('input',{bubbles:true}));await new Promise(r=>setTimeout(r,800));})()
execAndWait document.querySelector('.pagefind-ui__result-link').click()

// browse-journey.wpt
navigate    %PREVIEW%/guides/deploying-hugo/
navigate    %PREVIEW%/guides/caching-hugo-builds/
navigate    %PREVIEW%/reference/configuration/

Each navigate and execAndWait becomes a measured step with its own filmstrip, waterfall and metrics.

The search journey as measured steps Four steps in sequence. Step 0, load the homepage, is setup and not logged. Step 1 opens search and measures the dialog's script load. Step 2 types a query and measures time until results render. Step 3 clicks the first result and measures the navigation to the answer page. Each measured step has its own filmstrip and waterfall. One journey, three measured steps 0 · load home logData 0 (setup) 1 · open search script + CSS load 2 · type query fragments → results 3 · click result navigation, warm cache each measured step: filmstrip · waterfall · LCP · TBT · CLS · requests Lighthouse sees only a cold load of step 0; every regression below happened in steps 1–3
Splitting a journey into logged steps turns "search feels slow" into a specific step with its own waterfall.

Step 3: Run From CI Against the Preview

Substitute the preview URL, submit each script, wait for results and write a summary:

# .github/workflows/preview.yml (excerpt)
wpt:
  needs: deploy-preview
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - run: npm ci
    - name: Run WebPageTest journeys
      env:
        WPT_API_KEY: ${{ secrets.WPT_API_KEY }}
        PREVIEW: ${{ needs.deploy-preview.outputs.url }}
      run: |
        for s in search browse; do
          sed "s#%PREVIEW%#$PREVIEW#g" wpt/$s-journey.wpt > /tmp/$s.wpt
          npx webpagetest test /tmp/$s.wpt --key "$WPT_API_KEY" \
            --location ec2-us-east-1:Chrome.4G --runs 3 --first --poll 10 \
            --reporter json > wpt-$s.json
        done
        npx webpagetest test "$PREVIEW/guides/deploying-hugo/" --key "$WPT_API_KEY" \
          --location ec2-ap-southeast-1:Chrome.4G --runs 3 --poll 10 --reporter json > wpt-far.json
        node scripts/wpt-summary.mjs wpt-*.json >> "$GITHUB_STEP_SUMMARY"

The summary script reads the median run of each step and compares it against thresholds. Start with thresholds as warnings — real browsers on real networks vary more than Lighthouse's simulation — and promote the stable ones to failures after a few weeks, as described in Reducing Lighthouse Score Variance in CI.

Step 4: Read the Filmstrip

When a step regresses, the filmstrip is the fastest explanation. Link it from the step summary so reviewers can see, frame by frame, what readers see. The waterfall then shows why: a request that used to be cached, a script that moved earlier, a font that now blocks text.

Filmstrip of the search step before and after a regression Two rows of frames at 100 millisecond intervals after clicking search. On main, the dialog appears by 100 milliseconds and the input is ready by 200. On the pull request, the dialog frame stays empty until 600 milliseconds and the input is ready at 740, because a large bundle now loads first. Search step filmstrip, 100 ms per frame main ready this PR ready 0 100 200 400 600 ms Six empty frames made the regression obvious in review before anyone opened the waterfall
A filmstrip turns a number into something a designer or product owner can see and agree is a problem.

The waterfall for the regressed step showed a 190 KB chunk requested on click, where main had loaded a 22 KB one; the pull request had imported the search UI's full component library into the dialog. Moving the import back behind a dynamic import() fixed it. Without the filmstrip, the discussion would have been about whether 740 ms "really" mattered; with it, the answer was visible.

What the Journeys Caught

Over four months on the docs site, the scripted journeys flagged six regressions that Lighthouse CI passed:

RegressionJourney stepMeasured changeRoot cause
Search dialog opened slowlysearch · step 1180 → 740 msUI bundle imported eagerly into the dialog chunk
Results rendered latesearch · step 2310 → 920 msIndex entry file cached for a year, stale fragments refetched
Second page re-downloaded fontsbrowse · step 20 → 96 KB fontsCache-Control missing on /fonts/ after a host migration
Reference page long taskbrowse · step 3TBT 40 → 380 msTable-sorting script running on load
Far location LCPfar · load2.1 → 3.6 sNew image domain not behind the CDN
Layout shift on navigationbrowse · step 2CLS 0.01 → 0.14Sticky header height changing after font swap
Where the six regressions occurred A timeline of a reader journey with six markers. Two regressions occurred in the search dialog, three during subsequent navigations with a warm cache, and one only from a distant location. None occurred in a cold single-page load from a nearby location, which is what Lighthouse CI tests. Six regressions, none in a nearby cold page load Cold load, nearby what Lighthouse tests 0 Interaction search dialog 2 Next pages warm cache, navigation 3 Far location Singapore, São Paulo 1 Four months of pull requests on a 900-page docs site; all six passed Lighthouse CI
Journeys, warm caches and distance are exactly the three dimensions a single cold Lighthouse run holds constant.

Three of the six were caching problems, which is the category Lighthouse is structurally blind to: it always starts cold. The caching fixes themselves are covered in Setting Cache-Control Headers on Cloudflare Pages and Cache Busting with Content-Hashed Filenames.

Measured Impact

MeasureValue
Journeys per pull request3 (search, browse, far location)
Median job duration3 min 40 s, parallel with Lighthouse CI
Regressions caught that Lighthouse CI passed6 in four months
False alarms (threshold crossed, no real change)4, all in the far-location test
Tests used per monthabout 540 (60 pull requests × 3 journeys × 3 runs)

The far-location test was the noisiest, because real network paths across oceans vary. Its threshold stayed a warning; the other two journeys were promoted to failures after five weeks.

Choosing Locations and Devices

The location and device list decides what the journeys can find, so choose it from your audience rather than from the default. Start with the analytics breakdown of sessions by country and by device category. On this docs site, 38% of sessions came from North America, 31% from Europe, 17% from South and Southeast Asia and 6% from South America; mobile was 29% overall but 52% in Southeast Asia. That produced three test configurations: US East on emulated 4G for the main journeys, Frankfurt as a second nearby check, and Singapore on a real mid-range Android device for the far-location test.

Real devices matter most for interaction-heavy steps. A Moto G-class phone in WebPageTest's device lab took 3.1 times longer than a desktop agent to run the search dialog's script, which is roughly what CPU throttling in Lighthouse predicts, but the real device also exposed a touch-event handler that emulation did not trigger. Use real devices for the one or two journeys where input handling matters, and cheaper desktop agents with throttling for the rest. Revisit the list twice a year; audience mixes shift, and a new market can make a previously irrelevant location the most important one to test.

Pitfalls & Rollback

  • Scripting every page. Journeys are for behaviour Lighthouse cannot see. Three good journeys beat twenty page loads.
  • Brittle selectors. Scripts that click .css-1x2y3z break on every style change. Add stable data-testid hooks to the elements your journeys use.
  • Failing on the first run. Real devices vary; use the median of three and warnings until the spread is known.
  • Exhausting the test allowance. Skip WebPageTest on documentation-only pull requests, or run it only when templates, scripts or headers change.
  • Rollback: the journeys are a separate job. Removing it from required checks makes it advisory; deleting it removes it.

Conclusion

WebPageTest scripts fill the gaps a cold, nearby Lighthouse run leaves: interactions, warm-cache navigations and distant readers. Three journeys against each preview deploy caught six regressions in four months that Lighthouse CI had passed — half of them caching problems no cold load can reveal. Keep the journeys few and tied to real reader behaviour, start with warnings, and let the filmstrips do the explaining in review.

FAQ

When is WebPageTest worth adding next to Lighthouse CI?

When regressions hide in journeys rather than cold single-page loads, when your readers are far from your CDN or on specific devices, or when you need filmstrips and waterfalls to explain a regression in review. For a simple content site, Lighthouse CI alone is usually enough.

Does WebPageTest cost money?

The hosted service has a free tier with a monthly test allowance and paid plans with API access for CI. You can also run a private WebPageTest instance with your own agents, which suits teams that test on every pull request.

How long does a scripted test take?

A three-step script with three runs and a repeat view took about three to four minutes per location on the hosted service, depending on queue length. Run it after the preview deploy and in parallel with Lighthouse CI.

Can WebPageTest fail a pull request?

Yes. The CLI or API returns metrics as JSON, and a small script can compare them against thresholds and exit non-zero. Treat journey metrics as warnings at first, because real-device results vary more than emulated ones.