Tracking Deploy Markers in Analytics

A dashboard shows LCP at p75 rising from 1.6 s to 2.1 s on Tuesday afternoon. Was it a deploy? A traffic spike from a region with slower networks? A third-party script that changed on its own? Without knowing exactly when each version of the site went live, the investigation starts with a trawl through CI history and a guess. Static sites deploy often — several times a day for an active documentation team — which makes the guessing harder and the answer more valuable.

This guide makes every deploy visible where metrics are read: the build writes its version into every page, RUM, analytics and error events carry that version, and a CI step posts a deploy marker to each dashboard. It is part of Monitoring Static Sites in Production, and makes the dashboard in Building a Core Web Vitals Dashboard from RUM Data much more useful.

Prerequisites

• A RUM pipeline (the web-vitals library sending beacons to your own endpoint or an analytics tool), an analytics tool, and optionally an error tracker.
• CI that deploys the site and has the commit SHA available.
• API tokens for any tools that accept annotations or releases.

Step 1: Stamp the Version Into Every Page

The build writes the commit SHA, the build time and the pull request number into the root element:

---
const version = import.meta.env.PUBLIC_BUILD_SHA?.slice(0, 7) ?? 'dev';
const builtAt = import.meta.env.PUBLIC_BUILD_TIME ?? '';
---
<html lang="en" data-version={version} data-built={builtAt}>

- run: npm run build
  env:
    PUBLIC_BUILD_SHA: ${{ github.sha }}
    PUBLIC_BUILD_TIME: ${{ github.event.head_commit.timestamp }}

Only the root element carries the version, not every page body, so unchanged pages in a changed-files deploy remain byte-identical except for one attribute. On sites that deploy only changed files, as in Deploying Only Changed Files, put the version in a tiny /version.json instead, fetched once per session and cached, so a version bump does not mark every page as changed.

Step 2: Tag Every Event With the Version

Each script that reports data reads the version once and adds it to what it sends:

const version = document.documentElement.dataset.version;
onLCP((m) => beacon({ ...toPayload(m), version }));
onINP((m) => beacon({ ...toPayload(m), version }));
onCLS((m) => beacon({ ...toPayload(m), version }));
plausible('pageview', { props: { version } });
Sentry.init({ dsn: DSN, release: `docs@${version}` });

Readers who keep a tab open across a deploy continue to report the version their page was built with, which is correct: their experience comes from that version.

Step 3: Post Deploy Markers From CI

After a successful production deploy, one CI step records the deploy everywhere. Each tool has its own API; the pattern is the same:

- name: Record deploy markers
  if: success()
  env:
    SHA: ${{ github.sha }}
    MSG: ${{ github.event.head_commit.message }}
  run: |
    short=${SHA:0:7}
    # RUM store (own endpoint)
    curl -fsS -X POST https://rum.example.com/api/deploys -H "Authorization: Bearer $RUM_TOKEN" \
      -d "{\"version\":\"$short\",\"at\":\"$(date -u +%FT%TZ)\",\"message\":$(jq -Rs . <<<"$MSG")}"
    # Grafana annotation
    curl -fsS -X POST "$GRAFANA_URL/api/annotations" -H "Authorization: Bearer $GRAFANA_TOKEN" -H 'Content-Type: application/json' \
      -d "{\"tags\":[\"deploy\",\"docs\"],\"text\":\"docs $short: $(head -1 <<<"$MSG" | sed 's/\"/\\\\\"/g')\"}"
    # Sentry release
    npx @sentry/cli releases new "docs@$short" && npx @sentry/cli releases finalize "docs@$short" \
      && npx @sentry/cli releases deploys "docs@$short" new -e production

A failure here must not fail the deploy — the site is already live — so the step logs errors and continues. The same step can post a message to the team channel with the version, the pull requests included and a link to the dashboards.

Step 4: Read Metrics by Version

With version tags, dashboards can compare versions directly instead of reading lines around a marker. The RUM dashboard gained a table of p75 LCP, INP and CLS per version for the last ten deploys, with sample counts. On the Tuesday in question, the table showed LCP at p75 of 1.61 s for version 4c1e9a2 and 2.08 s for 9b7d013, deployed at 14:12. The diff for 9b7d013 included a new hero image component on the landing template without fetchpriority — found in five minutes instead of an afternoon.

Sample size matters: a version live for twenty minutes may have too few page views for a stable p75. The table greys out versions below a threshold (500 views per template) and merges consecutive versions with no changes to the template in question.

Step 5: Alert on Version-Level Regressions

Once metrics are split by version, alerting can compare a new version with the previous one rather than with a fixed threshold. A scheduled job an hour after each deploy compares p75 LCP, INP and CLS and the JavaScript error rate for the new version against the previous version on each template, and posts to the team channel when any worsens by more than 10% with enough samples. It fired four times in six months; three were real regressions, one was a traffic-mix change from a marketing campaign in a region with slower networks — which the alert message helped rule out by including the country mix.

Markers for Changes That Are Not Deploys

Not every change to what readers experience is a deploy of the site. A tag manager publish can add a script to every page without touching the repository; a CDN configuration change can alter caching; a third-party vendor can ship a new version of its widget; a DNS or certificate change can affect connection times. Each of these deserves a marker too. The team added three sources: the tag manager's publish webhook posts a marker tagged gtm, CDN configuration changes made through Terraform post one from the Terraform pipeline, and a weekly job records the version hash of each third-party script the site loads, posting a marker when one changes. The first vendor-change marker explained a two-day INP regression that no site deploy could account for: a chat vendor had shipped a heavier widget, which the facade approach in Facades for Chat Widgets and Maps then removed from the page load entirely.

Markers also make postmortems faster: the timeline of a regression can be read straight off the dashboard, with each deploy, tag publish and vendor change already labelled.

Measured Impact

Pitfalls & Rollback

• Version in every page body. Makes every file "changed" on every deploy; use one attribute or a version file.
• Failing the deploy on a marker API error. The site is already live; log and continue.
• Comparing versions with too few samples. p75 from a few hundred views is noisy; set thresholds.
• Forgetting previews. Tag preview traffic with its own environment so it never mixes with production.
• Rollback: markers and tags are additive; removing the CI step and the attribute stops them without affecting the site.

Conclusion

Deploy markers turn "something changed on Tuesday" into "version 9b7d013 changed the landing page". Stamp the version into the page, tag every RUM, analytics and error event with it, post a marker to every dashboard after each deploy, and compare versions automatically an hour later. On this site it cut the time to explain a metric change from hours to minutes, and moved the discovery of regressions from readers to the team.

FAQ

What is a deploy marker?

A timestamped annotation on dashboards showing when a new version of the site went live, usually with the commit SHA, pull request and a short description. It makes it obvious whether a change in a metric lines up with a deploy.

How does a static page know which version it is?

The build writes the commit SHA and build time into the HTML, for example as a meta tag or a data attribute on the html element. Scripts for RUM, analytics and error tracking read it and attach it to every event they send.

Where should deploy markers be sent?

To every tool where someone looks at trends: the real-user monitoring dashboard, product analytics, the error tracker and the uptime or synthetic monitoring tool. Most offer an API for annotations or releases; a single CI step can call all of them.

What if several deploys happen in a short time?

Tag data with the version rather than relying on timestamps alone. With every event carrying the build SHA, dashboards can split metrics by version, which remains precise even when deploys are minutes apart or rolled out gradually.

Related

• Parent: Monitoring Static Sites in Production — watching a live static site.
• Real-User Error Tracking on Static Sites — errors tagged with the same version.
• Measuring INP on Static Sites with Real User Monitoring — the RUM beacons.
• Tracking Page Weight Over Time — the lab-side history per commit.
• Canary Releases for Static Sites — version tags during gradual rollouts.