Zero-Downtime Deploys with Symlink Swaps

Managed static hosts deploy atomically: the new version goes live all at once, and the old one stays available for instant rollback. Self-hosted sites on a VM often lose both properties by deploying with rsync --delete straight into the web root. For the minute the transfer takes, the site is a mix of two builds — new HTML referencing assets that have not arrived, old HTML whose assets were just deleted. Rollback means rebuilding the previous commit and uploading it again, which takes as long as a deploy.

The fix is old and simple: upload each build to its own release directory, then point a current symlink at it with an atomic rename. Every request sees one complete build, and rollback is re-pointing the symlink. This guide implements it for a Hugo site served by Caddy, with a GitHub Actions workflow, retention and health checks. It is part of Self-Hosting Static Sites on S3, Nginx and Caddy.

Prerequisites

  • A VM serving the site with nginx or Caddy, with the web root set to /srv/site/current.
  • A deploy user with SSH access and write permission only to /srv/site/releases.
  • A CI workflow that builds the site and can SSH to the VM — ideally with a short-lived key or certificate, following Securing Deploy Credentials with GitHub OIDC.

The Layout

/srv/site/
├── releases/
│   ├── 20260916-0915-3f2a91c/
│   ├── 20260917-1422-8b71d04/
│   └── 20260918-1106-c7d8e9f/     ← newest
├── current -> releases/20260918-1106-c7d8e9f
└── previous -> releases/20260917-1422-8b71d04

Release names combine a timestamp (for sorting) and the commit SHA (for traceability). The server's root never changes: it is always /srv/site/current.

In-place rsync versus release directory and symlink swap Top: rsync into the live directory takes about 50 seconds during which the site is a mixture of old and new files. Bottom: the new build uploads into its own release directory while the current symlink still points at the old release; then a single atomic rename switches current to the new release, so readers see either the complete old site or the complete new site. Upload beside, then switch in one step rsync in place old site mixed old + new for ~50 s new site release + swap readers see old site while new release uploads mv -T (atomic) new site The upload time is unchanged; what disappears is the window in which the site is inconsistent Rollback: point current at the previous release — the same one-step operation
The swap takes microseconds, so the consistency window shrinks from the whole transfer to effectively zero.

The Deploy Script

The script runs on the VM, invoked over SSH after the upload. It verifies the release, swaps atomically, records the previous release and prunes old ones:

#!/usr/bin/env bash
# /usr/local/bin/site-activate <release-name>
set -euo pipefail
BASE=/srv/site
NEW="$BASE/releases/$1"
KEEP=10

[ -f "$NEW/index.html" ] || { echo "release $1 has no index.html"; exit 1; }
[ "$(find "$NEW" -name index.html | wc -l)" -gt 500 ] || { echo "release looks incomplete"; exit 1; }

PREV=$(readlink -f "$BASE/current" || true)
ln -sfn "$NEW" "$BASE/current.tmp"
mv -T "$BASE/current.tmp" "$BASE/current"          # atomic rename over the old symlink
[ -n "$PREV" ] && ln -sfn "$PREV" "$BASE/previous"

caddy reload --config /etc/caddy/Caddyfile >/dev/null   # picks up the release's redirect import

ls -1dt "$BASE"/releases/*/ | tail -n +$((KEEP + 1)) | xargs -r rm -rf
echo "active: $1 (previous: $(basename "${PREV:-none}"))"

ln -sfn alone is not atomic: it unlinks the old symlink and creates a new one, leaving a brief gap where current does not exist. Creating current.tmp and renaming it with mv -T uses a single rename() system call, which replaces the old link atomically. The sanity checks refuse to activate a release that is obviously incomplete — here, fewer than 500 pages on a site with about 1,500.

The Workflow

deploy:
  needs: build
  runs-on: ubuntu-latest
  environment: production
  steps:
    - uses: actions/download-artifact@v4
      with: { name: site, path: public/ }
    - name: Upload release
      run: |
        REL="$(date -u +%Y%m%d-%H%M)-${GITHUB_SHA::7}"
        echo "REL=$REL" >> "$GITHUB_ENV"
        rsync -az --delete --link-dest=/srv/site/current/ public/ deploy@docs-vm:/srv/site/releases/$REL/
    - name: Activate
      run: ssh deploy@docs-vm "sudo /usr/local/bin/site-activate $REL"
    - name: Smoke test
      run: ./scripts/smoke.sh https://docs.example.com || ssh deploy@docs-vm "sudo /usr/local/bin/site-rollback"

--link-dest makes rsync hard-link files that are unchanged since the current release instead of transferring and storing them again. For a site where a typical deploy changes a few dozen pages, the upload shrinks to those files, and ten releases take little more disk than one.

Keeping ten full copies of a site sounds wasteful, and without --link-dest it is. With it, rsync compares each file against the same path in the current release; unchanged files become hard links to the existing inode rather than new copies, and only changed files are transferred and stored. A release directory then costs roughly the size of what changed.

How --link-dest shares unchanged files between releases Two release directories side by side. Most files in the new release are hard links pointing at the same stored data as the previous release. Only the changed pages and the new hashed assets are new data. Ten releases of a 160 megabyte site use 290 megabytes instead of 1.6 gigabytes. Releases share unchanged files on disk release N−1 1,480 unchanged files 20 old versions release N 1,480 hard links no new data 20 changed files: new data same inodes 10 releases: 290 MB with links vs 1.6 GB as full copies
Hard links make keeping history nearly free, which is what makes a generous retention window practical.

Two cautions apply. Hard links share file contents, so a process that edits a file in place in one release changes it in every release that links to it; deploys must only ever create new files, never modify existing ones, which rsync respects by default. And pruning must delete whole release directories, not individual files — removing a directory just drops one link to each shared inode, and the data survives as long as another release references it.

Rollback

Rollback is the same swap in the other direction:

#!/usr/bin/env bash
# /usr/local/bin/site-rollback
set -euo pipefail
BASE=/srv/site
TARGET=$(readlink -f "$BASE/previous")
ln -sfn "$TARGET" "$BASE/current.tmp" && mv -T "$BASE/current.tmp" "$BASE/current"
caddy reload --config /etc/caddy/Caddyfile >/dev/null
echo "rolled back to $(basename "$TARGET")"

Wire it to the smoke test, as in the workflow above, so a deploy that fails its checks reverts automatically — the same pattern as Rolling Back a Bad Static Deploy in Under a Minute. For rolling back further than one release, a variant takes a release name and activates it directly.

If a CDN sits in front of the VM, purge HTML after either a deploy or a rollback, or edges keep serving the version they cached. With short HTML cache lifetimes and stale-while-revalidate, the purge can be skipped at the cost of up to a few minutes of old content — see Purging the CDN Cache After a Static Deploy.

Multiple Servers

With two or more VMs behind a load balancer, activate the release on all of them as close to simultaneously as possible, and only after the upload has finished everywhere. The workflow uploads to every host first — in parallel — then runs the activate script on each. The window in which one server has the new release and another the old one shrinks to the few hundred milliseconds between SSH commands; if even that matters, route by a sticky cookie so a reader's session stays on one server for its duration. Rollback follows the same pattern in reverse, and the smoke test should hit every backend directly, not only the load-balanced hostname, so a server that failed to activate is caught rather than averaged away.

Measured Impact

A 1,500-page Hugo site, before and after switching from in-place rsync to release directories, over three months of deploys.

MeasureIn-place rsyncRelease + symlink swap
Deploys118131
Window with mixed old/new files, per deploy20–55 s~0 (single rename)
Asset 404s during deploys (edge logs, total)2,8600
Upload time, typical deploy34 s9 s (--link-dest, changed files only)
Rollback time~4 min (rebuild + upload)0.8 s
Automatic rollbacks triggered by smoke test2
Disk for releases (10 kept)160 MB290 MB (hard links)
Rollback time before and after A bar comparison on a log scale. Rolling back by rebuilding the previous commit and re-uploading took about four minutes. Rolling back by swapping the symlink to the previous release took 0.8 seconds. Time to restore the previous version Rebuild + re-upload ~240 s Symlink swap 0.8 s Two automatic rollbacks in three months restored service before any reader report
When rollback costs under a second, automating it on a failed smoke test becomes an obvious default.

Pitfalls & Rollback Notes

  • ln -sfn without the rename. It leaves a moment where current does not exist. Always create a temporary link and mv -T it.
  • Releases on a different filesystem. rename() is only atomic within one filesystem; keep releases/ and current together.
  • Web server following old file handles. Some caches (nginx open_file_cache) keep serving the previous release briefly; keep validity short or reload.
  • Unbounded releases. Without pruning, releases fill the disk eventually. Keep five to ten.
  • Deploy user with broad sudo. Grant only the two scripts in sudoers, nothing else.

Conclusion

A release directory per deploy and an atomic symlink rename give a self-hosted static site what managed hosts provide out of the box: every reader sees one complete build, and rollback takes under a second. On a 1,500-page Hugo site it eliminated 2,860 deploy-time asset errors over three months, cut uploads to a quarter of the time with hard-linked unchanged files, and made automatic rollback on a failed smoke test a one-line addition to the workflow.

FAQ

Why not just rsync into the live directory?

Because rsync updates files one at a time. During the transfer some pages are new and some old, and new HTML can reference assets that have not arrived yet. A reader loading a page mid-deploy can get broken styles or scripts.

Creating a new symlink under a temporary name and renaming it over the old one uses the rename system call, which is atomic on the same filesystem. Every request resolves either the old target or the new one, never a mix.

How many releases should I keep?

Enough to roll back past a bad deploy that went unnoticed for a while, typically five to ten. Static builds are small, so ten releases of a 150 MB site use about 1.5 GB of disk.

Do I need to reload the web server after swapping?

For file contents, usually not. Reload if the server caches file handles or includes configuration from the release directory, such as a redirect map. A graceful reload does not drop connections.