All projects

replaceable-vps

A server kept in code, so swapping in a new machine is one routine command and the DNS follows it.

Role
Design and build
Client
Personal project
Year
2026
Stack
Infrastructure
OpenTofu, Hetzner Cloud, Cloudflare DNS, Cloudflare R2
Configuration
Ansible, Ubuntu 24.04, Docker, ufw, fail2ban
Scripts
Bash, Python, ntfy
Checks
GitHub Actions, ShellCheck

A server that can be rebuilt from its files

A server set up by hand carries its own setup: the packages someone installed and the config files they edited are on it and nowhere else. The DNS records that send visitors to it hold its address as someone typed it. That’s fine until the machine has to change. It fails, or has to move, or a better one is worth moving to, and all of it gets rebuilt from memory. A missed setting or a record still pointing at the old address shows up as a broken site.

replaceable-vps does that work ahead of time. It describes one server in files, so a new machine can be built from them at any time and replacing the old one is routine.

The whole server is these files:

infra/
  iac/                 OpenTofu: the server, its firewall, its DNS records
    server.tf  dns.tf  variables.tf  outputs.tf  backend.tf  versions.tf
  ansible/             Ansible: what goes on the machine
    site.yml  bootstrap.yml
    roles/  base  ssh  firewall  fail2ban  auto-updates  docker
scripts/
  preflight.sh  wait-for-capacity.sh
docs/
  replace-a-server.md

OpenTofu, which creates cloud resources from a description, holds the server on Hetzner, its firewall, and its DNS records on Cloudflare. Ansible, which sets up a machine from a list of tasks, holds what goes on it: SSH that only takes keys, a second firewall, fail2ban to block repeated failed logins, security updates that install themselves, and Docker. Whatever runs on the server is deployed separately, so the machine can be replaced without touching the apps.

Replacing it is one command

The DNS records follow the server. Each one takes its value from the server itself, hcloud_server.prod.ipv4_address, so when the server is replaced, every record picks up the new address in the same run.

Replacing it is one command. tofu apply -replace=hcloud_server.prod destroys the machine and builds a new one from a snapshot, a saved image of its disk. The new one can be a different server type or in a different location. A runbook in the repo puts checks on both sides of the command.

before check the credentials and that the type's on sale snapshot the machine and dump its data elsewhere note what it's serving the apply switch protections off then read the plan tofu apply -replace new server from the snapshot DNS records follow after check against the notes protections back on defaults moved first if it's wrong
The highlighted step is the command that swaps the machine. If the new one fails its check, applying again from the snapshot brings the old one back, and the dump covers anything written after the snapshot.

Data has a way back. Snapshots and the provider’s daily backups, on by default, sit in the same account as the server. So before a replacement, anything holding data is also dumped somewhere else. If the new machine fails its checks, applying again from the snapshot brings the old one back, and the dump covers what was written since.

Hard to delete by mistake. The provider’s delete protection is on, and OpenTofu’s prevent_destroy also blocks tofu destroy. A replacement turns them off with a file that’s deleted afterwards and a commit of its own. Every plan, OpenTofu’s preview of a change, warns while the provider’s protection is off.

The machine doesn’t drift from the files. On a fresh server, a second run of the Ansible setup changed nothing, and a later run puts back any hand edit to the files it manages. After a replacement, the defaults in the OpenTofu files move to the new server type too, or the next plan would offer to change the machine back.

Before anything changes. Server types sell out, and old ones get retired. A script asks the provider which of your chosen locations can sell the type you want, and sends a push notification when the answer changes to yes. Another tests the Hetzner and Cloudflare tokens, so a revoked one turns up before the replacement starts.

Checked on every change. GitHub Actions validates the OpenTofu files, syntax-checks both playbooks and runs ShellCheck on both scripts, on every push to main and every pull request.