Getting a homelab ready to run for three months without me
No sleep, a watchdog, key-only SSH, a wiped HDD turned into storage and backups, a second way in, and an outside alarm for when the box goes dark.
On this page
- The failure points
- Never sleep, and reboot if frozen
- SSH: keys only, verified before switching
- A second way in
- Wiping the HDD: storage and backups
- Backups that are tested, not assumed
- Docker and the demo sites
- Trying network storage, then turning it off
- An alarm that works when the server doesn't
- Still on the list
I'll be out of the country for three months with no physical access to the server, and I had about a month to prepare. After the audit, the question became concrete: what could take this machine down or lock me out while I'm away, and how would I even know?
I started a new Claude Code session on the server and asked it to find every failure point and fix what it could.
The failure points
The first survey sorted the risks into "critical before you leave" and the rest:
- The machine could fall asleep. It's a laptop, and closing the lid suspends it by default. A suspended server is a dead server.
- Power cuts. The battery is at about 74% of its design capacity after 1,490 cycles, and there's no UPS. After an outage, the machine only comes back if the BIOS powers it on when AC returns.
- One way in. One SSH key, and one other device on Tailscale. Lose that laptop and I'm locked out for three months.
- No backups. The only copy of everything, including
.envfiles and tunnel credentials, was on one SSD. - SSH accepted passwords on every interface.
Never sleep, and reboot if frozen
The easy wins went in first:
- a
logind.conf.ddrop-in so the lid switch, idle and the power key are ignored; sleep.target,suspend.targetandhibernate.targetmasked, so nothing can suspend it by accident;- a hardware watchdog that reboots the machine if it freezes for 30 seconds;
- a full system upgrade (including Tailscale), the journal capped at 500 MB and 3 months, and 2 GB of Docker build cache cleared;
/bootpermissions tightened, so the random seed is no longer world-readable.
SSH: keys only, verified before switching
My past logins had used a password, so turning passwords off blind could have locked me out. The order mattered:
- I tested a key-only login from my laptop:
ssh -o PasswordAuthentication=no .... - Only after that worked did
PasswordAuthentication nogo in, along with no root login,AllowUsers, 3 attempts maximum and a short login grace time. - fail2ban watches sshd, with my own networks on the ignore list so I can't ban myself.
A second way in
A second device on Tailscale and a second SSH key, on different hardware:
- I added my phone to the tailnet.
- I generated an ed25519 key in Termux. My first try failed because I typed
ssh keygeninstead ofssh-keygen, and ssh tried to connect to a host called "keygen". - Claude added the public key, I logged in from the phone on mobile data with Wi-Fi off, and the SSH log confirmed it was the phone's key that got in.
- I turned off Tailscale key expiry for the server, which otherwise drops off the tailnet after about 180 days.
Wiping the HDD: storage and backups
The 1 TB HDD only held old Windows data, so I asked for it to be wiped. Before touching it, Claude checked the drive's serial number matched the HDD, not the NVMe. That's a cheap check against the most expensive possible mistake.
It's now two ext4 partitions, both mounted with nofail so a dead drive can't stop the boot:
| Mount | Size | For |
|---|---|---|
/mnt/storage |
462 GB | A network storage project |
/mnt/backup |
454 GB | Backups, root-only |
Backups that are tested, not assumed
The backups use restic: a nightly systemd timer backs up /home and /etc to /mnt/backup, keeps 7 daily, 4 weekly and 6 monthly snapshots, and runs restic check afterwards.
The first run used 35 GB. The same job then restored a file from the snapshot and compared it with the original, and they matched. A backup you've never restored from is only a hope.
The weak spot is clear: backups on the same machine as the data don't survive theft, fire or a power surge. An offsite copy is still on the list.
Docker and the demo sites
- The two app ports that were published on every interface are now bound to
127.0.0.1. The tunnel and Caddy reach them locally. - Container logs rotate at 10 MB × 3 files.
- Fresh Caddy and cloudflared images, all containers recreated, and all three sites returned 200 afterwards.
Trying network storage, then turning it off
I tried a Samba share for /mnt/storage. It was set up as tightly as I could think of:
- SMB3 with encryption required;
- no guest access;
- bound to the Tailscale address only, with an nftables rule dropping SMB from anything else;
- a dedicated Samba password.
Windows rejected the first password (the fix was a simpler password and the username written as server\user), but after that it worked. Then I asked for it to be turned off again. I want to understand the trade-offs before leaving a file share running for three months. It's disabled with the config kept, so turning it back on is one command.
An alarm that works when the server doesn't
The server already sends health reports to Discord, but if it dies, it goes quiet, and silence looks exactly like "everything's fine". So we added a dead-man's switch with healthchecks.io:
- Heartbeat: every 5 minutes, a script checks that all mounts are present, all containers are running, the sites respond on localhost, there are no failed units, the disk is under 90% and Tailscale is up. If all of that holds it pings healthchecks.io. If not, it pings the
/failURL with the list of problems. If the server is dead, the pings stop, and that's the alert. - Backups: the nightly job reports start, success and failure to a second check.
- Alerts go to email, WhatsApp and Discord.
We tested the failure path for real: stopping a container produced "only 3/4 containers running" within one run.
Still on the list
- Checking the BIOS for a "power on after AC loss" option. Lenovo consumer laptops rarely have one, so the fallback is a trusted person who can press the power button.
- A low-battery clean shutdown, to stop adding to that unsafe-shutdown count.
- A full reboot test, so every timer and service gets exercised from a cold start.
- An offsite backup copy.
- Rotating the credentials that passed through chat while we were moving fast.
The reboot test turned out to matter more than I thought. Within a day, two of the new jobs failed for real. That's part 3.