Rahul's Build Log

My "LAN speed test" was actually an internet speed test

2026-10-22T06:01:00Z

I wanted to know how fast my server's network actually is. The standard tool is iperf3: run a server on one end and a client on the other, and it measures raw throughput with the disks out of the picture.

iperf3 -s                    # on the server, listens on port 5201
iperf3 -c <server-ip>        # on the client
iperf3 -c <server-ip> -R     # reverse direction: server sends to the client

Attempt 1: timed out

I pointed my Windows PC at the server's wired LAN address and got:

iperf3: error - unable to connect to server - server may have stopped running
or use a different port, firewall issue, etc.: Connection timed out

My first thought was the firewall. But ping to the same address timed out too, so this wasn't about iperf3 or a port at all. The PC couldn't reach that address.

The clue I'd left out

Then I mentioned that I was talking to Claude over SSH from that same PC. If SSH works but the LAN address doesn't, the SSH session must be taking a different path. Claude checked where the session was coming from: a Tailscale address. The PC wasn't on my home LAN at all. It was reaching the server through the Tailscale tunnel.

Attempt 2: about 30 Mbit/s

Pointed at the server's Tailscale address, iperf3 connected and reported about 30 Mbit/s. That's terrible for gigabit Ethernet, so I asked whether something was wrong.

It wasn't. tailscale status showed a direct connection (not relayed through a DERP server), with the PC at a public IP on a different network about 10 ms away. So the test measured:

the slower of the PC's download speed and the server's home upload speed, plus WireGuard overhead.

30 Mbit/s is a normal home upload speed. The number was right. I was just measuring something other than what I thought.

Checking the actual LAN link

To rule the cable out, Claude ran a few checks from the server towards the router:

Link speed: ethtool reported 1000Mb/s, so the Cat6 cable and router port negotiated gigabit. A damaged cable often falls back to 100 Mbit/s.
Latency to the router: about 0.8 ms average, with 0% packet loss.

The wired link is healthy. Measuring its real throughput needs a second wired device on the same LAN running iperf3 -s, since most routers can't act as an iperf3 endpoint.

Takeaways

A timeout on both ping and the app means a reachability problem, not a port or firewall problem in the app.
Know which path your traffic takes. With Tailscale running, "the server's address" can mean a LAN IP, a tailnet IP or something routed in between. Check with tailscale status, and echo $SSH_CLIENT shows where an SSH session is coming from.
Over a VPN, iperf3 measures the internet. Over the internet, home upload speed is almost always the bottleneck.
ethtool <iface> | grep Speed is the 5-second cable check before you go buying new cables.

My new monitoring paged me twice on day one. Good.

2026-10-22T05:45:00Z

The hardening work added two new scheduled jobs: a 5-minute heartbeat to healthchecks.io and a nightly restic backup. Both had passed their tests. Within 24 hours, both triggered a Discord alert.

Alert 1: the heartbeat "failed"

archlinux: Failed systemd units: server-heartbeat.service

The server was fine. The journal showed two failed pings to hc-ping.com: a timeout, and then an HTTP/2 error on the next try. My own connection looked healthy (0% loss to 1.1.1.1 at about 2.5 ms), but the route to the ping service was about 240 ms away and dropping a lot of ICMP.

The real bug was the design. The heartbeat script exited non-zero when a ping failed, so systemd marked the unit failed, and my existing health monitor (which reports failed units to Discord) raised the alarm.

But a missed ping is already the signal. healthchecks.io alerts when pings stop arriving, so making the unit fail too just creates a second, noisier alert. The fix:

ping over HTTP/1.1, which avoids the HTTP/2 failure;
retry up to 5 times, 3 seconds apart;
never fail the unit because a ping failed. If the pings really stop, healthchecks.io alerts after the 10-minute grace period.

The backup job's pings got the same treatment.

The good news: the alert chain worked end to end. A problem was detected and reached my phone within minutes.

Alert 2: the backup failed at 03:36

⚠️ archlinux: Failed systemd units: server-backup.service

This one was a genuine failure, and nothing had been backed up that night. restic stopped within a second because it couldn't find a cache directory. When systemd runs a service, HOME isn't set, and restic puts its cache under $HOME/.cache.

The first run had worked because it was started by hand from a shell that has a home directory. The fix was one line in the unit and one in the script:

# /etc/systemd/system/server-backup.service
[Service]
Environment=HOME=/root

# /usr/local/bin/server-backup
export RESTIC_CACHE_DIR=/var/cache/restic

Running it exactly the way the timer does took about 12 seconds, the repository check came back clean, and the backup check on healthchecks.io went green again.

The lesson: test the way it will actually run

Both jobs passed their tests. Both tests ran in a different environment from the real one:

Job	How it was tested	How it really runs
Heartbeat	One manual ping, on a good connection	Every 5 minutes, all day, over a flaky route
Backup	Started from an interactive shell	Started by systemd at 03:30, with no `HOME`

If I'd found the backup bug in month two instead of on day one, I'd have had weeks of "backup failed" alerts to deal with from abroad, or worse, I'd have missed them. Before I leave:

Run every unit with systemctl start, not by running the script by hand. That's the environment the timer gives it.
Do a full reboot test, so everything starts from cold the way it will after a power cut.
Make alerts mean one thing. One problem should raise one alert, from the system best placed to notice it.

Two pages on day one is annoying. Two pages in month two, from another country, would have been much worse.

Getting a homelab ready to run for three months without me

2026-10-21T08:33:00Z

I'll be out of the country for three months with no physical access to the server, and I had about a month to prepare. After the audit, the question became concrete: what could take this machine down or lock me out while I'm away, and how would I even know?

I started a new Claude Code session on the server and asked it to find every failure point and fix what it could.

The failure points

The first survey sorted the risks into "critical before you leave" and the rest:

The machine could fall asleep. It's a laptop, and closing the lid suspends it by default. A suspended server is a dead server.
Power cuts. The battery is at about 74% of its design capacity after 1,490 cycles, and there's no UPS. After an outage, the machine only comes back if the BIOS powers it on when AC returns.
One way in. One SSH key, and one other device on Tailscale. Lose that laptop and I'm locked out for three months.
No backups. The only copy of everything, including .env files and tunnel credentials, was on one SSD.
SSH accepted passwords on every interface.

Never sleep, and reboot if frozen

The easy wins went in first:

a logind.conf.d drop-in so the lid switch, idle and the power key are ignored;
sleep.target, suspend.target and hibernate.target masked, so nothing can suspend it by accident;
a hardware watchdog that reboots the machine if it freezes for 30 seconds;
a full system upgrade (including Tailscale), the journal capped at 500 MB and 3 months, and 2 GB of Docker build cache cleared;
/boot permissions tightened, so the random seed is no longer world-readable.

SSH: keys only, verified before switching

My past logins had used a password, so turning passwords off blind could have locked me out. The order mattered:

I tested a key-only login from my laptop: ssh -o PasswordAuthentication=no ....
Only after that worked did PasswordAuthentication no go in, along with no root login, AllowUsers, 3 attempts maximum and a short login grace time.
fail2ban watches sshd, with my own networks on the ignore list so I can't ban myself.

A second way in

A second device on Tailscale and a second SSH key, on different hardware:

I added my phone to the tailnet.
I generated an ed25519 key in Termux. My first try failed because I typed ssh keygen instead of ssh-keygen, and ssh tried to connect to a host called "keygen".
Claude added the public key, I logged in from the phone on mobile data with Wi-Fi off, and the SSH log confirmed it was the phone's key that got in.
I turned off Tailscale key expiry for the server, which otherwise drops off the tailnet after about 180 days.

Wiping the HDD: storage and backups

The 1 TB HDD only held old Windows data, so I asked for it to be wiped. Before touching it, Claude checked the drive's serial number matched the HDD, not the NVMe. That's a cheap check against the most expensive possible mistake.

It's now two ext4 partitions, both mounted with nofail so a dead drive can't stop the boot:

Mount	Size	For
`/mnt/storage`	462 GB	A network storage project
`/mnt/backup`	454 GB	Backups, root-only

Backups that are tested, not assumed

The backups use restic: a nightly systemd timer backs up /home and /etc to /mnt/backup, keeps 7 daily, 4 weekly and 6 monthly snapshots, and runs restic check afterwards.

The first run used 35 GB. The same job then restored a file from the snapshot and compared it with the original, and they matched. A backup you've never restored from is only a hope.

The weak spot is clear: backups on the same machine as the data don't survive theft, fire or a power surge. An offsite copy is still on the list.

Docker and the demo sites

The two app ports that were published on every interface are now bound to 127.0.0.1. The tunnel and Caddy reach them locally.
Container logs rotate at 10 MB × 3 files.
Fresh Caddy and cloudflared images, all containers recreated, and all three sites returned 200 afterwards.

Trying network storage, then turning it off

I tried a Samba share for /mnt/storage. It was set up as tightly as I could think of:

SMB3 with encryption required;
no guest access;
bound to the Tailscale address only, with an nftables rule dropping SMB from anything else;
a dedicated Samba password.

Windows rejected the first password (the fix was a simpler password and the username written as server\user), but after that it worked. Then I asked for it to be turned off again. I want to understand the trade-offs before leaving a file share running for three months. It's disabled with the config kept, so turning it back on is one command.

An alarm that works when the server doesn't

The server already sends health reports to Discord, but if it dies, it goes quiet, and silence looks exactly like "everything's fine". So we added a dead-man's switch with healthchecks.io:

Heartbeat: every 5 minutes, a script checks that all mounts are present, all containers are running, the sites respond on localhost, there are no failed units, the disk is under 90% and Tailscale is up. If all of that holds it pings healthchecks.io. If not, it pings the /fail URL with the list of problems. If the server is dead, the pings stop, and that's the alert.
Backups: the nightly job reports start, success and failure to a second check.
Alerts go to email, WhatsApp and Discord.

We tested the failure path for real: stopping a container produced "only 3/4 containers running" within one run.

Still on the list

Checking the BIOS for a "power on after AC loss" option. Lenovo consumer laptops rarely have one, so the fallback is a trusted person who can press the power button.
A low-battery clean shutdown, to stop adding to that unsafe-shutdown count.
A full reboot test, so every timer and service gets exercised from a cold start.
An offsite backup copy.
Rotating the credentials that passed through chat while we were moving fast.

The reboot test turned out to matter more than I thought. Within a day, two of the new jobs failed for real. That's part 3.

ExamX, part 2: cloud models, a public site, and pressing pause

2026-10-21T00:00:00Z

Part 1 ended with a working generator and grader, held back by a local model that was too small to write good questions. Around midnight I asked the obvious question: would a free cloud model fix it?

Would Gemini solve the problem?

Mostly, yes, with a few catches that were worth spelling out before building anything:

It's no longer local. That was fine: I dropped the "runs locally" requirement.
The free tier may use your prompts for training. That would include past-paper text and my answers.
It needs an API key, which has to live in the environment and never in the repo.
The exact checkers stay in charge of anything they can check. A better model makes fewer mistakes, but not zero.

Three back ends and a model switcher

The LLM client got three interchangeable back ends, chosen with a provider:model spec:

examx generate --model qwen3:8b                  # local, via Ollama
examx generate --model gemini                    # Google's default model
examx generate --model claude:claude-sonnet-5-5  # Anthropic
examx models                                     # which back ends are ready

The web UI got a matching AI model page. It shows whether each provider is ready and how to set it up if not, and a chip in the header always shows the current model. The UI also gained dark mode, a phone layout, past-paper diagrams inside questions, and answers saved in the browser as you type.

The free tier is 20 requests a day

I set up a Gemini key (in a file only my user can read, loaded by ~/.bashrc, nowhere near the repo). The first surprise: gemini-2.5-flash was no longer offered to new keys, so the default moved to gemini-3.5-flash. It answered a test question about decidability correctly in under 4 seconds.

Then a full paper ran for 22 minutes and produced almost nothing from Gemini:

the free tier allows 20 requests per day for that model, and one AI-written paper needs 15–25;
several calls failed earlier anyway with "model experiencing high demand";
my retry logic waited and retried for minutes, even after the daily quota was gone.

The fix was to treat "quota used up for hours" differently from "slow down for a minute". ExamX now stops at once, skips that model for the rest of the run, and says when the quota resets. In practice the free tier gives you about one AI-written paper or one marked attempt a day.

Putting it online

I decided to try Claude Sonnet instead and asked for the app at exam-x.vrp2206.com. The server already ran Caddy and a Cloudflare Tunnel for my hackathon demo sites, so ExamX went in as a Docker container behind its own tunnel.

Two design decisions from this part are worth keeping:

API keys live in the visitor's browser, not on the server. You paste your key into the AI model page. It's stored as an HttpOnly cookie, checked once against the provider's free model-list endpoint, used only while your job runs, and never written to disk or logged. By default it's forgotten when you close the browser.
Fail closed. The app first refused every request that hadn't passed Cloudflare Access.

Cloudflare Access then asked for a credit card, so I asked to drop it. Before switching the guard off, Claude added the protections a public site needs:

visitors can only use their own Claude or Gemini key, never my local GPU models or a server-side key;
each browser gets an anonymous ID and only sees its own papers and reports;
the model choice is per browser, and each visitor has daily job limits;
the owner still has full access from the server itself.

Pressing pause

That's where I stopped. The Claude API is billed separately from a Claude Pro subscription. The free Gemini tier is too small to rely on. The local models are too weak to write new questions. And the real goal was passing the exam, not shipping a product the night before.

So ExamX was shut down cleanly:

both containers (the app and its tunnel) removed, and the Caddy route removed and reloaded, with the other sites checked afterwards;
a "Project status: paused" section at the top of the README, covering what's done, what I learned and the next steps;
the code tagged v0.1-paused.

I kept the Ollama models (they're useful for other ML work) and left the GitHub repo public rather than archived, in case someone else wants to pick it up.

What I took away

Let code mark what code can mark. The exact checkers were the most reliable part of the whole project, and they need no model at all.
Small local models are good at following a template, not at inventing correct maths. Using real past questions with official solutions beat every LLM-written alternative I tried.
Free tiers have shapes. "20 requests a day" changes the whole design of a tool that needs 25 requests per run.
Don't paste secrets into a chat. I pasted a tunnel token into the conversation to save time, which meant treating it as leaked and replacing it. The final version of the app asks for keys through a form instead.
Stopping is a valid ending. The tool still works offline for practice: examx generate --no-llm and exact marking, no AI needed.

Auditing a gaming laptop that thinks it's a server

2026-10-20T14:21:00Z

My homelab server is a Lenovo Legion gaming laptop running Arch Linux. It hosts my hackathon demo sites (kept online for my CV), Docker, Tailscale and a Cloudflare Tunnel. It had been running for a while without anyone looking at it critically, so I asked Claude Code to audit the quality of the system and how it was set up.

The ground rules

Read-only first. The first pass ran without root.
Don't touch the other Claude. Another Claude Code session was busy building ExamX in a tmux window on the same machine. The audit stayed away from that directory and that session.
Root without sharing the password. A ! command in Claude Code has no terminal, so sudo can't prompt there. Instead of passing my password into the session, Claude wrote a read-only audit script. I read it, ran it with sudo in my own terminal, and Claude read the output file.

That last pattern is worth reusing: the AI writes the script, you review and run it, and it reads the results. No credentials ever enter the chat.

What was already fine

No failed systemd units, and the system was up to date.
Only 59 explicitly installed packages. Package signature checking was on.
sudo requires a password (no NOPASSWD anywhere).
Every SSH login in the journal was key-based, from my own laptop over Tailscale.
Both drives passed SMART. The HDD had 0 reallocated or pending sectors, and the NVMe had 0 media errors.
No port forwarding: remote access goes through Tailscale and a Cloudflare Tunnel.

What wasn't, worst first

No host firewall at all. The INPUT policy was ACCEPT for IPv4 and IPv6. SSH, HTTP, the tunnel's metrics port and two Docker-published app ports were listening on every interface. On a machine that has been moved between networks, that matters.
SSH accepted passwords. sshd -T confirmed PasswordAuthentication yes. I only ever use keys, so turning passwords off costs nothing.
/boot was world-readable, including the random seed file, which systemd flags as a security hole.
86 unsafe shutdowns on the NVMe, and 17% wear after only about 3,000 power-on hours. The last boot had ended with watchdog did not stop!. That's what a laptop that sometimes gets hard-powered-off looks like.
No backups. Nothing was configured, /home lived on a single drive, and the 1 TB HDD held old Windows data, mounted read-only.
Smaller items: membership of the docker group (root-equivalent), unpinned :latest images, 589 MB of reclaimable build cache, orphaned packages and a BIOS from 2022.

There was also one thing the audit couldn't fully explain: / was 20 GB used of 32 GB, but Docker only accounted for 0.4 GB of it.

The output: a plan, not changes

The result wasn't a pile of edits. It was a single Markdown file:

a snapshot of the system and what's already good;
an 11-item TODO list in priority order;
copy-pasteable fixes, each with a rollback note;
a suggested order of work and the open questions for me to decide.

I sent it to my Windows laptop with Taildrop (tailscale file cp). Taildrop needs root, or a one-time tailscale set --operator=<user>. Claude wouldn't change my Tailscale settings without asking, so I ran the send myself.

Nothing was changed in this session. The point of an audit is to know what you're dealing with before you touch it.

The next day this audit turned into a much bigger job. I'll be out of the country for three months with no physical access to the box. Part 2 is about getting it ready for that.

ExamX, part 1: a practice-exam generator that marks its own papers

2026-10-20T13:35:00Z

My FIT2014 exam (Theory of Computation at Monash) was coming up, and I had five past papers but only two sets of solutions. I wanted more practice than that, so I wrote a brief for a system and handed it to Claude Code, running on my homelab server: build a tool that generates new practice papers with worked solutions and marks my attempts, using only free tools and a local model.

This post covers that first evening. Part 2 covers cloud models, putting it online, and why I paused it.

The brief

Data: the 2013 sample exam plus the 2013, 2014, 2015 and 2017 papers. Only two of them came with solutions.
Hardware: the server has 8 CPU cores, 31 GB of RAM and a GTX 1650 with 4 GB of VRAM. That's a small budget for running an LLM.
Rules: free tools only, a Python virtual environment inside the repo, small commits pushed often, and no credentials stored anywhere.

Step 1: getting the papers in

All the PDFs had real text in them, so no OCR was needed. Ingestion used PyMuPDF to split each paper into questions and parts, tag each one with a topic, and match it to its official solution where one existed.

One check made me trust the parser: every paper's marks added up to exactly 120. That caught several splitting bugs early.

Two details mattered more than I expected:

Superscripts. Expressions like a^mbⁿ lose their meaning if the exponent becomes plain text. The parser reads font sizes and positions so they survive as a^{m}b^{n}.
Colour. In the solution PDFs, the answers and the marker's comments are in different colours, so the parser separates them by colour.

The result was 101 structured questions in JSON, with the diagrams saved as images.

Step 2: let code mark what code can mark

Much of FIT2014 has answers you can check exactly. Two regular expressions either describe the same language or they don't, and a Turing machine either computes the right output or it doesn't. So before any LLM was involved, the project got a set of exact checkers:

Checker	What it decides
Regex / automata	Language equivalence, with the shortest string where two answers differ
Subset construction and minimisation	Whether an NFA→DFA conversion or a minimised DFA is correct
Context-free grammars	Membership (Earley parsing), derivations, bounded language comparison
Turing machines and PDAs	Simulation on test inputs
Propositional logic	Truth tables and CNF

Because the checkers name a concrete counterexample, the feedback is specific: "your regex accepts abba, the language doesn't".

One bug was a good lesson. Some reference Turing machines kept crashing off the left end of the tape. FIT2014 uses one-way infinite tapes, so a machine can't detect the left end unless it marks it first. The fix belonged in the machines, not in the simulator.

Step 3: generating new papers

The generator samples a blueprint from the past papers (which topics, how many marks) and fills each slot in one of two ways:

Procedural questions. Code generates the automata, grammar, logic and decidability questions, so their solutions are correct by construction. Each model solution is fed back through the grader and must score full marks.
LLM questions. For proofs and explanations, a local model writes new questions using past-paper questions as examples.

Papers come out as Markdown, PDF and JSON. The PDF path is Markdown → HTML → PyMuPDF, so no LaTeX install was needed.

Where the local model fell over

I benchmarked the Qwen3 models through Ollama. The 8B model was the largest that ran at a usable speed (about 8 tokens/s, split between the 4 GB GPU and system RAM), and the only one that wrote a correct proof in the benchmark.

A full paper took 19 minutes, and about half of the LLM-written questions were poor:

near-duplicate questions;
a "prove that…" of a statement that is false;
wrong answers on decidability;
an NP-completeness question that didn't make sense.

So we added quality controls: near-duplicate drafts got rejected, a self-review pass checked each question, and LLM-written questions got flagged in the solutions. The second run took 39 minutes and was worse. The self-review changed a correct answer ("is this grammar property decidable? No") into a wrong one. Five questions fixated on the same grammar, and two of them asked students to prove something it doesn't generate.

An 8B model on a 4 GB card can't be trusted to invent exam questions and their solutions. The checkers can't save a question that's wrong to begin with.

The fix was to give the LLM less to do. Decidability tables and language-class questions moved to code. Proof and explanation slots now use real past questions with their official solutions by default. LLM-written questions became an opt-in --prose llm experiment.

Marking proofs

Proofs can't be checked by code, so an LLM marks them against a rubric. The 8B model gave a correct proof 1–3 out of 7 and claimed a true step was false. Moving proof marking to qwen3:14b fixed most of that:

the correct proof got 5/7, with accurate feedback;
a deliberately bogus proof got 0/7, so the 14B model wasn't just being lenient;
each proof took about 2 minutes to mark.

Proof marks are still labelled advisory in the report.

Where it ended up after one evening

A CLI (ingest, generate, grade, check, serve and more) and a small web UI built on Python's standard-library HTTP server.
108 passing tests, many built from real past-paper answers.
An example paper, a deliberately mixed sample attempt, and its marking report in the repo.
A README that records both LLM runs and their failures as well as the successes.

examx generate --no-llm makes a fresh paper in about a second, and the exact checkers mark it instantly. That alone was worth building. The open question was whether a better model would fix the question writing, which is where part 2 picks up.

Hello, world (again)

2026-10-19T00:00:00Z

My portfolio shows the finished projects. This blog is for everything around them: the half-working prototypes, the hackathon weekends, the homelab rabbit holes, and the bits of CS that finally clicked.

What I'll post here

Project write-ups: how something was built, what went wrong, and what I'd do differently.
Hackathon recaps: the idea, the 3am pivots, and the result.
HomeLab notes: configs, dashboards and the occasional broken server.
Things I'm learning: algorithms, AI tooling, cloud, whatever I'm deep in that month.

Writing it down is the fastest way to find out whether I actually understand it.

More soon.