FROM THE HOME RACK TO THE PUBLIC INTERNET
The public side of a project that started on a tiny Atom box
The original public server was the same Atom D525 / 8 GB Supermicro that still sits in the rack today. It started from a simple question: can I collect data continuously and show it somewhere?
Eventually eight-ish observers were enough to make the little machine say “bro, that's plenty”. The box survived the migration and became Supermicro B, the lab's current helper server.
Rented in January 2026 and hosted in a German datacenter near Braunschweig. Ryzen 9 9900X3D, 64 GB DDR5, 2×4 TB NVMe and a 2 Gbit/s LACP uplink. Proxmox separates the website, observer jobs and other private services into Debian guests.
Direct SSH is restricted to the fixed public IP of the home connection and intended for Cthulhu only. Keys only, no password login. Away from home, the provider panel is reserved for broad emergency actions such as restart, stop or restoring a provider backup.
The first observer was Area51: a tiny private experiment that counted publicly visible JANET-style and other flights via the ADS-B Exchange API. At first it was just text and numbers for a month — essentially “5 / 5 / 5 / 13 / 5” — to see whether repeated public observations became interesting.
Small visual cards came next. Around late 2024 the first public World Observer page appeared. Categories and detailed subpages came later as the collection grew. Today there are 41 active observers spanning Internet, Media, Society, Hometown, Earth & Space and Technology.
One readable schedule file defines when observers should run; cron triggers the machinery. That avoids a forest of unrelated cron entries and makes overlaps obvious.
A failed collection retries every ten minutes for roughly one hour — about six attempts. After that the value stays unavailable. A noon fuel price does not quietly turn into a 23:00 fuel price.
No carry-forward, no interpolation and no invented zero. A partial observer may publish valid sub-values while leaving the failed series empty. Statistics ignore missing observations rather than pretending they existed.
Observers emit values and diagnostics. The website knows which JSON belongs to which card and decides whether a value is euros, degrees, a count or an index. Change the labels on the website and you can mislabel perfectly good source data — the collector has no opinion about presentation.
This separation is deliberate. Someone can reuse the observer pattern for ten pet-food prices from retailer XYZ, build a completely different front end and ignore Heartbeat, guestbook, traffic and weather entirely.
The platform writes one independent heartbeat at every full UTC hour and keeps only the latest 24. A missed expected beat changes the public status to STALE and preserves the last known heartbeat time. A later beat returns the status to ALIVE automatically.
Heartbeat does not retry. A missing hour remains a missing hour. It is committed separately from observer data so platform liveness cannot be confused with a broken API or one degraded collector.
The website and World Observer are separate repositories. Current code can be redeployed from GitHub; long-running observation history cannot be recollected after the fact. That history is therefore included in provider backups and also versioned into a private GitHub repository every few days.
The code can be redeployed. The history cannot be recollected.
Observers run when the measurement is meaningful, not because every job deserves the same cadence. Fuel is measured around noon; tea waits until the local Combi API has reliably updated; the media-language observer samples a stable headline basket every morning. All scheduling and timestamps stay in UTC.