Listening for events…
← All Docs

Ops note for Brad — 2026-07-11

Two box-level items surfaced during a read-only fragility check on impera today (2026-07-11 ~14:23 local, uptime 11d). Both are in your lane (kernel/USB/global config); flagging, not touching. Full context: docs/fragility-assessment-2026-07-11.md.

1. Hardware watchdog didn't persist across the last reboot

The sp5100_tco watchdog we stood up on 2026-06-07 is gone this boot:

  • /dev/watchdog — absent.
  • systemd is still configured for it (RuntimeWatchdogUSec=30s, RebootWatchdogUSec=10min) but has no device to pet, so there's effectively no auto-recovery — a hard freeze would again need a manual power-cycle (the thing that cost ~36h on 06-01).
  • /etc/modules-load.d/sp5100_tco.conf was supposed to auto-load it at boot. Either it's not loading (blacklist? kernel update dropped the module? load order?) or the file didn't survive. Worth a modprobe sp5100_tco + dmesg | grep -i tco to see if it even binds on the current kernel.

2. /mnt/ursa is throwing USB resets + EXT4 errors again

Same 7.3 TB drive that dropped in the 05-30 crash, and it still hosts the (down) PG15 cluster at /mnt/ursa/data/postgresql. Live signature this boot:

usb 4-2.4: reset SuperSpeed USB device number 119 using xhci_hcd   (×12 in the window)
EXT4-fs (sdm1): error count since last fsck: 1
EXT4-fs (sdm1): initial error at time 1783613764: ext4_journal_check_start:84
EXT4-fs (sdm1): last error at time 1783613764: ext4_journal_check_start:84

The device is also re-enumerating — it showed as sdh1 and sdm1 across two reads seconds apart, which is the flapping-USB pattern from last time. Suggests cabling/hub/drive on port 4-2.4, not filesystem-level rot, but the ext4 error count means it's worth an fsck on a clean unmount.

CORRECTION (same day): an earlier version of this note said "no TerraPulse impact — the live DB is PG16 on internal NVMe, independent of /mnt/ursa." The DB is on NVMe, but the platform is not independent of /mnt/ursa. Repo symlinks put a large part of TerraPulse's data layer on that drive:

  • data/event_storehouse → /mnt/ursa — every Eventdex (eq, tor, cme, flare, wspr_event_window, ~40 kinds).
  • Site render caches → /mnt/ursa — stats_cache.json, timeline, geoindex, graph_cache, sources_cache (live pages read these).
  • data/duckdb staging + Postgres backups + the 200 GB WSPR raw archive → /mnt/ursa.

So if /mnt/ursa drops like May 30, TerraPulse loses the Eventdexes and the site's render caches — a live-site outage, not just staging. PG16 and the newer storehouse families (year/location/platform/celestial/directory/yearlocation) stay up (internal NVMe).

Suggested: (a) re-arm the watchdog and confirm it binds on the current kernel; (b) fsck + reseat/re-cable the 4-2.4 drive; (c) migrate event_storehouse + the site caches off /mnt/ursa onto internal NVMe (same move PG16 got) so a drive drop doesn't take the dex platform + site with it; (d) finish PG15 → NVMe (Phase 4).

— Claude (TerraPulse ops)

Live Feed