Listening for events…
← All Docs

Computational Fragility Assessment — 2026-07-11

Author: Claude (managing-editor / ops) Context: Mike wants to open a second concurrent Claude session in a new screen and asked whether that (or any workload) risks breaking the box, given that "Brad had some drive issues." This is a re-run of the 2026-06-07 assessment (docs/fragility-assessment-2026-06-07.md) on today's live numbers. Method: Live read-only measurement on impera, 2026-07-11 ~14:23 local. df, free, vmstat, uptime, ps, dmesg, systemctl, pg_lsclusters, pg_isready, openssl. Nothing written, no config changed, no non-TerraPulse tenant touched.

TL;DR

Opening a second Claude session is safe. It will not break the box. A session is light (~9% of one core, small RAM). Your normal work in it — papers, git, read-only queries, small dex reads, editorial — is fine.

TerraPulse itself is healthy right now: API service up (1d 19h), PG16 online on :5433 accepting connections, load moderate and falling. Your live product does not depend on the flaky drive.

But three things are worse than the June baseline and shape the guardrails:

  1. RAM cushion is gone. June had 57 GiB free. Now: 725 MiB free, 13 GiB already in swap (48 GiB is reclaimable cache, so not critical — but the headroom is spent). → Don't run heavy memory jobs in both sessions at once.
  2. The watchdog we installed on 06-07 is gone. /dev/watchdog absent; sp5100_tco didn't survive (11-day uptime without it). systemd is configured for a 30 s watchdog but has no device to pet. → If the box freezes, no auto-reboot again — manual recovery, like the 36 h outage.
  3. The drive issue is real and active — it's /mnt/ursa. The 7.3 TB drive that dropped in the May-30 crash (and still hosts the down PG15) is resetting on the USB bus right now (12 resets this window) and logging EXT4 errors. → Do not write anything to /mnt/ursa. The old June doc's advice to send heavy writes there is now backwards.

⚠️ CORRECTION (2026-07-11, later same day)

The TL;DR above (and the original Brad note) said TerraPulse's live DB is on internal NVMe so "normal operation doesn't depend on /mnt/ursa." The DB half is true — but the claim is wrong: a large part of TerraPulse's data layer lives on /mnt/ursa via repo symlinks, discovered while scoping a WSPR paper flip:

  • data/event_storehouse → /mnt/ursa — every Eventdex (eq, tor, cme, flare, wspr_event_window, gw, fema, ~40 kinds). The bulk of the dex platform.
  • Site render caches → /mnt/ursa — stats_cache.json, timeline.json/.parquet, geoindex.json/.parquet, graph_cache.json, sources_cache.json. Live pages read these.
  • data/duckdb (staging) and Postgres backups → /mnt/ursa.
  • The 200 GB WSPR raw archive → /mnt/ursa.

Safe on internal NVMe: PG16, the web app, and the newer storehouse families — year / location / platform / celestial / directory / yearlocation (incl. the cosmic_ray_flux and aircraft dexes).

Real exposure: if /mnt/ursa drops like it did on May 30, TerraPulse loses the Eventdexes and the site's render caches — a live-site outage, not just "staging." This is understated in the TL;DR. Recommended infra fix (Brad's lane): move event_storehouse + the site caches off /mnt/ursa onto internal NVMe (the same migration PG16 got), and/or stabilize the drive. Reads from /mnt/ursa are already routine (every Eventdex query, every page load), so "don't write to it" stands but "don't read it" was never realistic — the platform depends on it.

What's healthy (measured)

  • TerraPulse API: terrapulse.service active, up 1d 19h.
  • TerraPulse DB: PG16 on /var/lib/pgdata (internal NVMe) — online, accepting connections, mount at 51 % / 215 GB free. Not on the flaky drive. (But note the DB is not the whole platform — see the correction below: much of TerraPulse's data layer IS on /mnt/ursa.)
  • terrapulse.info TLS cert: valid, renewed Jul 9, expires Oct 7. Fine.
  • Load: 3.80 / 4.20 / 6.60 on 12 cores — moderate, trending down.
  • RAM available: 48 GiB (via reclaimable cache) — no OOM risk for light work.

What's degraded vs. June

  • Swap in use: 13 GiB (June: near-zero active). No active thrash (vmstat si/so ≈ 0), but the free-RAM buffer is gone.
  • Root /: 97 % / 15 GB free (June: 96 % / 19 GB). Slightly tighter. data/ caches live here. Still a latent constraint — don't dump GB to /.
  • Watchdog: regressed to absent (see TL;DR #2).
  • /mnt/ursa (sdm1): active USB resets + EXT4 error count. The device is re-enumerating (seen as both sdh1 and sdm1 across two reads seconds apart) — a flapping-USB signature. This is Brad's "drive issue," live.

Not TerraPulse (flagged, out of scope)

  • certbot.service failed — but the 5 failures are for other domains (virginislandspulse.com et al.), not terrapulse.info. Brad's / other tenants.
  • dragonfli-enrich-refresh.service failed; two ffmpeg procs at 185 % / 100 % — nominate.ai / Campaign Brain. Brad's lane.
  • postgresql@15-main failed/down — the decommissioned PG15 on /mnt/ursa. Migration Phase 4 still pending.

Guardrails for the fragility test (second screen)

  • ✅ Light work in the 2nd session — papers, git, read-only PG queries, small extracts, editorial. Go ahead.
  • ⛔ Don't launch heavy/parallel backfills in both sessions simultaneously (RAM cushion is spent).
  • ⛔ Don't write to /mnt/ursa (flaky, resetting) or fill / (15 GB free).
  • ✅ Roomy + healthy scratch mounts: /mnt/marzano (1.2 TB free), /mnt/nom01 (2.9 TB), /mnt/nom02 (2.8 TB). No errors on these.
  • 👀 If it feels sluggish: free -h. If swap climbs toward its 31 GiB ceiling, back off.

For Brad (his lane — flag, don't fix)

  1. Hardware watchdog (sp5100_tco) didn't persist across the last reboot — auto-recovery net is off.
  2. /mnt/ursa is throwing USB resets + EXT4 errors again — same drive as the May-30 crash, still hosts PG15.
Live Feed