Dex-build plan — paper-data standardization campaign
Drafted 2026-07-07. Turns docs/paper-source-inventory.md into a build roadmap.
The campaign, in Mike's sequencing (2026-07-07)
- Leave every shipped paper's current data sourcing in place. Papers are not touched during the build track.
- Separate build track: create/extend dexes to hold each paper's actual data, in whatever shape and size works — the dex is shaped to fit the paper's need, not the reverse.
- The very last step of the whole campaign is the nuts-and-bolts flip of each paper's data source from its raw pull to the dex. One contained step at the end, not a running hazard.
Shape is a per-phenomenon categorization call. Each build item below carries a proposed shape and its open shape questions; Mike rules the shape before that item is built.
Ruling already given (2026-07-07): earthquakes get magnitude-tiered eventdexes (e.g. M<3, M<2, M<1 tiers) sitting below the existing M4.5+ notable-event spine. Complete-catalog dexes coexist with the spine; we do not stretch the spine.
Build items, ranked by leverage (papers unlocked)
| # | Dex to build / extend | Shape (proposed) | Source(s) | Papers unlocked | Effort |
|---|---|---|---|---|---|
| B | WSPR dex (shape SETTLED 2026-07-07; no station Locationdex) | (1) wspr_spot_census month-grain Yeardex — BUILT (221 month slots, carry-both: day×band aggregate working surface + raw archive pointer); (2) wspr_event_window event-window Eventdex — BUILT (65 slots = 17 outbreaks + 44 controls + 3 aircraft corridors + 1 storm-season; slot = a paper's event window, carve-both: cell×15min×band SNR aggregate working surface + raw partition pointer; 61 carved, 4 in the un-backfilled 2026 tail pending WS-4; spine-parquet, out of the shared index) |
wsprnet / wspr.live (raw already on /mnt/ursa; no fetch) | 6 (corrected from 8) | HIGH (volume + shape) |
| A | Earthquake magnitude tiers | Magnitude-tiered eq eventdexes below the M4.5 spine (Mike-ruled) + register EMSC/GFZ/ISC |
usgs_earthquake (+ emsc/gfz/isc, already in observations) |
4 | MED |
| C | Space-weather index cluster | "Geophysical index" dex family (continuous measured scalar series), sibling to the gst/Dst dex |
DSCOVR solar wind, SILSO sunspots, GOES X-ray flux, NOAA Kp | 4–5 | MED |
| D | Aircraft moving-platform dex | MovingPlatform family (sibling to mobile_radiosonde); aircraft tracks |
OpenSky, ADS-B (adsb.fi) | 2 | MED |
| E | Safecast radiation | Locationdex (device roster); second source alongside radiation_monitor (EPA RadNet) |
Safecast | 1 | LOW–MED |
DROPPED 2026-07-07 — used by 0 shipped papers (skinwalker's bolide layer is CNEOS, already in entry); AMS is an inactive 0-row stub needing an API key (#94). Not a campaign item. |
— | 0 | — |
Not dexable — model output (bright-line OUT, cannot build)
All three live only in skinwalker-ranch-seven-layers, already held out of its measured tests:
- SPC convective outlook — forecast product.
- open_meteo / open_meteo_aqi — forecast.
- era5_cloud — reanalysis grid.
Structural consequence: skinwalker can never be 100% dex-sourced. At the flip, its model
layers stay as documented held-out context (their current status) or are dropped; they are never
dexed. This is the one shipped paper that cannot fully standardize. Flag, don't force.
Already dexed — verify grain (likely low/no build; confirm the dex holds the paper's grain)
- lunar-syzygy-seismicity →
eqnotable spine (M6+): verify the spine is complete for global M6.0+ 1973–present. Likely yes; if so, this repoints with no new build. - earthquake-lights-104-years →
eq+nuforc: temporal grain gap — the paper spans 104 years but theeqdex starts 1973 (53 yr). Needs pre-1973 quakes the dex may not hold. Flag for aneqhistorical-extension decision. - tornado-alley-east →
tordex: verify grain. - fireball-surge-2026 →
neo/entrydex: verify grain. - Dexed portions of the compound papers (
sky-anomalies,solar-watch-april-2026,starlink-thermosphere-storm-pilot): verify each dexed layer's grain at flip time.
Proposed build order
Two defensible orderings — Mike picks:
- Leverage-first: B (WSPR, 8) → A (earthquakes, 4) → C (space-weather, 4–5) → D/E/F tail → verify-grain sweep. Unlocks the most papers soonest, but starts on the hardest shape.
- Tractability-first (recommended): A (earthquakes — shape already ruled, facts fresh, proves the "complete-catalog dex alongside spine" pattern) → B (WSPR — begin shaping in parallel) → C → D/E/F → verify-grain sweep.
Either way, the repoint flip is the final campaign phase, paper by paper, after the dexes exist.
Per-item open shape questions (for Mike to rule, at build time)
- A / earthquakes: finer tiers (M<3, M<2, M<1) are globally incomplete below ~M4.5 — build them regional (US) where complete, or global-and-null-the-gaps per "data is data"? Register EMSC/GFZ/ISC as separate network dexes or as multi-source slots within a tier?
B / WSPR: the whole shapeSETTLED 2026-07-07 (Mike). Two spot structures, no station Locationdex (every spot self-carries tx/rx lat/lon in the raw; no paper asks a station-centric question). (1)wspr_spot_censusmonth-grain Yeardex — BUILT: one slot per calendar month over the 200 GB raw already on /mnt/ursa; "carry both" — the 5 MB day×band aggregate is the working surface (inlined series), the raw partition is the archive pointer for per-spot re-slicing. Serves the solar-cycle paper (needs the full census). (2) event-window spot Eventdex — PENDING: slot = event window (a tornado outbreak / control day / the aircraft-capture day / the April precursor window), spots inside; inherits the 5 event papers' already-enumerated windows (no outbreak-detector — the papers found the precursor responds to any tornado-producing convection, not outbreak severity). Carved from the same /mnt/ursa raw; no fetching.- C / space-weather: how does a continuous scalar index become a dex (not an event, not a
station)? One "space_weather_index" family with sub-kinds, or a dex per index? Does it extend the
existing
gstspace-weather dex? - D / aircraft: MovingPlatform per aircraft, or per flight-track?
- E / Safecast: its own kind, or a second source under a shared radiation station dex?
- F / AMS: fold into
entry, or standalone AMS fireball eventdex?