153 Commits

Author SHA1 Message Date
James Smith a3c509aa94 feat: add HTML scaffolding for pager directory and sensor dashboard views
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-21 12:39:57 +01:00
James Smith b30d883974 fix: resolve CI test failures and OOM kill in satellite tests
- pyproject.toml: sync missing deps (flask-wtf, flask-compress,
  simple-websocket, gunicorn, gevent, psutil, cryptography, meshcore,
  pre-commit) so test_requirements integrity check passes
- tests/conftest.py: set INTERCEPT_DISABLE_AUTH=1 so auth routes
  return 200 instead of 302 in tests
- routes/bluetooth_v2.py: add device_to_dict() helper that flattens
  heuristics to top level for test_bluetooth_api serialization tests
- utils/bluetooth/heuristics.py: evaluate() now returns the device so
  callers can chain; was returning None
- tests/test_satellite.py: reduce hours 48→2 in pass-prediction test
  to prevent OOM kill on GitHub Actions 7GB runner at the 59% mark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 22:34:02 +01:00
James Smith dbe2003d75 fix: guard _looked_up_icaos popitem against concurrent clear(); add eviction tests
contextlib.suppress(KeyError) around popitem prevents a crash in the SBS
parser thread if stop_adsb() calls clear() concurrently between the len()
check and the popitem call.

Two unit tests verify FIFO eviction semantics and duplicate-key no-op.
2026-05-19 17:47:21 +01:00
James Smith e6e6cb3b9a fix: always update fingerprint stability; assert tracker fields preserved on skip
- Move fingerprint stability update before early return so it updates even when payload hash matches
- Remove duplicate stability assignment from detect_tracker result block
- Add assertion in test to verify tracker fields are preserved when detection is skipped

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 14:49:51 +01:00
James Smith 260240728a perf: skip tracker signature scan when BLE payload fingerprint is unchanged 2026-05-19 13:14:16 +01:00
James Smith 0e0e17b089 style: document cleanup now-capture and widen test sleep margin 2026-05-19 13:10:44 +01:00
James Smith efc14b4de0 test: use threading to correctly exercise cleanup re-validation guard 2026-05-19 13:08:32 +01:00
James Smith 6ed24b758d test: fix cleanup re-validation test to exercise actual cleanup()
Replace manual reimplementation of snapshot/delete logic with actual
store.cleanup() call. Uses mocked time.time to simulate the scenario
where entries refreshed between snapshot and deletion survive due to
re-validation guard.

Fixes: test was passing without actually calling the subject under test
2026-05-19 12:36:14 +01:00
James Smith 646eb09e1d perf: minimize DataStore cleanup lock hold time
Modify DataStore.cleanup() to minimize lock hold duration:
- Snapshot timestamps under lock (brief O(1) list copy)
- Compute expired keys outside lock (no contention during O(n) scan)
- Re-acquire lock only for deletion with re-validation
  (ensures entries refreshed between snapshot and deletion are not deleted)

This reduces blocking of reader threads and prevents latency spikes
during periodic cleanup of large stores (10K+ entries).

Also adds tests:
- test_cleanup_removes_expired_keeps_fresh: basic cleanup behavior
- test_cleanup_does_not_delete_refreshed_entry: re-validation guard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 12:27:25 +01:00
James Smith e5c5afb158 test(meshcore): assert error message propagated in on_error test
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:54:30 +01:00
James Smith f2af2ad0b6 test(meshcore): fix BLE docker test and add library boundary isolation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:51:45 +01:00
James Smith 3f6f8a5695 test(meshcore): add integration tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:49:05 +01:00
James Smith 0d9bb53722 test(meshcore): strengthen connect and send boundary tests
Add 237-char boundary test proving the send limit accepts exactly 237
characters, and upgrade connect tests to assert the correct config
dataclass type and field values are passed to connect().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:47:10 +01:00
James Smith d84cd41896 test(meshcore): add missing coverage for 7 endpoints + SSE keepalive
Adds 16 new tests covering POST /disconnect, GET /ble/scan, GET /stream
(keepalive and event data), GET /messages, GET /nodes, GET /contacts,
GET /telemetry/<node_id>, and GET /repeaters, bringing total from 17 to 33.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:44:11 +01:00
James Smith ed4b6ef897 test(meshcore): add route tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:41:49 +01:00
James Smith 80bbdb2c09 fix(meshcore): fix thread safety in _set_state/connect, add missing tests
- Lock-protect `get_state` and `_set_state` to prevent data race
  between Flask and asyncio daemon threads
- Atomically check-and-set CONNECTING guard in `connect()` to close
  TOCTOU window between concurrent Flask threads
- Push status events outside the lock in both `_set_state` and
  `connect()` to avoid potential deadlock
- Add TestMeshcoreContact, TestMeshcoreClientStateMachine tests
  covering to_dict keys, queue push on state change, message append
  and 500-item cap (9 -> 13 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:12:11 +01:00
James Smith 6807ee6878 fix(meshcore): fix test isolation, enum value assertions, and os import 2026-05-11 10:39:32 +01:00
James Smith 04d2e1a7bf feat(meshcore): add data model, connection config, MeshcoreClient skeleton
Implements utils/meshcore.py with all dataclasses (MeshcoreMessage,
MeshcoreNode, MeshcoreContact, MeshcoreTelemetry, MeshcoreTraceroute),
connection configs (SerialConfig, TCPConfig, BLEConfig), ConnectionState
enum, serial port discovery, and the MeshcoreClient singleton skeleton.
Adds tests/test_meshcore_client.py covering all dataclasses, availability
check, and state enum (8/8 tests passing).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 10:12:23 +01:00
James Smith f9e8fa896d feat(drone): add Flask blueprint, register routes, wire drone_queue
Implements Task 5: creates routes/drone.py with /status, /contacts,
/start, /stop, and /stream (SSE fanout) endpoints; registers the
drone_bp blueprint in routes/__init__.py; adds drone_queue to app.py;
adds opendroneid>=1.0 to requirements.txt. All 39 drone tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:37:02 +01:00
James Smith 59713ffc22 fix(drone): harden RFDetector threading, subprocess lifecycle, and frequency accuracy
- Replace _running bool with threading.Event for correct cross-thread visibility
- Add _proc_lock to guard _rtl_proc/_hackrf_proc across worker/main threads
- Use register_process + safe_terminate (pipe close + SIGKILL fallback on timeout)
- Compute HackRF frequency as band midpoint (hz_low+hz_high)//2, not hz_low
- Guard start() for idempotency — double-call no longer leaks threads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:33:55 +01:00
James Smith 681a498461 test(drone): fix test_start_stop isolation and add out-of-band filter coverage
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:53:41 +01:00
James Smith e8b94b6efc feat(drone): add RFDetector for rtl_433 and hackrf_sweep control-link detection
Implements RFDetector class that wraps rtl_433 (433/868MHz) and hackrf_sweep
(2.4/5.8GHz) subprocesses, emitting RFObservation objects onto a shared queue.
Includes signature matching, frequency band validation, and power thresholding.

- _handle_rtl433_line(): Parse JSON output, filter drone bands, emit observations
- _handle_hackrf_line(): Parse CSV output, average power levels, threshold at -90dBm
- start()/stop(): Manage subprocess threads for concurrent RF detection
- Graceful handling of missing tools (rtl_433, hackrf_sweep)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:52:03 +01:00
James Smith a6ce5d5426 feat(drone): add RemoteIDScanner with BLE/WiFi ASTM F3411 parsing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:39:06 +01:00
James Smith 772b5d0973 feat(drone): add DroneCorrelator with TTL store and risk scoring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:11:04 +01:00
James Smith b707468cb6 feat(drone): add data models and RF signature table 2026-05-03 11:45:59 +01:00
James Smith 3693b02cb9 refactor(radiosonde): use MapUtils.init + Settings tile layer on sonde map
Replace hardcoded L.map() + CartoCD dark tile layer with MapUtils.init()
and add tactical overlays. Adds test verifying the cartocdn URL is gone.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 23:27:36 +01:00
James Smith 24f12b1220 refactor(aprs,gps): use MapUtils.init + HUD panels on APRS and GPS maps
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 23:12:46 +01:00
James Smith 7dfefb48e6 refactor(satellite): use MapUtils.init + HUD on ground track map
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:56:20 +01:00
James Smith 12af6e250e refactor(ais): use MapUtils.init + HUD panels on vessel map
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:41:26 +01:00
James Smith 5ffd9e5fb3 refactor(adsb): use MapUtils.init + tactical overlays on radar map
Replace custom createFallbackGridLayer/upgradeRadarTilesFromSettings with
MapUtils.init(), add range ring + reticle + HUD panel overlays via
MapUtils.addTacticalOverlays(), and wire updateCount/updateReticle into
the SSE aircraft handler and drawRangeRings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:35:43 +01:00
James Smith 99edea33e3 feat(maps): add MapUtils shared map initialisation utility
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:19:44 +01:00
James Smith f97782724e fix(maps): exclude stadia key from localStorage cache
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:11:34 +01:00
James Smith e4df3eaecb feat(maps): add Stadia dark + tactical tile providers with API key support
- Add offline.stadia_key to OFFLINE_DEFAULTS in routes/offline.py
- Add stadia_dark and tactical tile providers to Settings.tileProviders
- Update getTileConfig() to inject Stadia API key or fall back to CartoDB dark
- Add setStadiaKey() method for saving and applying the API key
- Show/hide Stadia key row in setTileProvider() and _updateUI()
- Add Stadia options to tile provider select in settings modal
- Add Stadia API key input row to settings modal
- Add TDD tests for stadia_key backend

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:04:07 +01:00
James Smith 16b95e4804 test: add system group to nav state coverage 2026-04-13 21:10:14 +01:00
James Smith f1a029262b feat: persist nav group open/closed state to localStorage
Adds initNavGroupState() and saveNavGroupState() functions so the
open/closed state of each .mode-nav-dropdown survives page reloads.
Active groups are never force-closed even if localStorage says closed.
Adds test_nav_state.py with two tests verifying presence of the
functions and data-group attributes on all five nav groups.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 21:08:27 +01:00
James Smith 34e1d25069 Fix ruff lint errors to unblock CI (import sorting, unused imports, style) 2026-03-20 13:51:30 +00:00
James Smith d240ae06e3 fix(satellite): populate currentPos with full telemetry in pass predictions
Previously currentPos only had lat/lon, so the updateTelemetry fallback
(used before first live position arrives) always showed '---' for
altitude/elevation/azimuth/distance. currentPos now includes all fields
computed from the request observer location. updateTelemetry simplified
to delegate to applyTelemetryPosition.
2026-03-19 21:48:33 +00:00
James Smith d84237dbb4 feat(satellite): add 24-hour periodic TLE auto-refresh
TLE data was only refreshed once at startup. After each refresh, a new
24-hour timer is now scheduled in a finally block so it fires even on
refresh failure. threading moved to module-level import.
2026-03-19 21:47:38 +00:00
James Smith d20808fb35 fix(satellite): strip observer-relative fields from SSE tracker
SSE runs server-wide with DEFAULT_LAT/LON defaults of 0,0. Emitting
elevation/azimuth/distance/visible from the tracker produced wrong
values (always visible:False) that overwrote correct data from the
per-client HTTP poll every second.

The HTTP poll (/satellite/position) owns all observer-relative data.
SSE now only emits lat/lon/altitude/groundTrack. Also removes the
unused DEFAULT_LATITUDE/DEFAULT_LONGITUDE import.
2026-03-19 21:45:48 +00:00
James Smith cbe7f591e3 Fix weather sat test failures: login auth, timer refs, and SDR device mocking
- Add authenticated client fixture to test_weather_sat_routes.py so
  require_login() before_request doesn't redirect test clients to /login
- Save timer mock references before disable()/skip_pass() clear _timer = None
- Patch app.claim_sdr_device to return None in execute_capture and
  scheduling cycle tests to avoid real USB hardware probing in CI

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 20:56:01 +00:00
James Smith 4607c358ed Add ground station automation with 6-phase implementation
Phase 1 - Automated observation engine:
- utils/ground_station/scheduler.py: GroundStationScheduler fires at AOS/LOS,
  claims SDR, manages IQBus lifecycle, emits SSE events
- utils/ground_station/observation_profile.py: ObservationProfile dataclass + DB CRUD
- routes/ground_station.py: REST API for profiles, scheduler, observations, recordings,
  rotator; SSE stream; /ws/satellite_waterfall WebSocket
- DB tables: observation_profiles, ground_station_observations, ground_station_events,
  sigmf_recordings (added to utils/database.py init_db)
- app.py: ground_station_queue, WebSocket init, scheduler startup in _deferred_init
- routes/__init__.py: register ground_station_bp

Phase 2 - Doppler correction:
- utils/doppler.py: generalized DopplerTracker extracted from sstv_decoder.py;
  accepts satellite name or raw TLE tuple; thread-safe; update_tle() method
- utils/sstv/sstv_decoder.py: replace inline DopplerTracker with import from utils.doppler
- Scheduler runs 5s retune loop; calls rotator.point_to() if enabled

Phase 3 - IQ recording (SigMF):
- utils/sigmf.py: SigMFWriter writes .sigmf-data + .sigmf-meta; disk-free guard (500MB)
- utils/ground_station/consumers/sigmf_writer.py: SigMFConsumer wraps SigMFWriter

Phase 4 - Multi-decoder IQ broadcast pipeline:
- utils/ground_station/iq_bus.py: IQBus single-producer fan-out; IQConsumer Protocol
- utils/ground_station/consumers/waterfall.py: CU8→FFT→binary frames
- utils/ground_station/consumers/fm_demod.py: CU8→FM demod (numpy)→decoder subprocess
- utils/ground_station/consumers/gr_satellites.py: CU8→cf32→gr_satellites (optional)

Phase 5 - Live spectrum waterfall:
- static/js/modes/ground_station_waterfall.js: /ws/satellite_waterfall canvas renderer
- Waterfall panel in satellite dashboard sidebar, auto-shown on iq_bus_started SSE event

Phase 6 - Antenna rotator control (optional):
- utils/rotator.py: RotatorController TCP client for rotctld (Hamlib line protocol)
- Rotator panel in satellite dashboard; silently disabled if rotctld unreachable

Also fixes pre-existing test_weather_sat_predict.py breakage:
- utils/weather_sat_predict.py: rewritten with self-contained skyfield implementation
  using find_discrete (matching what committed tests expected); adds _format_utc_iso
- tests/test_weather_sat_predict.py: add _MOCK_WEATHER_SATS and @patch decorators
  for tests that assumed NOAA-18 active (decommissioned Jun 2025, now active=False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 17:36:55 +00:00
Smittix b5115d4aa1 Fix observer location persistence and APRS defaults 2026-03-15 17:49:46 +00:00
Smittix e00fbfddc1 v2.26.0: fix SSE fanout crash and branded logo FOUC
- Fix SSE fanout thread AttributeError when source queue is None during
  interpreter shutdown by snapshotting to local variable with null guard
- Fix branded "i" logo rendering oversized on first page load (FOUC) by
  adding inline width/height to SVG elements across 10 templates
- Bump version to 2.26.0 in config.py, pyproject.toml, and CHANGELOG.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 11:51:27 +00:00
Smittix 90281b1535 fix(modes): deep-linked mode scripts fail when body not yet parsed
ensureModeScript() used document.body.appendChild() to load lazy mode
scripts, but the preload for ?mode= query params runs in <head> before
<body> exists, causing all deep-linked modes to silently fail.

Also fix cross-mode handoffs (BT→BT Locate, WiFi→WiFi Locate,
Spy Stations→Waterfall) that assumed target module was already loaded.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 20:49:08 +00:00
thatsatechnique 0afa15bb16 Merge remote-tracking branch 'upstream/main' 2026-03-06 12:47:51 -08:00
thatsatechnique 7b4ad20805 fix(ook): address upstream PR review — SDR tracking, validation, cleanup, XSS
Critical:
- Pass sdr_type_str to claim/release_sdr_device (was missing 3rd arg)
- Add ook_active_sdr_type module-level var for proper device registry tracking
- Add server-side range validation on all timing params via validate_positive_int

Major:
- Extract cleanup_ook() function for full teardown (stop_event, pipes, process,
  SDR release) — called from both stop_ook() and kill_all()
- Replace Popen monkey-patching with module-level _ook_stop_event/_ook_parser_thread
- Fix XSS: define local _esc() fallback in ook.js, never use raw innerHTML
- Remove dead inversion code path in utils/ook.py (bytes.fromhex on same
  string that already failed decode — could never produce a result)

Minor:
- Status event key 'status' → 'text' for consistency with other modules
- Parser thread logging: debug → warning for missing code field and errors
- Parser thread emits status:stopped on exit (normal EOF or crash)
- Add cache-busting ?v={{ version }}&r=ook1 to ook.js script include
- Fix gain/ppm comparison: != '0' (string) → != 0 (number)

Tests: 22 → 33 (added start success, stop with process, SSE stream,
timing range validation, stopped-on-exit event)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 16:32:31 -08:00
Smittix 1403d49049 fix: restore HackRF One/Pro detection when PATH is restricted 2026-03-05 09:31:21 +00:00
thatsatechnique 9090b415cc Merge remote-tracking branch 'upstream/main' 2026-03-04 14:54:56 -08:00
thatsatechnique 18db66bce3 fix(ook): harden for upstream review — tests, cleanup, CSS extraction
- Add kill_all() handler for OOK process cleanup on global reset
- Fix stop_ook() to close pipes and join parser thread (prevents hangs)
- Add ook.css with CSS classes, replace inline styles in ook.html
- Register ook.css in lazy-load style map (INTERCEPT_MODE_STYLE_MAP)
- Fix frontend frequency min=24 to match backend validation
- Add 22 unit tests for decode_ook_frame, ook_parser_thread, and routes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 14:52:32 -08:00
Smittix 10077eee60 fix: HackRF One support — detection, ADS-B, waterfall, and error handling
- Parse hackrf_info stderr (newer firmware) and handle non-zero exit codes
- Fix gain_max from 62 to 102 (combined LNA 40 + VGA 62)
- Apply resolved readsb binary path for all SDR types, not just RTL-SDR
- Add HackRF/SoapySDR-specific error messages in ADS-B startup
- Add HackRF waterfall support via rx_sdr IQ capture + FFT
- Add 17 tests for HackRF detection and command builder

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 22:31:51 +00:00