Commit Graph

320 Commits

Author SHA1 Message Date
James Smith 753a08234e fix: tracker signature scoring — gate boost/length signals, name-only detects LOW
confidence_boost and the manufacturer-data-length signal applied without
any identifying indicator match, giving every device a phantom AirTag
baseline (a 22+ byte payload from any vendor scored 0.30 and was flagged
as an AirTag). Both now require a matched indicator, mirroring the
score>0 gating already used in _check_generic_tracker_indicators.

Name-pattern weight raised 0.15 -> 0.30 so a device advertising a known
tracker name yields a LOW-confidence detection, consistent with the
TSCM BLE scanner's name-only detection and the engine docstring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 18:43:20 +01:00
James Smith 5cff7de117 refactor: single dependency probe in capability detection; real test coverage
detect_mode_availability accepts a pre-computed dep_status so the agent
probes once; interface and fallback paths now have content-level tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:37:30 +01:00
James Smith 0588055d1f refactor: extract shared capability detection from agent
utils/capabilities.py now owns interface detection and mode
availability; the agent delegates via detect_interfaces() and
detect_mode_availability(). The agent keeps config gating and
tool_details population to preserve its result shape exactly.

The moved fallback path uses utils.dependencies.check_tool instead of
the agent's old shutil.which fallback; check_tool also searches
Homebrew paths, a strict superset (strictly better detection).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 08:31:54 +01:00
James Smith 0af3028151 fix: point doppler and ground-station scheduler at unified TLE store
Both silently fell back to static bundled TLEs after the removal of
routes.satellite._tle_cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:25:29 +01:00
James Smith 5e996654fe refactor: weather sat prediction reads TLEs from unified store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:23:20 +01:00
James Smith 74d5663f73 fix: harden TLE store for cross-process use
- busy_timeout so concurrent app+agent writers wait instead of raising
- seed from _connect() so update-before-first-read can't drop the seed
- regression tests: seed ordering, concurrent writer, default DB path

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 17:07:30 +01:00
James Smith f4a9cb7da6 feat: add unified SQLite-backed TLE store
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:58:09 +01:00
James Smith d4652017f5 fix: stabilize test suite and repair frontend/backend wiring
- meshcore pin >=2.3.0 (EventType.STATS_CORE floor); setup.sh derives
  optional packages from requirements.txt; Python 3.10 warning
- agent-mode wifi clients proxy route + bare-array response handling
- remove dead AIS/ACARS/VDL2 SPA wiring and orphaned partials/CSS
- agent TLE download to data/tle/ (was littering repo root as gp.php)
- gate deferred background init off under pytest (mock-pollution race)
- complete Popen mocks (context manager protocol, communicate tuples)
- real pipe fds in weather-sat decoder tests (fd 10/11 collision caused
  10s SQLite stalls); satellite tests no longer rewrite data/satellites.py
- register 'live' pytest marker, excluded by default
- update stale test assertions to current APIs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 16:42:33 +01:00
James Smith b30d883974 fix: resolve CI test failures and OOM kill in satellite tests
- pyproject.toml: sync missing deps (flask-wtf, flask-compress,
  simple-websocket, gunicorn, gevent, psutil, cryptography, meshcore,
  pre-commit) so test_requirements integrity check passes
- tests/conftest.py: set INTERCEPT_DISABLE_AUTH=1 so auth routes
  return 200 instead of 302 in tests
- routes/bluetooth_v2.py: add device_to_dict() helper that flattens
  heuristics to top level for test_bluetooth_api serialization tests
- utils/bluetooth/heuristics.py: evaluate() now returns the device so
  callers can chain; was returning None
- tests/test_satellite.py: reduce hours 48→2 in pass-prediction test
  to prevent OOM kill on GitHub Actions 7GB runner at the 59% mark

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-20 22:34:02 +01:00
James Smith e6e6cb3b9a fix: always update fingerprint stability; assert tracker fields preserved on skip
- Move fingerprint stability update before early return so it updates even when payload hash matches
- Remove duplicate stability assignment from detect_tracker result block
- Add assertion in test to verify tracker fields are preserved when detection is skipped

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 14:49:51 +01:00
James Smith 260240728a perf: skip tracker signature scan when BLE payload fingerprint is unchanged 2026-05-19 13:14:16 +01:00
James Smith 0e0e17b089 style: document cleanup now-capture and widen test sleep margin 2026-05-19 13:10:44 +01:00
James Smith 646eb09e1d perf: minimize DataStore cleanup lock hold time
Modify DataStore.cleanup() to minimize lock hold duration:
- Snapshot timestamps under lock (brief O(1) list copy)
- Compute expired keys outside lock (no contention during O(n) scan)
- Re-acquire lock only for deletion with re-validation
  (ensures entries refreshed between snapshot and deletion are not deleted)

This reduces blocking of reader threads and prevents latency spikes
during periodic cleanup of large stores (10K+ entries).

Also adds tests:
- test_cleanup_removes_expired_keeps_fresh: basic cleanup behavior
- test_cleanup_does_not_delete_refreshed_entry: re-validation guard

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 12:27:25 +01:00
James Smith 77cdd56641 debug(meshcore): enable BLE debug mode and disconnect before connect
- Enable debug=True on MeshCore.create_ble() to surface verbose logs
- Disconnect any existing BlueZ connection before bleak connects to
  avoid conflicts from prior bluetoothctl/pairing sessions

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 21:15:24 +01:00
James Smith 98e01f4c5b fix(meshcore): surface backend error messages and extend polling window
- Store last status message on MeshcoreClient so error details survive
  beyond the SSE event (which isn't active during connecting state)
- Status endpoint now returns message field so the frontend can show
  the real reason (e.g. 'Connection failed after retries: ...')
- Extend JS polling from 30s to 90s to outlast the backend's 65s
  retry sequence (5+15+45s delays) before declaring timeout

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 20:59:33 +01:00
James Smith bf50cb4acd fix(meshcore): run BLE scan in dedicated thread to avoid gevent conflict
asyncio.run() called from a gevent-patched Flask thread fails under
gunicorn+gevent. Run the one-shot scan in a ThreadPoolExecutor thread
with its own event loop, matching how AsyncWorker handles it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 20:52:44 +01:00
James Smith 7d537998ca fix(meshcore): revert wrong scan_ble_sync route call, fix scan without worker
- Revert route to scan_ble() — scan_ble_sync() lives on AsyncWorker,
  not MeshcoreClient; the 500 was caused by our previous fix
- MeshcoreClient.scan_ble() now runs a one-shot asyncio scan when no
  worker is active, so Scan works before Connect is pressed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 20:49:44 +01:00
James Smith 71eaf1d22a fix(meshcore): fix traceroute logging, asyncio task leaks, stop race, channel sender, and battery cap
- Log node_id hint in request_traceroute instead of silently dropping it
- Replace asyncio.shield/wait_for pattern with _wait_or_stop() to prevent orphan tasks on retry delays
- Poll _stop_event every 1s in _do_connect keep-alive loop to handle stop() race before _asyncio_stop is set
- Extract pubkey_prefix/sender_id in _on_channel_msg instead of hardcoding "unknown"
- Close coroutine and log in _submit() when worker is not running to prevent ResourceWarning
- Cap battery_pct at 100 to prevent values exceeding 100%

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:33:31 +01:00
James Smith ba02f761c6 feat(meshcore): add async worker bridge (utils/meshcore_client.py)
Implements AsyncWorker — the daemon asyncio thread that owns the meshcore
library connection, subscribes to all relevant EventTypes, and feeds events
back into MeshcoreClient via on_message/on_node/on_telemetry/on_traceroute/
on_connected/on_error. Includes retry-with-backoff (3 attempts: 5s/15s/45s),
thread-safe send_text/request_traceroute/scan_ble_sync for Flask callers,
and a standalone _scan_ble() coroutine using bleak.BleakScanner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:18:50 +01:00
James Smith 80bbdb2c09 fix(meshcore): fix thread safety in _set_state/connect, add missing tests
- Lock-protect `get_state` and `_set_state` to prevent data race
  between Flask and asyncio daemon threads
- Atomically check-and-set CONNECTING guard in `connect()` to close
  TOCTOU window between concurrent Flask threads
- Push status events outside the lock in both `_set_state` and
  `connect()` to avoid potential deadlock
- Add TestMeshcoreContact, TestMeshcoreClientStateMachine tests
  covering to_dict keys, queue push on state change, message append
  and 500-item cap (9 -> 13 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:12:11 +01:00
James Smith 6807ee6878 fix(meshcore): fix test isolation, enum value assertions, and os import 2026-05-11 10:39:32 +01:00
James Smith 04d2e1a7bf feat(meshcore): add data model, connection config, MeshcoreClient skeleton
Implements utils/meshcore.py with all dataclasses (MeshcoreMessage,
MeshcoreNode, MeshcoreContact, MeshcoreTelemetry, MeshcoreTraceroute),
connection configs (SerialConfig, TCPConfig, BLEConfig), ConnectionState
enum, serial port discovery, and the MeshcoreClient singleton skeleton.
Adds tests/test_meshcore_client.py covering all dataclasses, availability
check, and state enum (8/8 tests passing).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 10:12:23 +01:00
James Smith 333c5cf8d3 feat(drone): merge Drone Intelligence module
Multi-vector UAV detection mode: Remote ID (WiFi/BLE ASTM F3411),
RTL-SDR 433/868MHz control-link detection, HackRF 2.4/5.8GHz wideband.

Workers feed a shared observation queue; DroneCorrelator merges into
DroneContact objects with TTL store, risk scoring, and SSE streaming.
Frontend: two-panel sidebar + Leaflet map with contact cards and trails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 08:36:27 +01:00
James Smith 3b480eb183 fix(hackrf): resolve 'Tools Missing' on RPi when hackrf_info is present
Two root causes behind HackRF showing as unavailable when tools are installed:

1. get_tool_path() didn't search /usr/local/bin on Linux. HackRF tools built
   from source (as in the Dockerfile) land there, but the path wasn't checked
   when sudo/service environments have a restricted PATH.

2. check_hackrf() only tested hackrf_transfer, but the health check tests
   hackrf_info — both come from the same apt package but a user could have one
   visible and not the other. Now either binary confirms the tools are present.
   hackrf_transfer is still required for actual RX/TX operations.

Fixes #212

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-05 08:36:16 +01:00
James Smith 8632e31c01 fix(drone): resolve critical pipeline, frontend, and input validation issues
Data pipeline (critical): scanners/detectors now write to a separate _obs_queue;
a relay thread reads observations and calls correlator.process(), which emits
processed DroneContact dicts to drone_queue for SSE. Without this the SSE stream
received raw unserializable dataclass objects causing JSON errors.

Frontend (critical):
- Add droneContactList container to drone.html so contact cards render
- Add droneMap container and initialize Leaflet in drone.js init()
- Define dsc-distress-pulse keyframes in drone.css (was referenced but missing)
- Fix SSE reconnect: null _sse before setTimeout to prevent _connectSSE no-op loop

Other fixes:
- Validate rtl_sdr_index with validate_device_index(), return 400 on bad input
- Move _ensure_workers() inside _drone_lock to prevent double-initialization race
- Add double-call guard to RemoteIDScanner.start()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 21:47:12 +01:00
James Smith 59713ffc22 fix(drone): harden RFDetector threading, subprocess lifecycle, and frequency accuracy
- Replace _running bool with threading.Event for correct cross-thread visibility
- Add _proc_lock to guard _rtl_proc/_hackrf_proc across worker/main threads
- Use register_process + safe_terminate (pipe close + SIGKILL fallback on timeout)
- Compute HackRF frequency as band midpoint (hz_low+hz_high)//2, not hz_low
- Guard start() for idempotency — double-call no longer leaks threads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 17:33:55 +01:00
James Smith e8b94b6efc feat(drone): add RFDetector for rtl_433 and hackrf_sweep control-link detection
Implements RFDetector class that wraps rtl_433 (433/868MHz) and hackrf_sweep
(2.4/5.8GHz) subprocesses, emitting RFObservation objects onto a shared queue.
Includes signature matching, frequency band validation, and power thresholding.

- _handle_rtl433_line(): Parse JSON output, filter drone bands, emit observations
- _handle_hackrf_line(): Parse CSV output, average power levels, threshold at -90dBm
- start()/stop(): Manage subprocess threads for concurrent RF detection
- Graceful handling of missing tools (rtl_433, hackrf_sweep)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:52:03 +01:00
James Smith 5dda961dbb fix(drone): assign self._sniffer only after successful AsyncSniffer.start()
Prevents a non-running sniffer object being stored when start() raises
(e.g. permission denied or interface not found).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:42:42 +01:00
James Smith a6ce5d5426 feat(drone): add RemoteIDScanner with BLE/WiFi ASTM F3411 parsing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:39:06 +01:00
James Smith 772b5d0973 feat(drone): add DroneCorrelator with TTL store and risk scoring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 12:11:04 +01:00
James Smith b707468cb6 feat(drone): add data models and RF signature table 2026-05-03 11:45:59 +01:00
James Smith 58222b3474 fix(hackrf): resolve 'Tools Missing' on RPi when hackrf_info is present
Two root causes behind HackRF showing as unavailable when tools are installed:

1. get_tool_path() didn't search /usr/local/bin on Linux. HackRF tools built
   from source (as in the Dockerfile) land there, but the path wasn't checked
   when sudo/service environments have a restricted PATH.

2. check_hackrf() only tested hackrf_transfer, but the health check tests
   hackrf_info — both come from the same apt package but a user could have one
   visible and not the other. Now either binary confirms the tools are present.
   hackrf_transfer is still required for actual RX/TX operations.

Fixes #212

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 09:24:22 +01:00
Mitch Ross 4a149525bd Merge main into misc-fixes and address PR #202 review
Sync with upstream main and fix required items from review:

- updateTimelineLabels() now uses InterceptTime API (getTimezone/getIANA)
  instead of the stale selectedTimezone/TZ_MAP globals that were removed
  during the earlier InterceptTime refactor — fixes ReferenceError on TZ
  change and pass refresh.

- Remove profiles: [basic] from the intercept service in
  docker-compose.yml so bare `docker compose up -d` still starts the
  main service. Profile-gated services (intercept-history, adsb_db)
  stay as-is.
2026-04-24 16:34:09 -04:00
James Smith a982ff5885 fix(wefax): include station JSON in Docker image and handle missing file
data/*.json was excluded by .dockerignore, so wefax_stations.json was
never copied into the container image. The volume mounts in docker-compose
only cover subdirectories (weather_sat, adsb, etc.), leaving the stations
file inaccessible at runtime — causing the /wefax/stations route to 500
and the station/frequency dropdowns to appear empty.

Also adds a graceful file-existence check in load_stations() so a missing
file logs a warning and returns an empty list instead of an unhandled
FileNotFoundError.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-21 20:51:24 +01:00
James Smith f12f4145ef docs(alerts): document webhook/notification system in USAGE.md
fix(db): use gevent-safe local storage for DB connections under gunicorn+gevent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:45:58 +01:00
Smittix b01598753d fix(adsb): disable bias-T on stop and warn when toggled while running (#207)
* fix(adsb): disable bias-T on stop and warn when toggled while running

The RTL-SDR bias-T hardware register persists after the device is closed,
so toggling bias-T off in the UI and stopping the SDR had no effect on the
actual hardware — verified with a multimeter in issue #205.

- Add disable_bias_t_via_rtl_biast() to rtlsdr.py (mirrors enable, uses -b 0)
- Track adsb_bias_t_active in adsb.py; call disable on stop_adsb() so the
  hardware register is cleared when ADS-B is stopped
- Show an inline warning in the UI when the bias-T checkbox is toggled while
  any SDR mode is active, since the setting only takes effect at start time

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(lint): remove unused imports in tscm sweep.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 13:18:50 +01:00
James Smith 0210791c69 feat(export): AIS UDP NMEA forward and JSON export endpoints for AIS/ADS-B
AIS:
- New optional NMEA UDP forwarding via AIS-catcher's -u flag, configurable
  from the AIS sidebar (host + port). Lets OpenCPN and other NMEA tools
  receive live vessel data directly. All SDR builders updated.
- New GET /ais/vessels endpoint — clean JSON snapshot of tracked vessels
  for REST integration

ADS-B:
- New GET /adsb/aircraft endpoint — JSON snapshot of all tracked aircraft,
  with optional ?icao= and ?military=true filters. Response includes a
  reminder that port 30003 (SBS) is already available for tools like
  Virtual Radar Server and OpenCPN's AIS/target plugin.

Closes #90
2026-04-05 16:20:10 +01:00
James Smith 592e97719b feat(gain): normalize gain controls across modes
- Pager and sensor gain inputs changed from unvalidated text fields to
  number inputs with min/max/step constraints
- ADS-B dashboard now exposes a gain input in the tracking strip;
  previously gain was hardcoded to 40 dB with no user control
- validate_gain() ceiling raised from 50 to 102 dB to support HackRF
  (LNA 40 + VGA 62 = 102 dB combined) and LimeSDR (73 dB)
- sdrCapabilities gain_max values corrected: HackRF 62→102, Airspy 21→45
- onSDRTypeChanged() now propagates gain_max to all mode gain inputs so
  HTML constraints match the selected SDR's actual range

Closes #162
2026-04-05 16:02:03 +01:00
mitchross 43fb735e4e Fix Meteor LRPT decoding in Docker and enhance weather satellite UI
Docker fixes:
- Add missing COPY for /usr/local/share/ (pipeline definitions were never
  reaching the runtime image — root cause of silent SatDump failures)
- Add libfftw3-double3 and libfftw3-single3 runtime dependencies
- Handle arm64 vs x86 install path differences (/usr vs /usr/local)
- Split SatDump compile and staging into separate layers for better caching
- Add build-time assertions to catch missing pipelines early

UI enhancements:
- Timezone selector (UTC, Local, Eastern, Central, Mountain, Pacific)
  with localStorage persistence — all time displays update instantly
- Pass analysis bar showing 24h quality breakdown and best upcoming pass
- Enhanced pass cards with cardinal direction (NW→SE), BEST badge
- Console timestamps, log level filters (ALL/SIGNAL/PROG/ERR), COPY/CLR
- Pass count in stats strip
- Demo data mode for UI testing without SDR or live satellite pass
- Meteor M2-4 80k baud fallback pipeline option

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 00:05:31 -04:00
James Smith af2ab567ca Persist aircraft DB under data/adsb/ for Docker volume compatibility
Move aircraft_db.json and aircraft_db_meta.json from the project root
to data/adsb/ so they survive container restarts and rebuilds. Add
matching volume mount to both Docker Compose profiles.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 16:56:34 +00:00
James Smith 89c7c2fb07 Fix 5GHz WiFi scanning failures in deep scan and band detection
- Fix deep scan with 'All bands' never scanning 5GHz: band='all' now
  correctly passes --band abg to airodump-ng (previously no flag was
  added, causing airodump-ng to default to 2.4GHz-only)
- Fix APs first seen without channel info permanently stuck at
  band='unknown': _update_access_point now backfills channel, frequency,
  and band when a subsequent observation resolves the channel
- Fix legacy /wifi/scan/start combining mutually exclusive --band and -c
  flags: --band is now only added when no explicit channel list is given,
  and the interface is always placed as the last argument

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 21:50:47 +00:00
James Smith 34e1d25069 Fix ruff lint errors to unblock CI (import sorting, unused imports, style) 2026-03-20 13:51:30 +00:00
James Smith f05a5197cd Fix satellite target switching regression 2026-03-19 14:55:48 +00:00
James Smith 18b442eb21 Fix dashboard startup regressions and mode utilities 2026-03-19 10:37:21 +00:00
James Smith 62ee2252a3 Fix satellite dashboard refresh flows 2026-03-18 22:53:36 +00:00
James Smith 6fd5098b89 Clear stale telemetry and add transmitter fallbacks 2026-03-18 22:43:40 +00:00
James Smith 985c8a155a Harden satellite dashboard telemetry loading 2026-03-18 22:35:31 +00:00
James Smith d0402f4746 Refresh weather decoder docs for Meteor flow 2026-03-18 22:30:54 +00:00
James Smith 6dc0936d6d Align Meteor tracking defaults 2026-03-18 22:28:04 +00:00
James Smith 4cf394f92e Persist Meteor decode job state 2026-03-18 22:20:24 +00:00