Commit Graph

932 Commits

Author SHA1 Message Date
Smittix
7c3ec9e920 chore: commit all changes and remove large IQ captures from tracking
Add .gitignore entry for data/subghz/captures/ to prevent large
IQ recording files from being committed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-12 23:30:37 +00:00
Smittix
4639146f05 fix: Remove incomplete MLAT feature causing ImportError on startup
The partially-added MLAT support was out of sync between config and
routes, causing an ImportError when importing adsb_bp. Remove all MLAT
additions from config, template UI/JS, and docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 20:55:21 +00:00
Smittix
a354fee792 fix: Resolve listening post audio stuttering introduced in v2.15.0
Throttle audio waterfall rendering (50ms→200ms), eliminate per-frame
Array.from() allocation, drain stale pipe buffer before streaming,
increase chunk size to 8192, and remove debug logging from animation
hot paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 20:24:51 +00:00
Smittix
a1cb6b2692 feat: Add SatDump to setup.sh for local (non-Docker) installs
Weather satellite decoding (NOAA APT & Meteor LRPT) was added in the
Dockerfile but setup.sh had no SatDump support, leaving local installs
with a broken weather satellite mode. Adds build-from-source functions
for both Debian and macOS, a check_optional entry, and prompted install
steps in both platform installers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-10 09:15:53 +00:00
Smittix
8376415074 feat: Add weather satellite decoder (NOAA APT & Meteor LRPT)
feat: Add weather satellite decoder (NOAA APT & Meteor LRPT)   -  Alpha
2026-02-10 08:36:34 +00:00
Mitch Ross
b25615317b Merge upstream/main: sync fork with latest DMR fixes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:40:25 -05:00
Mitch Ross
311d268b10 Explicitly remove libgtk-3-dev in Dockerfile cleanup step
Adds libgtk-3-dev to the apt-get remove list so it doesn't remain
in the final image. Runtime GTK libs stay for slowrx.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:09:50 -05:00
Mitch Ross
6581620cb0 Merge pull request #2 from mitchross/copilot/add-test-coverage-weather-satellite
[WIP] Add test coverage for weather satellite decoder modules
2026-02-09 16:58:09 -05:00
copilot-swe-agent[bot]
aa963519e9 Fix str(e) in error responses, remove location modal, document GTK dependency
Co-authored-by: mitchross <6330506+mitchross@users.noreply.github.com>
2026-02-09 21:55:30 +00:00
copilot-swe-agent[bot]
4a6dddbb48 Add comprehensive test coverage for weather satellite modules
- Created test_weather_sat_routes.py with 42 tests for all endpoints
- Created test_weather_sat_decoder.py with 47 tests for WeatherSatDecoder class
- Created test_weather_sat_predict.py with 14 tests for pass prediction
- Created test_weather_sat_scheduler.py with 31 tests for auto-scheduler
- Total: 134 test functions across 14 test classes
- All tests follow existing patterns (mocking, fixtures, docstrings)
- Tests cover happy paths, error handling, and edge cases
- Mock all external subprocess calls and HTTP requests

Co-authored-by: mitchross <6330506+mitchross@users.noreply.github.com>
2026-02-09 21:50:22 +00:00
copilot-swe-agent[bot]
f217230ef4 Initial plan 2026-02-09 21:41:46 +00:00
Mitch Ross
e27b4d78cb Merge pull request #1 from mitchross/copilot/fix-security-issues
Address code review feedback for weather satellite decoder
2026-02-09 16:09:39 -05:00
copilot-swe-agent[bot]
d41ba61aee Fix security issues, breaking changes, and code cleanup for weather satellite PR
Co-authored-by: mitchross <6330506+mitchross@users.noreply.github.com>
2026-02-09 20:58:26 +00:00
copilot-swe-agent[bot]
35cf01c11e Initial plan 2026-02-09 20:52:52 +00:00
Smittix
00c9a6fdd9 Fix DMR audio/text deadlock: start ffmpeg per-client, not at launch
Starting ffmpeg at decoder launch caused a pipe-buffer deadlock: ffmpeg
stdout filled up (~64KB on Linux) before the browser connected to the
audio stream, back-pressuring the entire pipeline and freezing dsd-fme
stderr output (no text data, no syncs, no calls).

New architecture: a mux thread always drains dsd-fme stdout to keep the
pipeline flowing. ffmpeg starts lazily per-client when /dmr/audio/stream
is requested (matching the listening post pattern). The mux forwards
decoded audio to the active ffmpeg with silence fill during voice gaps,
and discards audio when no client is connected.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 19:25:07 +00:00
Smittix
fce66a6a60 Fix DMR audio stream failing with "no supported source found"
Digital voice is intermittent — dsd-fme only outputs PCM during active
voice transmissions. Without input, ffmpeg never wrote the WAV header
and the browser got an empty response. Add an audio bridge thread that
feeds 100ms silence chunks during voice gaps so ffmpeg always has input
and the browser receives a continuous WAV stream. Add auto-reconnect
on the frontend if the audio stream drops while the decoder is running.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 18:14:33 +00:00
Smittix
b023e4cdc7 Add DMR audio output, frequency persistence, and bookmarks
Stream decoded digital voice audio to the browser via ffmpeg pipeline
(dsd-fme 8kHz PCM → ffmpeg → 44.1kHz WAV → chunked HTTP). Persist
frequency/protocol/gain/ppm settings in localStorage so they survive
page navigation. Add bookmark system for saving and recalling frequencies.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 18:05:27 +00:00
Smittix
a8f2912b90 Fix waterfall-to-listen SDR busy race condition
Wait for server-side WebSocket stop confirmation before closing the
connection, ensuring the IQ process is fully terminated and the USB
device released. Add retry logic with back-off in the audio start
endpoint as defense-in-depth for any remaining timing gaps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 13:58:42 +00:00
Smittix
a2a7ac8fec Fix banner filter eating dsd-fme data lines and add event log capture
The box-drawing character filter was dropping ANY line containing │ or ─,
including dsd-fme data lines that use these as column separators (e.g.
"DMR BS │ Slot 1 │ TG: 12345 │ SRC: 67890"). Now only filters lines
that are purely decorative (no alphanumeric content).

Also adds -J /dev/stderr so dsd-fme writes its event log to stderr
where we capture it, and debug logging of raw stderr lines.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 11:41:32 +00:00
Smittix
4e168ff502 Fix dsd-fme DMR flag (-fd is D-STAR, not DMR) and audio output
-fd means D-STAR in dsd-fme, not DMR — causing sync detection
(shared C4FM modulation) but no decoded data. DMR Simplex is -fs.
Also fix -o - (invalid in dsd-fme) to -o null for headless servers,
add D-STAR flag mapping, and handle TGT/SRC output format in parser.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 10:31:44 +00:00
Smittix
51aba87852 Bump version to 2.15.0
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
v2.15.0
2026-02-09 09:22:41 +00:00
Smittix
4c13e98091 Fix dsd-fme protocol flags, device label, and add tuning controls
dsd-fme remapped several flags from classic DSD: -fp is ProVoice (not
P25), -fi is NXDN48 (not D-Star), -fv doesn't exist. This caused P25
to trigger ProVoice decoding and D-Star to trigger NXDN48. Corrected
flag table and added C4FM modulation hints for better sync reliability.

Also fixes: device panel showing "DMR" regardless of protocol, signal
activity status flip-flopping between LISTENING and IDLE, and rtl_fm
squelch chopping the bitstream mid-frame. Adds PPM correction and
relax CRC controls for fine-tuning on marginal signals.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-09 08:44:23 +00:00
Mitch Ross
54c849ab60 Fix weather satellite decoder security, architecture, and race conditions
Security: replace path traversal-vulnerable str().startswith() with
is_relative_to(), anchor path checks to app root, strip filesystem
paths from error responses, add decoder-level path validation.

Architecture: use safe_terminate/register_process for subprocess
lifecycle, replace custom SSE generator with sse_stream(), use
centralized validate_* functions, remove unused app.py declarations.

Bugs: add thread-safe singleton locks, protect _images list across
threads, move blocking process.wait() to async daemon thread, fix
timezone handling for tz-aware datetimes, use full path for image
deduplication, guard TLE auto-refresh during tests, validate
scheduler parameters to avoid 500 errors.

Docker: pin SatDump to v1.2.2 and slowrx to ca6d7012, document
INTERCEPT_IMAGE fallback pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:29:45 -05:00
Mitch Ross
94ee22fdd4 Merge upstream/main: sync fork with conflict resolution
Resolve conflicts keeping local GSM tools in kill_all() process list
and weather satellite config settings while merging upstream changes
including GSM spy removal, DMR fixes, USB device probe, APRS crash
fix, and cross-module frequency routing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:06:41 -05:00
Smittix
b96eb8ccba Fix DMR frontend/backend state desync causing 409 on start
When the backend has an active DMR session but the frontend lost track
(page refresh, broken flags causing silent running), clicking Start
returned 409 with no recovery path. Now the frontend resyncs on
"Already running" responses and checks backend status on tab activation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:20:02 +00:00
Smittix
b8a80460bf Fix digital voice decoder producing no output due to wrong dsd-fme flags
The _DSD_FME_PROTOCOL_FLAGS dictionary had every protocol flag wrong,
causing dsd-fme (the preferred binary) to receive invalid or mismatched
-f flags. Also fix orphaned process leak on startup failure and add
centralized input validation for frequency/gain/device.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 23:12:54 +00:00
Smittix
7130c2d4c4 Add cross-module frequency routing from Listening Post to decoders
Enable sending discovered frequencies from the Listening Post scanner,
signal identification panel, and waterfall display directly to Pager,
433 Sensor, or RTLAMR decoder modes with one click.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:45:47 +00:00
Smittix
62c34c1e95 Fix settings modal overflowing viewport on smaller screens
Constrain modal height to viewport and make tab content scrollable
so the modal no longer falls off the bottom of the screen.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:25:58 +00:00
Smittix
e413f54651 Add USB-level device probe to prevent cryptic rtl_fm crashes
When an external process (or stale handle from a crash) holds an SDR
device, claim_sdr_device() registry check passes but rtl_fm fails with
usb_claim_interface error -6. This adds a quick rtl_test probe inside
claim_sdr_device() so all modes get a clear error message before the
decoder pipeline is launched.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:24:02 +00:00
Smittix
1a4af214bf Fix APRS crash on large station count and station list overflow
- Fix infinite loop in updateAprsStationList: querySelectorAll returns a
  static NodeList so the while(cards.length > 50) loop never terminated,
  crashing the page. Use live childElementCount instead.
- Fix station list pushing map off-screen by adding overflow:hidden and
  min-height:0 to flex containers so only the station list scrolls.
- Cap backend aprs_stations dict at 500 entries with oldest-eviction to
  prevent unbounded memory growth.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:13:28 +00:00
Smittix
c2891938ab Remove GSM spy functionality for legal compliance
Remove all GSM cellular intelligence features including tower scanning,
signal monitoring, rogue detection, crowd density analysis, and
OpenCellID integration across routes, templates, utils, tests, and
build configuration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 22:04:12 +00:00
Smittix
2bed35dd64 Fix signal handler deadlock and add satellite TLE data
Remove logging and cleanup_all_processes() from signal handler to
prevent deadlocks when another thread holds the logging or process lock.
Process cleanup is handled by the atexit handler instead.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:50:13 +00:00
Smittix
0c656cff2b Fix heatmap for towers with CID=0 or no geocoded coordinates
The monitored tower may have CID=0 (partially decoded cell) which
OpenCellID can't geocode, leaving it without coordinates. The heatmap
now falls back through: monitored tower by ARFCN, any geocoded tower,
then observer location. Also tracks the monitored ARFCN so the fallback
can find the right tower even when CID matching fails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:27:36 +00:00
Smittix
e03ba3f5ed Fix heatmap: add type coercion, LAC matching, debug logging, and user feedback
The heatmap silently failed when: CID types mismatched (string vs number),
LAC wasn't checked (wrong tower matched), or no data existed yet (button
showed ON with no layer). Now coerces CID/LAC to Number for comparison,
validates coordinates with parseFloat, logs match diagnostics to console,
and only shows ON when the layer is actually rendered.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 21:22:23 +00:00
Smittix
c6ff8abf11 Add Leaflet.heat crowd density heatmap to GSM Spy dashboard
Adds a toggleable heatmap layer that visualizes crowd density data from
the existing /gsm_spy/crowd_density endpoint as a gradient overlay on the
map, with auto-refresh every 30s during active monitoring.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:55:06 +00:00
Smittix
eff6ca3e87 Add 2G generation label to GSM band selector options
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:41:02 +00:00
Smittix
1a5b076a8d Fix grgsm_scanner crash on unsupported band names (GSM800, EGSM900_EXT)
Add explicit band name mapping from internal names to grgsm_scanner's
accepted -b values (GSM900, GSM850, DCS1800, PCS1900). Bands without
a valid grgsm_scanner equivalent (GSM800, EGSM900_EXT) are skipped
with a log message instead of crashing the scanner. Remove GSM800
from the dashboard band selector since it can't be scanned.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:39:12 +00:00
Smittix
90e88fc469 Fix tshark hex parsing and add API key settings UI
Parse tshark GSM field values with int(value, 0) instead of int(value)
to auto-detect hex 0x-prefixed output (e.g. 0x039e for TMSI/LAC/CID).
Without this, every tshark line with hex values fails to parse, causing
0 devices to be captured during monitoring.

Also add API Keys tab to Settings modal for configuring OpenCellID key
via the UI (in addition to env var), with status display and usage bar.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:35:31 +00:00
Smittix
98f6d18bea Fix GSM dashboard counters, improve lists, add device detail modal
Wire SIGNALS/DEVICES/CROWD counters to monitor_heartbeat SSE data so
they update in real-time during monitoring. Redesign device list items
as richer cards with type badges, TA/distance, and observation counts.
Add clickable device detail modal with full device info and copy
support. Improve tower list with signal strength bars. Widen right
sidebar and bump list font sizes for readability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 20:24:51 +00:00
Smittix
7d69cac7e7 Fix geocoding: validate API responses, clean poisoned cache, improve logging
- Cache lookup now requires non-NULL lat/lon — previously a row with
  NULL coordinates counted as a cache hit, returning {lat: None, lon: None}
  which the frontend silently ignored (tower in list but no map pin)
- API response handler validates lat/lon exist before caching, preventing
  error responses (status 200 with error body) from poisoning the cache
- On geocoding worker start, delete any existing poisoned cache rows
- Geocoding worker now logs "API key not configured" vs "rate limit
  reached" so the actual problem is visible in logs
- API error responses now log the response body for easier debugging
2026-02-08 19:55:00 +00:00
Smittix
c6a8a4a492 Fix EGSM900 downlink frequency: 935 MHz not 925 MHz
The EGSM900 band table had start=925e6 but ARFCNs 0-124 use downlink
frequencies starting at 935 MHz (DL = 935 + 0.2*ARFCN). The 925 MHz
value is the E-GSM extension band (ARFCNs 975-1023).

This caused grgsm_livemon to tune 10 MHz too low — ARFCN 22 tuned to
929.4 MHz instead of 939.4 MHz, receiving no GSM frames and producing
zero GSMTAP packets for tshark to capture.

Also adds EGSM900_EXT band (ARFCNs 975-1023, DL 925.2-934.8 MHz)
and diagnostic logging in the monitor thread to track raw tshark
line counts vs parsed packets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 19:50:04 +00:00
Mitch Ross
ca15e227cd add test harness 2026-02-08 14:45:12 -05:00
Smittix
391aff52ce Fix OpenCellID integration: CID=0 handling, API key check, tab parsing
- /lookup_cell and /detect_rogue rejected CID=0 towers because
  `all([..., cid])` is falsy when cid=0; use `is not None` checks
- can_use_api() now returns False when GSM_OPENCELLID_API_KEY is empty,
  preventing the geocoding worker from wasting daily quota on doomed calls
- /lookup_cell returns 503 with clear message when API key not configured
- parse_tshark_output uses rstrip('\n\r') instead of strip() to preserve
  leading empty tab-separated fields (strip() ate leading tabs, shifting
  all columns when the first field was empty)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 19:37:06 +00:00
Smittix
3dc16b392b Remove tshark -Y display filter that blocked all GSM packets
The display filter `gsm_a.tmsi || e212.imsi` was too restrictive —
paging requests use different field paths for TMSI so nothing matched.
The capture filter (-f 'udp port 4729') already limits to GSMTAP, and
the parser discards rows without TMSI/IMSI identifiers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 19:24:10 +00:00
Smittix
4d7be047da Fix tshark crash by skipping invalid fields instead of using fallbacks
When tshark field discovery finds no valid candidate for a logical field
(e.g. timing_advance, cellid), the old code fell back to the first
candidate name even though it was known to be invalid. This caused tshark
to exit immediately with "Some fields aren't valid".

Now fields resolve to None when no valid candidate exists, and the tshark
command is built using only validated fields. The parser dynamically maps
columns via field_order instead of assuming a fixed 5-column layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 19:14:09 +00:00
Smittix
182e1f3239 Fix tshark field discovery to validate with actual extraction test
tshark -G fields lists fields that exist in the protocol tree but
aren't all valid for -T fields -e extraction. Changed discovery to
actually test candidates by running tshark -T fields -e <field> -r
/dev/null and parsing stderr for invalid field names. This correctly
identifies which fields work for extraction on the installed version.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:56:50 +00:00
Smittix
87782319f2 Auto-discover tshark field names for GSM protocol compatibility
tshark field names differ between Wireshark versions (3.x vs 4.x):
- 3.x: gsm_a.rr.timing_advance, gsm_a.tmsi, gsm_a.cellid
- 4.x: gsm_a_rr.timing_adv, gsm_a_dtap.tmsi, e212.ci

Added _discover_tshark_fields() that queries `tshark -G fields` to
find which field names are available on the installed version, then
uses the correct ones for the capture filter and field extraction.
Results are cached after first discovery.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:49:20 +00:00
Smittix
6b7f817aa6 Add live monitoring status overlay with heartbeat updates
Backend: monitor_thread sends periodic monitor_heartbeat events (every
5s) with elapsed time, packet count, and device count so the frontend
knows monitoring is active.

Frontend: new monitoring overlay replaces scan progress bar when
auto-monitor starts. Shows pulsing green indicator, ARFCN being
monitored, live elapsed timer, packet/device counts, and
"Listening..."/"Capturing" activity state.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:41:30 +00:00
Smittix
82f442ffb8 Fix tshark capture: add GSMTAP filter, line buffering, stderr capture
- Add capture filter (-f 'udp port 4729') to only capture GSMTAP packets
- Add -l flag for line-buffered output on live capture
- Add early exit detection for tshark with stderr capture
- Add stderr reader thread in monitor_thread for ongoing tshark diagnostics
- Clean up grgsm_livemon if tshark fails to start

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 18:37:48 +00:00
Mitch Ross
1924203c19 Merge upstream/main: add gsm_spy blueprint 2026-02-08 13:15:20 -05:00