Security: replace path traversal-vulnerable str().startswith() with
is_relative_to(), anchor path checks to app root, strip filesystem
paths from error responses, add decoder-level path validation.
Architecture: use safe_terminate/register_process for subprocess
lifecycle, replace custom SSE generator with sse_stream(), use
centralized validate_* functions, remove unused app.py declarations.
Bugs: add thread-safe singleton locks, protect _images list across
threads, move blocking process.wait() to async daemon thread, fix
timezone handling for tz-aware datetimes, use full path for image
deduplication, guard TLE auto-refresh during tests, validate
scheduler parameters to avoid 500 errors.
Docker: pin SatDump to v1.2.2 and slowrx to ca6d7012, document
INTERCEPT_IMAGE fallback pattern.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolve conflicts keeping local GSM tools in kill_all() process list
and weather satellite config settings while merging upstream changes
including GSM spy removal, DMR fixes, USB device probe, APRS crash
fix, and cross-module frequency routing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The _DSD_FME_PROTOCOL_FLAGS dictionary had every protocol flag wrong,
causing dsd-fme (the preferred binary) to receive invalid or mismatched
-f flags. Also fix orphaned process leak on startup failure and add
centralized input validation for frequency/gain/device.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix infinite loop in updateAprsStationList: querySelectorAll returns a
static NodeList so the while(cards.length > 50) loop never terminated,
crashing the page. Use live childElementCount instead.
- Fix station list pushing map off-screen by adding overflow:hidden and
min-height:0 to flex containers so only the station list scrolls.
- Cap backend aprs_stations dict at 500 entries with oldest-eviction to
prevent unbounded memory growth.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove all GSM cellular intelligence features including tower scanning,
signal monitoring, rogue detection, crowd density analysis, and
OpenCellID integration across routes, templates, utils, tests, and
build configuration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a toggleable heatmap layer that visualizes crowd density data from
the existing /gsm_spy/crowd_density endpoint as a gradient overlay on the
map, with auto-refresh every 30s during active monitoring.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add explicit band name mapping from internal names to grgsm_scanner's
accepted -b values (GSM900, GSM850, DCS1800, PCS1900). Bands without
a valid grgsm_scanner equivalent (GSM800, EGSM900_EXT) are skipped
with a log message instead of crashing the scanner. Remove GSM800
from the dashboard band selector since it can't be scanned.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Parse tshark GSM field values with int(value, 0) instead of int(value)
to auto-detect hex 0x-prefixed output (e.g. 0x039e for TMSI/LAC/CID).
Without this, every tshark line with hex values fails to parse, causing
0 devices to be captured during monitoring.
Also add API Keys tab to Settings modal for configuring OpenCellID key
via the UI (in addition to env var), with status display and usage bar.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Cache lookup now requires non-NULL lat/lon — previously a row with
NULL coordinates counted as a cache hit, returning {lat: None, lon: None}
which the frontend silently ignored (tower in list but no map pin)
- API response handler validates lat/lon exist before caching, preventing
error responses (status 200 with error body) from poisoning the cache
- On geocoding worker start, delete any existing poisoned cache rows
- Geocoding worker now logs "API key not configured" vs "rate limit
reached" so the actual problem is visible in logs
- API error responses now log the response body for easier debugging
The EGSM900 band table had start=925e6 but ARFCNs 0-124 use downlink
frequencies starting at 935 MHz (DL = 935 + 0.2*ARFCN). The 925 MHz
value is the E-GSM extension band (ARFCNs 975-1023).
This caused grgsm_livemon to tune 10 MHz too low — ARFCN 22 tuned to
929.4 MHz instead of 939.4 MHz, receiving no GSM frames and producing
zero GSMTAP packets for tshark to capture.
Also adds EGSM900_EXT band (ARFCNs 975-1023, DL 925.2-934.8 MHz)
and diagnostic logging in the monitor thread to track raw tshark
line counts vs parsed packets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- /lookup_cell and /detect_rogue rejected CID=0 towers because
`all([..., cid])` is falsy when cid=0; use `is not None` checks
- can_use_api() now returns False when GSM_OPENCELLID_API_KEY is empty,
preventing the geocoding worker from wasting daily quota on doomed calls
- /lookup_cell returns 503 with clear message when API key not configured
- parse_tshark_output uses rstrip('\n\r') instead of strip() to preserve
leading empty tab-separated fields (strip() ate leading tabs, shifting
all columns when the first field was empty)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The display filter `gsm_a.tmsi || e212.imsi` was too restrictive —
paging requests use different field paths for TMSI so nothing matched.
The capture filter (-f 'udp port 4729') already limits to GSMTAP, and
the parser discards rows without TMSI/IMSI identifiers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When tshark field discovery finds no valid candidate for a logical field
(e.g. timing_advance, cellid), the old code fell back to the first
candidate name even though it was known to be invalid. This caused tshark
to exit immediately with "Some fields aren't valid".
Now fields resolve to None when no valid candidate exists, and the tshark
command is built using only validated fields. The parser dynamically maps
columns via field_order instead of assuming a fixed 5-column layout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
tshark -G fields lists fields that exist in the protocol tree but
aren't all valid for -T fields -e extraction. Changed discovery to
actually test candidates by running tshark -T fields -e <field> -r
/dev/null and parsing stderr for invalid field names. This correctly
identifies which fields work for extraction on the installed version.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
tshark field names differ between Wireshark versions (3.x vs 4.x):
- 3.x: gsm_a.rr.timing_advance, gsm_a.tmsi, gsm_a.cellid
- 4.x: gsm_a_rr.timing_adv, gsm_a_dtap.tmsi, e212.ci
Added _discover_tshark_fields() that queries `tshark -G fields` to
find which field names are available on the installed version, then
uses the correct ones for the capture filter and field extraction.
Results are cached after first discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend: monitor_thread sends periodic monitor_heartbeat events (every
5s) with elapsed time, packet count, and device count so the frontend
knows monitoring is active.
Frontend: new monitoring overlay replaces scan progress bar when
auto-monitor starts. Shows pulsing green indicator, ARFCN being
monitored, live elapsed timer, packet/device counts, and
"Listening..."/"Capturing" activity state.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add capture filter (-f 'udp port 4729') to only capture GSMTAP packets
- Add -l flag for line-buffered output on live capture
- Add early exit detection for tshark with stderr capture
- Add stderr reader thread in monitor_thread for ongoing tshark diagnostics
- Clean up grgsm_livemon if tshark fails to start
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Set QT_QPA_PLATFORM=offscreen for both grgsm_livemon and
grgsm_scanner to prevent SIGABRT when no X11 display is available.
grgsm_livemon uses GNU Radio which loads Qt plugins — without a
display, Qt aborts with "could not connect to display".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add pre-flight checks (shutil.which) for grgsm_livemon and tshark
- Capture stderr when grgsm_livemon exits immediately (exit code 1)
- Start background stderr reader thread for ongoing livemon diagnostics
- Add idle_count grace period in SSE stream to handle scanner→monitor
transition without premature disconnect
- Forward monitor failure errors to SSE for frontend display
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The scanner and monitor are mutually exclusive (both need the SDR).
Previously auto-monitor tried to start mid-scan (causing device
conflicts) and required 3 towers (rarely achieved with weak signals).
Now after the first scan completes:
- If any towers were found, automatically stop scanner and start
grgsm_livemon + tshark on the strongest tower's ARFCN
- SDR handoff is clean (scanner process has already exited)
- If monitor fails to start, scanner loop resumes
- Scanner thread's finally block preserves SDR allocation when
monitor has taken over
- Frontend shows "Monitoring ARFCN X for devices..." status
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Forward scanner progress (%) and status to SSE stream
- Show progress bar and scan status in TRACKED TOWERS panel
- Send scan_complete event with tower count and duration
- Fix Europe BAND_CONFIG: only EGSM900 is recommended (GSM850/GSM800
are rarely used in Europe and waste scan time)
- DCS1800 available but not recommended (RTL-SDR sensitivity is lower)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CID=0 with valid MCC/MNC means the scanner found the cell but didn't
decode System Information 3/4 (which carries the Cell ID). These are
still valid towers worth displaying. Only filter when MCC=0 AND MNC=0
(truly unidentified signals).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Backend:
- Filter out CID=0 and MCC=0 entries (ARFCNs with no decoded cell identity)
Frontend:
- Move stats update before coordinate check so towers always counted
- Fix signal_strength display using null check instead of || (0 is falsy)
- Show operator name, frequency, and status in tower detail panel
- Show "Located" indicator in tower list for geocoded towers
- Fix selectTower crash when tower has no coordinates
- Update placeholder text to "Select a tower from the list"
- Add try/catch to selectTower for error resilience
Tests:
- Add tests for CID=0 and MCC=0 filtering
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Send all existing towers on SSE connect (fixes data loss on reconnect)
- Fix tower.signal -> tower.signal_strength field name in frontend
- Fix TypeError crash in selectTower when tower has no coordinates
- Add Connection: keep-alive header to SSE response
- Add comprehensive console.log debugging for SSE data flow
- Handle error/disconnected SSE event types in frontend
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
grgsm_scanner is a Python/GNU Radio script, so stdbuf has no effect.
Setting PYTHONUNBUFFERED=1 in the subprocess env forces Python to
flush stdout on every write, enabling real-time scan output.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
grgsm_scanner fully buffers stdout when piped, so scan results never
reach Python until the buffer fills or process exits. Wrapping with
stdbuf -oL forces line-buffered output for real-time data streaming.
Also increased scan timeout from 120s to 300s since scanning 4 bands
legitimately takes 2-3 minutes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The ptrkrysik/gr-gsm repo uses SWIG which is incompatible with
GNU Radio 3.10+. The bkerler fork supports modern GNU Radio and
builds successfully on current Ubuntu/Debian systems.
Updated all references in Dockerfile, setup.sh, dependencies.py,
and error messages.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
apt-packaged gr-gsm aborts with SIGABRT (-6) due to duplicate FSM
registration in libosmocore. Setting this env var suppresses the
fatal assertion, allowing grgsm_scanner to run normally.
Applied to both scanner and livemon subprocess spawns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
grgsm_scanner exits in <300ms with osmo_fsm assertion error due to
libosmocore incompatibility. Added crash detection: if process exits
in <5s with non-zero code, counts as crash. After 3 crashes, stops
retrying and sends error to SSE stream so the UI can display it.
Also drains remaining queue items after process exits and logs exit
code and scan duration for diagnostics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Global LOG_LEVEL defaults to WARNING, silencing all INFO/DEBUG logs.
GSM Spy needs verbose logging for scanner diagnostics. Override the
module logger level to DEBUG so scanner output is always visible.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gsm_spy.py used logging.getLogger() directly which returns a bare
logger with no handler. The parent 'intercept' logger has
propagate=False, so all GSM Spy logs were silently dropped.
Now uses utils.logging.get_logger() which adds a stderr handler
and sets the log level, matching all other route modules.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
grgsm_scanner (like many GNU Radio tools) writes scan results to
stderr, not stdout. The stderr reader was only logging at debug
level and discarding lines. Now feeds stderr into the parse queue.
Also added info-level logging for all scanner output lines to aid
debugging.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Parser expected pipe-delimited table rows but grgsm_scanner outputs
comma-separated key-value pairs like:
ARFCN: 975, Freq: 925.2M, CID: 13522, LAC: 38722, MCC: 262, MNC: 1, Pwr: -58
This was the root cause of no data appearing in GSM Spy.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
stop_scanner() cleared gsm_spy_active_device without calling
release_sdr_device(), so the device stayed claimed in the registry.
The scanner thread's finally block then saw None and skipped release.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GSM Spy was failing with FileNotFoundError because grgsm_scanner
wasn't installed. These tools are now installed automatically by
setup.sh (both Debian and macOS) and included in the Dockerfile,
matching how other tools like multimon-ng and ffmpeg are handled.
- setup.sh: Remove ask_yes_no prompts for gr-gsm and tshark, install
unconditionally; add check_recommended tier for final summary
- Dockerfile: Add tshark to apt layer, add gr-gsm RUN layer with
apt-then-source-build fallback, preseed debconf for tshark
- gsm_spy.py: Add shutil.which pre-check in start_scanner route,
catch FileNotFoundError in scanner_thread to stop retry loop
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- kill_all() now resets gsm_spy_scanner_running and related state so
the scanner thread stops after killall
- scanner_thread sets flag to False instead of None on exit
- Restore alert_rules, alert_events, recording_sessions tables and
wifi_clients column removed by PR in database.py
- Escape all server-sourced values in analysis modals with escapeHtml()
- Reset gsm_towers_found/gsm_devices_tracked on stop to prevent
counter drift across sessions
- Replace raw terminate/kill with safe_terminate() in scanner_thread
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move waterfall controls from the sidebar into a function-strip bar inside
#listeningPostVisuals so they sit directly above the waterfall canvas.
Also fix the "SDR device in use" error when clicking a waterfall frequency
to listen — the WebSocket waterfall's device claim wasn't being released
before the audio start request because the backend cleanup hadn't finished.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Zooming caused "I/Q capture process exited immediately" because the client
closed the WebSocket and opened a new one, racing with the old rtl_sdr
process releasing the USB device. Now zoom/retune sends a start command on
the existing WebSocket, and the server adds a USB release delay plus retry
loop when restarting capture within the same connection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After a WebSocket handler exits, flask-sock returns a Response to
Werkzeug which writes "HTTP/1.1 200 OK..." on the still-open socket.
Browsers see these HTTP bytes as a malformed WebSocket frame, causing
"Invalid frame header".
Now the handler explicitly closes the raw TCP socket after the
WebSocket close handshake, so Werkzeug's write harmlessly fails.
Applied to both waterfall and audio WebSocket handlers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
simple-websocket 1.1.0's receive(timeout=N) returns None on timeout
instead of raising TimeoutError. The handler treated None as
"connection closed" and broke out of the loop, causing Werkzeug to
write its HTTP 200 response on the still-open WebSocket socket.
The browser saw those HTTP bytes as an invalid WebSocket frame.
Now checks ws.connected to distinguish timeout (None + connected)
from actual close (None + not connected).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
agents.js syncLocalModeStates() expects these endpoints to check
whether each mode is running locally. Both were missing, causing
404 errors on mode switch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The fft_reader thread was calling ws.send() concurrently with
ws.receive() in the main loop. simple-websocket is not thread-safe
for simultaneous read/write, corrupting frame headers. Now the reader
thread enqueues frames and only the main loop touches the WebSocket.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the batch rtl_power SSE pipeline with continuous I/Q streaming
via WebSocket for smooth ~25fps waterfall display. The server captures
raw I/Q samples (rtl_sdr/rx_sdr), computes Hann-windowed FFT, and
sends compact binary frames (1035 bytes vs ~15KB JSON, 93% reduction).
Client falls back to existing SSE path if WebSocket is unavailable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
