diff --git a/CHANGELOG.md b/CHANGELOG.md index cbb6fdb..7d34f32 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,11 +15,13 @@ All notable changes to iNTERCEPT will be documented in this file. - **Multi-SDR WeFax** - Multiple SDR hardware support for WeFax decoder - **Tool Path Overrides** - `INTERCEPT_*_PATH` environment variables for custom tool locations - **Homebrew Tool Detection** - Native path detection for Apple Silicon Homebrew installations +- **Production Server** - `start.sh` with gunicorn + gevent for concurrent SSE/WebSocket handling — eliminates multi-client page load delays ### Changed - Morse decoder rebuilt with custom Goertzel decoder, replacing multimon-ng dependency - GPS mode upgraded to textured 3D globe visualization - Destroy lifecycle added to all mode modules to prevent resource leaks +- Docker container now uses gunicorn + gevent by default via `start.sh` ### Fixed - ADS-B device release leak and startup performance regression diff --git a/CLAUDE.md b/CLAUDE.md index 6e29477..654c9a9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,12 +28,11 @@ docker compose --profile basic up -d --build # Initial setup (installs dependencies and configures SDR tools) ./setup.sh -# Run the application (requires sudo for SDR/network access) -sudo -E venv/bin/python intercept.py +# Run with production server (gunicorn + gevent, handles concurrent SSE/WebSocket) +sudo ./start.sh -# Or activate venv first -source venv/bin/activate -sudo -E python intercept.py +# Or for quick local dev (Flask dev server) +sudo -E venv/bin/python intercept.py ``` ### Testing @@ -69,8 +68,9 @@ mypy . ## Architecture ### Entry Points -- `intercept.py` - Main entry point script -- `app.py` - Flask application initialization, global state management, process lifecycle, SSE streaming infrastructure +- `start.sh` - Production startup script (gunicorn + gevent auto-detection, CLI flags, HTTPS, fallback to Flask dev server) +- `intercept.py` - Direct Flask dev server entry point (quick local development) +- `app.py` - Flask application initialization, global state management, process lifecycle, SSE streaming infrastructure, conditional gevent monkey-patch ### Route Blueprints (routes/) Each signal type has its own Flask blueprint: @@ -121,7 +121,7 @@ Each signal type has its own Flask blueprint: ### Key Patterns -**Server-Sent Events (SSE)**: All real-time features stream via SSE endpoints (`/stream_pager`, `/stream_sensor`, etc.). Pattern uses `queue.Queue` with timeout and keepalive messages. +**Server-Sent Events (SSE)**: All real-time features stream via SSE endpoints (`/stream_pager`, `/stream_sensor`, etc.). Pattern uses `queue.Queue` with timeout and keepalive messages. Under gunicorn + gevent, each SSE connection is a lightweight greenlet instead of an OS thread. **Process Management**: External decoders run as subprocesses with output threads feeding queues. Use `safe_terminate()` for cleanup. Global locks prevent race conditions. @@ -152,7 +152,7 @@ Each signal type has its own Flask blueprint: - **Mode Integration**: Each mode needs entries in `index.html` at ~12 points: CSS include, welcome card, partial include, visuals container, JS include, `validModes` set, `modeGroups` map, classList toggle, `modeNames`, visuals display toggle, titles, and init call in `switchMode()` ### Docker -- `Dockerfile` - Single-stage build with all SDR tools compiled from source (dump1090, AIS-catcher, slowrx, SatDump, etc.) +- `Dockerfile` - Single-stage build with all SDR tools compiled from source (dump1090, AIS-catcher, slowrx, SatDump, etc.). CMD runs `start.sh` (gunicorn + gevent) - `docker-compose.yml` - Two profiles: `basic` (standalone) and `history` (with Postgres for ADS-B) - `build-multiarch.sh` - Multi-arch build script for amd64 + arm64 (RPi5) - Data persisted via `./data:/app/data` volume mount diff --git a/Dockerfile b/Dockerfile index 1d0b789..be90562 100644 --- a/Dockerfile +++ b/Dockerfile @@ -274,4 +274,4 @@ HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ CMD curl -sf http://localhost:5050/health || exit 1 # Run the application -CMD ["python", "intercept.py"] +CMD ["/bin/bash", "start.sh"] diff --git a/README.md b/README.md index be8a5a4..2e48f5d 100644 --- a/README.md +++ b/README.md @@ -50,47 +50,49 @@ Support the developer of this open-source project - **Meshtastic** - LoRa mesh network integration - **Space Weather** - Real-time solar and geomagnetic data from NOAA SWPC, NASA SDO, and HamQSL (no SDR required) - **Spy Stations** - Number stations and diplomatic HF network database -- **Remote Agents** - Distributed SIGINT with remote sensor nodes -- **Offline Mode** - Bundled assets for air-gapped/field deployments - ---- - -## CW / Morse Decoder Notes - -Live backend: -- Uses `rtl_fm` piped into `multimon-ng` (`MORSE_CW`) for real-time decode. - -Recommended baseline settings: -- **Tone**: `700 Hz` -- **Bandwidth**: `200 Hz` (use `100 Hz` for crowded bands, `400 Hz` for drifting signals) -- **Threshold Mode**: `Auto` -- **WPM Mode**: `Auto` - -Auto Tone Track behavior: -- Continuously measures nearby tone energy around the configured CW pitch. -- Steers the detector toward the strongest valid CW tone when signal-to-noise is sufficient. -- Use **Hold Tone Lock** to freeze tracking once the desired signal is centered. - -Troubleshooting (no decode / noisy decode): -- Confirm demod path is **USB/CW-compatible** and frequency is tuned correctly. -- If multiple SDRs are connected and the selected one has no PCM output, Morse startup now auto-tries other detected SDR devices and reports the active device/serial in status logs. -- Match **tone** and **bandwidth** to the actual sidetone/pitch. -- Try **Threshold Auto** first; if needed, switch to manual threshold and recalibrate. -- Use **Reset/Calibrate** after major frequency or band condition changes. -- Raise **Minimum Signal Gate** to suppress random noise keying. - ---- - -## Installation / Debian / Ubuntu / MacOS +- **Remote Agents** - Distributed SIGINT with remote sensor nodes +- **Offline Mode** - Bundled assets for air-gapped/field deployments + +--- + +## CW / Morse Decoder Notes + +Live backend: +- Uses `rtl_fm` piped into `multimon-ng` (`MORSE_CW`) for real-time decode. + +Recommended baseline settings: +- **Tone**: `700 Hz` +- **Bandwidth**: `200 Hz` (use `100 Hz` for crowded bands, `400 Hz` for drifting signals) +- **Threshold Mode**: `Auto` +- **WPM Mode**: `Auto` + +Auto Tone Track behavior: +- Continuously measures nearby tone energy around the configured CW pitch. +- Steers the detector toward the strongest valid CW tone when signal-to-noise is sufficient. +- Use **Hold Tone Lock** to freeze tracking once the desired signal is centered. + +Troubleshooting (no decode / noisy decode): +- Confirm demod path is **USB/CW-compatible** and frequency is tuned correctly. +- If multiple SDRs are connected and the selected one has no PCM output, Morse startup now auto-tries other detected SDR devices and reports the active device/serial in status logs. +- Match **tone** and **bandwidth** to the actual sidetone/pitch. +- Try **Threshold Auto** first; if needed, switch to manual threshold and recalibrate. +- Use **Reset/Calibrate** after major frequency or band condition changes. +- Raise **Minimum Signal Gate** to suppress random noise keying. + +--- + +## Installation / Debian / Ubuntu / MacOS **1. Clone and run:** ```bash git clone https://github.com/smittix/intercept.git cd intercept ./setup.sh -sudo -E venv/bin/python intercept.py +sudo ./start.sh ``` +> **Production vs Dev server:** `start.sh` auto-detects gunicorn + gevent and runs a production server with cooperative greenlets — handles multiple SSE/WebSocket clients without blocking. Falls back to Flask dev server if gunicorn is not installed. For quick local development, you can still use `sudo -E venv/bin/python intercept.py` directly. + ### Docker ```bash @@ -174,7 +176,7 @@ Set these as environment variables for either local installs or Docker: ```bash INTERCEPT_ADSB_AUTO_START=true \ INTERCEPT_SHARED_OBSERVER_LOCATION=false \ -sudo -E venv/bin/python intercept.py +sudo ./start.sh ``` **Docker example (.env)** diff --git a/app.py b/app.py index 9fedb75..20f268c 100644 --- a/app.py +++ b/app.py @@ -6,6 +6,15 @@ Flask application and shared state. from __future__ import annotations +import os as _os +if _os.environ.get('INTERCEPT_USE_GEVENT') == '1': + try: + from gevent import monkey as _monkey + _monkey.patch_all(subprocess=False) + except ImportError: + pass +del _os + import sys import site diff --git a/config.py b/config.py index 5eef162..f8bc76f 100644 --- a/config.py +++ b/config.py @@ -20,6 +20,7 @@ CHANGELOG = [ "WeFax (Weather Fax) decoder with auto-scheduler and broadcast timeline", "System Health monitoring mode with telemetry dashboard", "HTTPS support, HackRF TSCM RF scan, ADS-B voice alerts", + "Production server (start.sh) with gunicorn + gevent for concurrent multi-client support", "Multi-SDR support for WeFax, tool path overrides, native Homebrew detection", "GPS mode upgraded to textured 3D globe", "Destroy lifecycle added to all mode modules to prevent resource leaks", diff --git a/docker-compose.yml b/docker-compose.yml index 9253075..78f55e3 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,6 +1,8 @@ # INTERCEPT - Signal Intelligence Platform # Docker Compose configuration for easy deployment # +# Uses gunicorn + gevent production server via start.sh (handles concurrent SSE/WebSocket) +# # Basic usage (build locally): # docker compose --profile basic up -d --build # diff --git a/docs/FEATURES.md b/docs/FEATURES.md index b3c6eef..2696701 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -466,6 +466,7 @@ The settings modal shows availability status for each bundled asset: ## General - **Web-based interface** - no desktop app needed +- **Production server** - gunicorn + gevent via `start.sh` for concurrent SSE/WebSocket handling (falls back to Flask dev server) - **Live message streaming** via Server-Sent Events (SSE) - **Audio alerts** with mute toggle - **Message export** to CSV/JSON diff --git a/docs/HARDWARE.md b/docs/HARDWARE.md index 68bea41..0b91140 100644 --- a/docs/HARDWARE.md +++ b/docs/HARDWARE.md @@ -139,10 +139,13 @@ pip install -r requirements.txt After installation: ```bash -sudo -E venv/bin/python intercept.py +sudo ./start.sh # Custom port -INTERCEPT_PORT=8080 sudo -E venv/bin/python intercept.py +sudo ./start.sh -p 8080 + +# HTTPS +sudo ./start.sh --https ``` Open **http://localhost:5050** in your browser. diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 480e9bf..95c7285 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -18,10 +18,9 @@ By default, INTERCEPT binds to `0.0.0.0:5050`, making it accessible from any net echo "block in on en0 proto tcp from any to any port 5050" | sudo pfctl -ef - ``` -2. **Bind to Localhost**: For local-only access, set the host environment variable: +2. **Bind to Localhost**: For local-only access, set the host or use the CLI flag: ```bash - export INTERCEPT_HOST=127.0.0.1 - python intercept.py + sudo ./start.sh -H 127.0.0.1 ``` 3. **Trusted Networks Only**: Only run INTERCEPT on networks you trust. The application has no authentication mechanism. diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md index 17bf3fa..0396ea8 100644 --- a/docs/TROUBLESHOOTING.md +++ b/docs/TROUBLESHOOTING.md @@ -25,7 +25,7 @@ sudo apt install python3-flask python3-requests python3-serial python3-skyfield # Then create venv with system packages python3 -m venv --system-site-packages venv source venv/bin/activate -sudo venv/bin/python intercept.py +sudo ./start.sh ``` ### "error: externally-managed-environment" (pip blocked) @@ -61,7 +61,7 @@ sudo apt install python3.11 python3.11-venv python3-pip python3.11 -m venv venv source venv/bin/activate pip install -r requirements.txt -sudo venv/bin/python intercept.py +sudo ./start.sh ``` ### Alternative: Use the setup script @@ -336,7 +336,7 @@ rtl_fm -M am -f 118000000 -s 24000 -r 24000 -g 40 2>/dev/null | \ Run INTERCEPT with sudo: ```bash -sudo -E venv/bin/python intercept.py +sudo ./start.sh ``` ### Interface not found after enabling monitor mode diff --git a/docs/USAGE.md b/docs/USAGE.md index 37774ad..7d9d5ef 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -172,7 +172,7 @@ Set the following environment variables (Docker recommended): ```bash INTERCEPT_ADSB_AUTO_START=true \ INTERCEPT_SHARED_OBSERVER_LOCATION=false \ -sudo -E venv/bin/python intercept.py +sudo ./start.sh ``` **Docker example (.env)** @@ -518,10 +518,26 @@ INTERCEPT can be configured via environment variables: | `INTERCEPT_LOG_LEVEL` | `WARNING` | Log level (DEBUG, INFO, WARNING, ERROR) | | `INTERCEPT_DEFAULT_GAIN` | `40` | Default RTL-SDR gain | -Example: `INTERCEPT_PORT=8080 sudo -E venv/bin/python intercept.py` +Example: `INTERCEPT_PORT=8080 sudo ./start.sh` ## Command-line Options +### Production server (recommended) + +``` +./start.sh --help + + -p, --port PORT Port to listen on (default: 5050) + -H, --host HOST Host to bind to (default: 0.0.0.0) + -d, --debug Run in debug mode (Flask dev server) + --https Enable HTTPS with self-signed certificate + --check-deps Check dependencies and exit +``` + +`start.sh` auto-detects gunicorn + gevent and runs a production WSGI server with cooperative greenlets — this handles multiple SSE streams and WebSocket connections concurrently without blocking. Falls back to the Flask dev server if gunicorn is not installed. + +### Development server + ``` python3 intercept.py --help diff --git a/docs/index.html b/docs/index.html index 4a11de1..ffe5544 100644 --- a/docs/index.html +++ b/docs/index.html @@ -331,7 +331,7 @@
git clone https://github.com/smittix/intercept.git
cd intercept
./setup.sh
-sudo -E venv/bin/python intercept.py
+sudo ./start.sh
Requires Python 3.9+ and RTL-SDR drivers
diff --git a/requirements.txt b/requirements.txt index 6b9fdfb..1b72bb0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -47,3 +47,7 @@ websocket-client>=1.6.0 # System health monitoring (optional - graceful fallback if unavailable) psutil>=5.9.0 + +# Production WSGI server (optional - falls back to Flask dev server) +gunicorn>=21.2.0 +gevent>=23.9.0 diff --git a/setup.sh b/setup.sh index a31cb04..94fb87b 100755 --- a/setup.sh +++ b/setup.sh @@ -337,7 +337,8 @@ install_python_deps() { info "Installing optional packages..." for pkg in "numpy>=1.24.0" "scipy>=1.10.0" "Pillow>=9.0.0" "skyfield>=1.45" \ "bleak>=0.21.0" "psycopg2-binary>=2.9.9" "meshtastic>=2.0.0" \ - "scapy>=2.4.5" "qrcode[pil]>=7.4" "cryptography>=41.0.0"; do + "scapy>=2.4.5" "qrcode[pil]>=7.4" "cryptography>=41.0.0" \ + "gunicorn>=21.2.0" "gevent>=23.9.0"; do pkg_name="${pkg%%>=*}" if ! $PIP install "$pkg" 2>/dev/null; then warn "${pkg_name} failed to install (optional - related features may be unavailable)" @@ -1590,6 +1591,9 @@ final_summary_and_hard_fail() { echo "============================================" echo echo "To start INTERCEPT:" + echo " sudo ./start.sh" + echo + echo "Or for quick local dev:" echo " sudo -E venv/bin/python intercept.py" echo echo "Then open http://localhost:5050 in your browser" diff --git a/start.sh b/start.sh new file mode 100755 index 0000000..72422ec --- /dev/null +++ b/start.sh @@ -0,0 +1,150 @@ +#!/usr/bin/env bash +# INTERCEPT - Production Startup Script +# +# Starts INTERCEPT with gunicorn + gevent for production use. +# Falls back to Flask dev server if gunicorn is not installed. +# +# Usage: +# ./start.sh # Default: 0.0.0.0:5050 +# ./start.sh -p 8080 # Custom port +# ./start.sh --https # HTTPS with self-signed cert +# ./start.sh --debug # Debug mode (Flask dev server) +# ./start.sh --check-deps # Check dependencies and exit + +set -euo pipefail + +# ── Defaults (can be overridden by env vars or CLI flags) ──────────────────── +HOST="${INTERCEPT_HOST:-0.0.0.0}" +PORT="${INTERCEPT_PORT:-5050}" +DEBUG=0 +HTTPS=0 +CHECK_DEPS=0 + +# ── Parse CLI arguments ───────────────────────────────────────────────────── +while [[ $# -gt 0 ]]; do + case "$1" in + -p|--port) + PORT="$2" + shift 2 + ;; + -H|--host) + HOST="$2" + shift 2 + ;; + -d|--debug) + DEBUG=1 + shift + ;; + --https) + HTTPS=1 + shift + ;; + --check-deps) + CHECK_DEPS=1 + shift + ;; + -h|--help) + echo "Usage: start.sh [OPTIONS]" + echo "" + echo "Options:" + echo " -p, --port PORT Port to listen on (default: 5050)" + echo " -H, --host HOST Host to bind to (default: 0.0.0.0)" + echo " -d, --debug Run in debug mode (Flask dev server)" + echo " --https Enable HTTPS with self-signed certificate" + echo " --check-deps Check dependencies and exit" + echo " -h, --help Show this help message" + exit 0 + ;; + *) + echo "Unknown option: $1" >&2 + exit 1 + ;; + esac +done + +# ── Export for config.py ───────────────────────────────────────────────────── +export INTERCEPT_HOST="$HOST" +export INTERCEPT_PORT="$PORT" + +# ── Dependency check (delegate to intercept.py) ───────────────────────────── +if [[ "$CHECK_DEPS" -eq 1 ]]; then + exec python intercept.py --check-deps +fi + +# ── Debug mode always uses Flask dev server ────────────────────────────────── +if [[ "$DEBUG" -eq 1 ]]; then + echo "[INTERCEPT] Starting in debug mode (Flask dev server)..." + export INTERCEPT_DEBUG=1 + exec python intercept.py --host "$HOST" --port "$PORT" --debug +fi + +# ── HTTPS certificate generation ──────────────────────────────────────────── +CERT_DIR="certs" +CERT_FILE="$CERT_DIR/intercept.crt" +KEY_FILE="$CERT_DIR/intercept.key" + +if [[ "$HTTPS" -eq 1 ]]; then + if [[ ! -f "$CERT_FILE" || ! -f "$KEY_FILE" ]]; then + echo "[INTERCEPT] Generating self-signed SSL certificate..." + mkdir -p "$CERT_DIR" + openssl req -x509 -newkey rsa:2048 \ + -keyout "$KEY_FILE" -out "$CERT_FILE" \ + -days 365 -nodes \ + -subj '/CN=intercept/O=INTERCEPT/C=US' 2>/dev/null + echo "[INTERCEPT] SSL certificate generated: $CERT_FILE" + else + echo "[INTERCEPT] Using existing SSL certificate: $CERT_FILE" + fi +fi + +# ── Detect gunicorn + gevent ───────────────────────────────────────────────── +HAS_GUNICORN=0 +HAS_GEVENT=0 + +if python -c "import gunicorn" 2>/dev/null; then + HAS_GUNICORN=1 +fi +if python -c "import gevent" 2>/dev/null; then + HAS_GEVENT=1 +fi + +# ── Start the server ───────────────────────────────────────────────────────── +if [[ "$HAS_GUNICORN" -eq 1 && "$HAS_GEVENT" -eq 1 ]]; then + echo "[INTERCEPT] Starting production server (gunicorn + gevent)..." + echo "[INTERCEPT] Listening on ${HOST}:${PORT}" + + export INTERCEPT_USE_GEVENT=1 + + GUNICORN_ARGS=( + -k gevent + -w 1 + --timeout 300 + --worker-connections 1000 + --bind "${HOST}:${PORT}" + --access-logfile - + --error-logfile - + ) + + if [[ "$HTTPS" -eq 1 ]]; then + GUNICORN_ARGS+=(--certfile "$CERT_FILE" --keyfile "$KEY_FILE") + echo "[INTERCEPT] HTTPS enabled" + fi + + exec gunicorn "${GUNICORN_ARGS[@]}" app:app +else + if [[ "$HAS_GUNICORN" -eq 0 ]]; then + echo "[INTERCEPT] gunicorn not found — falling back to Flask dev server" + fi + if [[ "$HAS_GEVENT" -eq 0 ]]; then + echo "[INTERCEPT] gevent not found — falling back to Flask dev server" + fi + echo "[INTERCEPT] Install with: pip install gunicorn gevent" + echo "" + + FLASK_ARGS=(--host "$HOST" --port "$PORT") + if [[ "$HTTPS" -eq 1 ]]; then + FLASK_ARGS+=(--https) + fi + + exec python intercept.py "${FLASK_ARGS[@]}" +fi