mirror of
https://github.com/smittix/intercept.git
synced 2026-04-27 08:10:00 -07:00
Add distributed agent architecture for multi-node signal intelligence
Features: - Standalone agent server (intercept_agent.py) for remote sensor nodes - Controller API blueprint for agent management and data aggregation - Push mechanism for agents to send data to controller - Pull mechanism for controller to proxy requests to agents - Multi-agent SSE stream for combined data view - Agent management page at /controller/manage - Agent selector dropdown in main UI - GPS integration for location tagging - API key authentication for secure agent communication - Integration with Intercept's dependency checking system New files: - intercept_agent.py: Remote agent HTTP server - intercept_agent.cfg: Agent configuration template - routes/controller.py: Controller API endpoints - utils/agent_client.py: HTTP client for agents - utils/trilateration.py: Multi-agent position calculation - static/js/core/agents.js: Frontend agent management - templates/agents.html: Agent management page - docs/DISTRIBUTED_AGENTS.md: System documentation Modified: - app.py: Register controller blueprint - utils/database.py: Add agents and push_payloads tables - templates/index.html: Add agent selector section
This commit is contained in:
cemaxecuter
409
docs/DISTRIBUTED_AGENTS.md
Normal file
409
docs/DISTRIBUTED_AGENTS.md
Normal file
@@ -0,0 +1,409 @@
|
||||
# Intercept Distributed Agent System
|
||||
|
||||
This document describes the distributed agent architecture that allows multiple remote sensor nodes to feed data into a central Intercept controller.
|
||||
|
||||
## Overview
|
||||
|
||||
The agent system uses a hub-and-spoke architecture where:
|
||||
- **Controller**: The main Intercept instance that aggregates data from multiple agents
|
||||
- **Agents**: Lightweight sensor nodes running on remote devices with SDR hardware
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
│ INTERCEPT CONTROLLER │
|
||||
│ (port 5050) │
|
||||
│ │
|
||||
│ - Web UI with agent selector │
|
||||
│ - /controller/manage page │
|
||||
│ - Multi-agent SSE stream │
|
||||
│ - Push data storage │
|
||||
└─────────────────────────────────┘
|
||||
▲ ▲ ▲
|
||||
│ │ │
|
||||
Push/Pull │ │ │ Push/Pull
|
||||
│ │ │
|
||||
┌────┴───┐ ┌────┴───┐ ┌────┴───┐
|
||||
│ Agent │ │ Agent │ │ Agent │
|
||||
│ :8020 │ │ :8020 │ │ :8020 │
|
||||
│ │ │ │ │ │
|
||||
│[RTL-SDR] │[HackRF] │ │[LimeSDR]
|
||||
└────────┘ └────────┘ └────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Start the Controller
|
||||
|
||||
The controller is the main Intercept application:
|
||||
|
||||
```bash
|
||||
cd intercept
|
||||
python app.py
|
||||
# Runs on http://localhost:5050
|
||||
```
|
||||
|
||||
### 2. Configure an Agent
|
||||
|
||||
Create a config file on the remote machine:
|
||||
|
||||
```ini
|
||||
# intercept_agent.cfg
|
||||
[agent]
|
||||
name = sensor-node-1
|
||||
port = 8020
|
||||
allowed_ips =
|
||||
allow_cors = false
|
||||
|
||||
[controller]
|
||||
url = http://192.168.1.100:5050
|
||||
api_key = your-secret-key-here
|
||||
push_enabled = true
|
||||
push_interval = 5
|
||||
|
||||
[modes]
|
||||
pager = true
|
||||
sensor = true
|
||||
adsb = true
|
||||
wifi = true
|
||||
bluetooth = true
|
||||
```
|
||||
|
||||
### 3. Start the Agent
|
||||
|
||||
```bash
|
||||
python intercept_agent.py --config intercept_agent.cfg
|
||||
# Runs on http://localhost:8020
|
||||
```
|
||||
|
||||
### 4. Register the Agent
|
||||
|
||||
Go to `http://controller:5050/controller/manage` and add the agent:
|
||||
- **Name**: sensor-node-1 (must match config)
|
||||
- **Base URL**: http://agent-ip:8020
|
||||
- **API Key**: your-secret-key-here (must match config)
|
||||
|
||||
## Architecture
|
||||
|
||||
### Data Flow
|
||||
|
||||
The system supports two data flow patterns:
|
||||
|
||||
#### Push (Agent → Controller)
|
||||
|
||||
Agents automatically push captured data to the controller:
|
||||
|
||||
1. Agent captures data (e.g., rtl_433 sensor readings)
|
||||
2. Data is queued in the `ControllerPushClient`
|
||||
3. Agent POSTs to `http://controller/controller/api/ingest`
|
||||
4. Controller validates API key and stores in `push_payloads` table
|
||||
5. Data is available via SSE stream at `/controller/stream/all`
|
||||
|
||||
```
|
||||
Agent Controller
|
||||
│ │
|
||||
│ POST /controller/api/ingest │
|
||||
│ Header: X-API-Key: secret │
|
||||
│ Body: {agent_name, scan_type, │
|
||||
│ payload, timestamp} │
|
||||
│ ──────────────────────────────► │
|
||||
│ │
|
||||
│ 200 OK │
|
||||
│ ◄────────────────────────────── │
|
||||
```
|
||||
|
||||
#### Pull (Controller → Agent)
|
||||
|
||||
The controller can also pull data on-demand:
|
||||
|
||||
1. User selects agent in UI dropdown
|
||||
2. User clicks "Start Listening"
|
||||
3. Controller proxies request to agent
|
||||
4. Agent starts the mode and returns status
|
||||
5. Controller polls agent for data
|
||||
|
||||
```
|
||||
Browser Controller Agent
|
||||
│ │ │
|
||||
│ POST /controller/ │ │
|
||||
│ agents/1/sensor/start│ │
|
||||
│ ─────────────────────► │ │
|
||||
│ │ POST /sensor/start │
|
||||
│ │ ────────────────────────► │
|
||||
│ │ │
|
||||
│ │ {status: started} │
|
||||
│ │ ◄──────────────────────── │
|
||||
│ {status: success} │ │
|
||||
│ ◄───────────────────── │ │
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
API key authentication secures the push mechanism:
|
||||
|
||||
1. Agent config specifies `api_key` in `[controller]` section
|
||||
2. Agent sends `X-API-Key` header with each push request
|
||||
3. Controller looks up agent by name in database
|
||||
4. Controller compares provided key with stored key
|
||||
5. Mismatched keys return 401 Unauthorized
|
||||
|
||||
### Database Schema
|
||||
|
||||
Two tables support the agent system:
|
||||
|
||||
```sql
|
||||
-- Registered agents
|
||||
CREATE TABLE agents (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT UNIQUE NOT NULL,
|
||||
base_url TEXT NOT NULL,
|
||||
api_key TEXT,
|
||||
capabilities TEXT, -- JSON: {pager: true, sensor: true, ...}
|
||||
interfaces TEXT, -- JSON: {devices: [...]}
|
||||
gps_coords TEXT, -- JSON: {lat, lon}
|
||||
last_seen TIMESTAMP,
|
||||
is_active BOOLEAN
|
||||
);
|
||||
|
||||
-- Pushed data from agents
|
||||
CREATE TABLE push_payloads (
|
||||
id INTEGER PRIMARY KEY,
|
||||
agent_id INTEGER,
|
||||
scan_type TEXT, -- pager, sensor, adsb, wifi, etc.
|
||||
payload TEXT, -- JSON data
|
||||
received_at TIMESTAMP,
|
||||
FOREIGN KEY (agent_id) REFERENCES agents(id)
|
||||
);
|
||||
```
|
||||
|
||||
## Agent REST API
|
||||
|
||||
The agent exposes these endpoints:
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/health` | GET | Health check (returns `{status: "healthy"}`) |
|
||||
| `/capabilities` | GET | Available modes, devices, GPS status |
|
||||
| `/status` | GET | Running modes, uptime, push status |
|
||||
| `/{mode}/start` | POST | Start a mode (pager, sensor, adsb, etc.) |
|
||||
| `/{mode}/stop` | POST | Stop a mode |
|
||||
| `/{mode}/status` | GET | Mode-specific status |
|
||||
| `/{mode}/data` | GET | Current data snapshot |
|
||||
|
||||
### Example: Start Sensor Mode
|
||||
|
||||
```bash
|
||||
curl -X POST http://agent:8020/sensor/start \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"frequency": 433.92, "device_index": 0}'
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"status": "started",
|
||||
"mode": "sensor",
|
||||
"command": "/usr/local/bin/rtl_433 -d 0 -f 433.92M -F json",
|
||||
"gps_enabled": true
|
||||
}
|
||||
```
|
||||
|
||||
### Example: Get Capabilities
|
||||
|
||||
```bash
|
||||
curl http://agent:8020/capabilities
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"modes": {
|
||||
"pager": true,
|
||||
"sensor": true,
|
||||
"adsb": true,
|
||||
"wifi": true,
|
||||
"bluetooth": true
|
||||
},
|
||||
"devices": [
|
||||
{
|
||||
"index": 0,
|
||||
"name": "RTLSDRBlog, Blog V4",
|
||||
"sdr_type": "rtlsdr",
|
||||
"capabilities": {
|
||||
"freq_min_mhz": 24.0,
|
||||
"freq_max_mhz": 1766.0
|
||||
}
|
||||
}
|
||||
],
|
||||
"gps": true,
|
||||
"gps_position": {
|
||||
"lat": 33.543,
|
||||
"lon": -82.194,
|
||||
"altitude": 70.0
|
||||
},
|
||||
"tool_details": {
|
||||
"sensor": {
|
||||
"name": "433MHz Sensors",
|
||||
"ready": true,
|
||||
"tools": {
|
||||
"rtl_433": {"installed": true, "required": true}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Controller API
|
||||
|
||||
### Agent Management
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/controller/agents` | GET | List all agents |
|
||||
| `/controller/agents` | POST | Register new agent |
|
||||
| `/controller/agents/{id}` | GET | Get agent details |
|
||||
| `/controller/agents/{id}` | DELETE | Remove agent |
|
||||
| `/controller/agents/{id}?refresh=true` | GET | Refresh agent capabilities |
|
||||
|
||||
### Proxy Operations
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/controller/agents/{id}/{mode}/start` | POST | Start mode on agent |
|
||||
| `/controller/agents/{id}/{mode}/stop` | POST | Stop mode on agent |
|
||||
| `/controller/agents/{id}/{mode}/data` | GET | Get data from agent |
|
||||
|
||||
### Push Ingestion
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/controller/api/ingest` | POST | Receive pushed data from agents |
|
||||
|
||||
### SSE Streams
|
||||
|
||||
| Endpoint | Description |
|
||||
|----------|-------------|
|
||||
| `/controller/stream/all` | Combined stream from all agents |
|
||||
|
||||
## Frontend Integration
|
||||
|
||||
### Agent Selector
|
||||
|
||||
The main UI includes an agent dropdown in supported modes:
|
||||
|
||||
```html
|
||||
<select id="agentSelect">
|
||||
<option value="local">Local (This Device)</option>
|
||||
<option value="1">● sensor-node-1</option>
|
||||
</select>
|
||||
```
|
||||
|
||||
When an agent is selected:
|
||||
1. Device list updates to show agent's SDR devices
|
||||
2. Start/Stop commands route through controller proxy
|
||||
3. Data displays with agent name badge
|
||||
|
||||
### Multi-Agent Mode
|
||||
|
||||
Enable "Show All Agents" checkbox to:
|
||||
- Connect to `/controller/stream/all` SSE
|
||||
- Display combined data from all agents
|
||||
- Show agent name badge on each data item
|
||||
|
||||
## GPS Integration
|
||||
|
||||
Agents can include GPS coordinates with captured data:
|
||||
|
||||
1. Agent connects to local `gpsd` daemon
|
||||
2. GPS position included in `/capabilities` and `/status`
|
||||
3. Each data snapshot includes `agent_gps` field
|
||||
4. Controller can use GPS for trilateration (multiple agents)
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Agent Config (`intercept_agent.cfg`)
|
||||
|
||||
```ini
|
||||
[agent]
|
||||
# Agent identity (must be unique across all agents)
|
||||
name = sensor-node-1
|
||||
|
||||
# Port to listen on
|
||||
port = 8020
|
||||
|
||||
# Restrict connections to specific IPs (comma-separated, empty = all)
|
||||
allowed_ips =
|
||||
|
||||
# Enable CORS headers
|
||||
allow_cors = false
|
||||
|
||||
[controller]
|
||||
# Controller URL (required for push)
|
||||
url = http://192.168.1.100:5050
|
||||
|
||||
# API key for authentication
|
||||
api_key = your-secret-key
|
||||
|
||||
# Enable automatic data push
|
||||
push_enabled = true
|
||||
|
||||
# Push interval in seconds
|
||||
push_interval = 5
|
||||
|
||||
[modes]
|
||||
# Enable/disable specific modes
|
||||
pager = true
|
||||
sensor = true
|
||||
adsb = true
|
||||
ais = true
|
||||
wifi = true
|
||||
bluetooth = true
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Agent not appearing in controller
|
||||
|
||||
1. Check agent is running: `curl http://agent:8020/health`
|
||||
2. Verify agent is registered in `/controller/manage`
|
||||
3. Check API key matches between agent config and controller registration
|
||||
4. Check network connectivity between agent and controller
|
||||
|
||||
### Push data not arriving
|
||||
|
||||
1. Check agent status: `curl http://agent:8020/status`
|
||||
- Verify `push_enabled: true` and `push_connected: true`
|
||||
2. Check controller logs for authentication errors
|
||||
3. Verify API key matches
|
||||
4. Check if mode is running and producing data
|
||||
|
||||
### Mode won't start on agent
|
||||
|
||||
1. Check capabilities: `curl http://agent:8020/capabilities`
|
||||
2. Verify required tools are installed (check `tool_details`)
|
||||
3. Check if SDR device is available (not in use by another process)
|
||||
|
||||
### No data from sensor mode
|
||||
|
||||
1. Verify rtl_433 is running: `ps aux | grep rtl_433`
|
||||
2. Check sensor status: `curl http://agent:8020/sensor/status`
|
||||
3. Note: Empty data is normal if no 433MHz devices are transmitting nearby
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. **API Keys**: Always use strong, unique API keys for each agent
|
||||
2. **Network**: Consider running agents on a private network or VPN
|
||||
3. **HTTPS**: For production, use HTTPS between agents and controller
|
||||
4. **Firewall**: Restrict agent ports to controller IP only
|
||||
5. **allowed_ips**: Use this config option to restrict agent connections
|
||||
|
||||
## Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `intercept_agent.py` | Standalone agent server |
|
||||
| `intercept_agent.cfg` | Agent configuration template |
|
||||
| `routes/controller.py` | Controller API blueprint |
|
||||
| `utils/agent_client.py` | HTTP client for agents |
|
||||
| `utils/database.py` | Agent CRUD operations |
|
||||
| `static/js/core/agents.js` | Frontend agent management |
|
||||
| `templates/agents.html` | Agent management page |
|
||||
Reference in New Issue
Block a user