intercept/docs/DISTRIBUTED_AGENTS.md

# Intercept Distributed Agent System

This document describes the distributed agent architecture that allows multiple remote sensor nodes to feed data into a central Intercept controller.

## Overview

The agent system uses a hub-and-spoke architecture where:
- **Controller**: The main Intercept instance that aggregates data from multiple agents
- **Agents**: Lightweight sensor nodes running on remote devices with SDR hardware

```
                    ┌─────────────────────────────────┐
                    │      INTERCEPT CONTROLLER       │
                    │         (port 5050)             │
                    │                                 │
                    │  - Web UI with agent selector   │
                    │  - /controller/manage page      │
                    │  - Multi-agent SSE stream       │
                    │  - Push data storage            │
                    └─────────────────────────────────┘
                         ▲           ▲           ▲
                         │           │           │
              Push/Pull  │           │           │  Push/Pull
                         │           │           │
                    ┌────┴───┐  ┌────┴───┐  ┌────┴───┐
                    │ Agent  │  │ Agent  │  │ Agent  │
                    │  :8020 │  │  :8020 │  │  :8020 │
                    │        │  │        │  │        │
                    │[RTL-SDR]  │[HackRF] │  │[LimeSDR]
                    └────────┘  └────────┘  └────────┘
```

## Quick Start

### 1. Start the Controller

The controller is the main Intercept application:

```bash
cd intercept
python app.py
# Runs on http://localhost:5050
```

### 2. Configure an Agent

Create a config file on the remote machine:

```ini
# intercept_agent.cfg
[agent]
name = sensor-node-1
port = 8020
allowed_ips =
allow_cors = false

[controller]
url = http://192.168.1.100:5050
api_key = your-secret-key-here
push_enabled = true
push_interval = 5

[modes]
pager = true
sensor = true
adsb = true
wifi = true
bluetooth = true
```

### 3. Start the Agent

```bash
python intercept_agent.py --config intercept_agent.cfg
# Runs on http://localhost:8020
```

### 4. Register the Agent

Go to `http://controller:5050/controller/manage` and add the agent:
- **Name**: sensor-node-1 (must match config)
- **Base URL**: http://agent-ip:8020
- **API Key**: your-secret-key-here (must match config)

## Architecture

### Data Flow

The system supports two data flow patterns:

#### Push (Agent → Controller)

Agents automatically push captured data to the controller:

1. Agent captures data (e.g., rtl_433 sensor readings)
2. Data is queued in the `ControllerPushClient`
3. Agent POSTs to `http://controller/controller/api/ingest`
4. Controller validates API key and stores in `push_payloads` table
5. Data is available via SSE stream at `/controller/stream/all`

```
Agent                           Controller
  │                                 │
  │  POST /controller/api/ingest    │
  │  Header: X-API-Key: secret      │
  │  Body: {agent_name, scan_type,  │
  │         payload, timestamp}     │
  │ ──────────────────────────────► │
  │                                 │
  │         200 OK                  │
  │ ◄────────────────────────────── │
```

#### Pull (Controller → Agent)

The controller can also pull data on-demand:

1. User selects agent in UI dropdown
2. User clicks "Start Listening"
3. Controller proxies request to agent
4. Agent starts the mode and returns status
5. Controller polls agent for data

```
Browser                 Controller                    Agent
   │                        │                           │
   │ POST /controller/      │                           │
   │   agents/1/sensor/start│                           │
   │ ─────────────────────► │                           │
   │                        │ POST /sensor/start        │
   │                        │ ────────────────────────► │
   │                        │                           │
   │                        │      {status: started}    │
   │                        │ ◄──────────────────────── │
   │    {status: success}   │                           │
   │ ◄───────────────────── │                           │
```

### Authentication

API key authentication secures the push mechanism:

1. Agent config specifies `api_key` in `[controller]` section
2. Agent sends `X-API-Key` header with each push request
3. Controller looks up agent by name in database
4. Controller compares provided key with stored key
5. Mismatched keys return 401 Unauthorized

### Database Schema

Two tables support the agent system:

```sql
-- Registered agents
CREATE TABLE agents (
    id INTEGER PRIMARY KEY,
    name TEXT UNIQUE NOT NULL,
    base_url TEXT NOT NULL,
    api_key TEXT,
    capabilities TEXT,      -- JSON: {pager: true, sensor: true, ...}
    interfaces TEXT,        -- JSON: {devices: [...]}
    gps_coords TEXT,        -- JSON: {lat, lon}
    last_seen TIMESTAMP,
    is_active BOOLEAN
);

-- Pushed data from agents
CREATE TABLE push_payloads (
    id INTEGER PRIMARY KEY,
    agent_id INTEGER,
    scan_type TEXT,         -- pager, sensor, adsb, wifi, etc.
    payload TEXT,           -- JSON data
    received_at TIMESTAMP,
    FOREIGN KEY (agent_id) REFERENCES agents(id)
);
```

## Agent REST API

The agent exposes these endpoints:

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/health` | GET | Health check (returns `{status: "healthy"}`) |
| `/capabilities` | GET | Available modes, devices, GPS status |
| `/status` | GET | Running modes, uptime, push status |
| `/{mode}/start` | POST | Start a mode (pager, sensor, adsb, etc.) |
| `/{mode}/stop` | POST | Stop a mode |
| `/{mode}/status` | GET | Mode-specific status |
| `/{mode}/data` | GET | Current data snapshot |

### Example: Start Sensor Mode

```bash
curl -X POST http://agent:8020/sensor/start \
  -H "Content-Type: application/json" \
  -d '{"frequency": 433.92, "device_index": 0}'
```

Response:
```json
{
  "status": "started",
  "mode": "sensor",
  "command": "/usr/local/bin/rtl_433 -d 0 -f 433.92M -F json",
  "gps_enabled": true
}
```

### Example: Get Capabilities

```bash
curl http://agent:8020/capabilities
```

Response:
```json
{
  "modes": {
    "pager": true,
    "sensor": true,
    "adsb": true,
    "wifi": true,
    "bluetooth": true
  },
  "devices": [
    {
      "index": 0,
      "name": "RTLSDRBlog, Blog V4",
      "sdr_type": "rtlsdr",
      "capabilities": {
        "freq_min_mhz": 24.0,
        "freq_max_mhz": 1766.0
      }
    }
  ],
  "gps": true,
  "gps_position": {
    "lat": 33.543,
    "lon": -82.194,
    "altitude": 70.0
  },
  "tool_details": {
    "sensor": {
      "name": "433MHz Sensors",
      "ready": true,
      "tools": {
        "rtl_433": {"installed": true, "required": true}
      }
    }
  }
}
```

## Controller API

### Agent Management

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/controller/agents` | GET | List all agents |
| `/controller/agents` | POST | Register new agent |
| `/controller/agents/{id}` | GET | Get agent details |
| `/controller/agents/{id}` | DELETE | Remove agent |
| `/controller/agents/{id}?refresh=true` | GET | Refresh agent capabilities |

### Proxy Operations

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/controller/agents/{id}/{mode}/start` | POST | Start mode on agent |
| `/controller/agents/{id}/{mode}/stop` | POST | Stop mode on agent |
| `/controller/agents/{id}/{mode}/data` | GET | Get data from agent |

### Push Ingestion

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/controller/api/ingest` | POST | Receive pushed data from agents |

### SSE Streams

| Endpoint | Description |
|----------|-------------|
| `/controller/stream/all` | Combined stream from all agents |

## Frontend Integration

### Agent Selector

The main UI includes an agent dropdown in supported modes:

```html
<select id="agentSelect">
    <option value="local">Local (This Device)</option>
    <option value="1">● sensor-node-1</option>
</select>
```

When an agent is selected:
1. Device list updates to show agent's SDR devices
2. Start/Stop commands route through controller proxy
3. Data displays with agent name badge

### Multi-Agent Mode

Enable "Show All Agents" checkbox to:
- Connect to `/controller/stream/all` SSE
- Display combined data from all agents
- Show agent name badge on each data item

## GPS Integration

Agents can include GPS coordinates with captured data:

1. Agent connects to local `gpsd` daemon
2. GPS position included in `/capabilities` and `/status`
3. Each data snapshot includes `agent_gps` field
4. Controller can use GPS for trilateration (multiple agents)

## Configuration Reference

### Agent Config (`intercept_agent.cfg`)

```ini
[agent]
# Agent identity (must be unique across all agents)
name = sensor-node-1

# Port to listen on
port = 8020

# Restrict connections to specific IPs (comma-separated, empty = all)
allowed_ips =

# Enable CORS headers
allow_cors = false

[controller]
# Controller URL (required for push)
url = http://192.168.1.100:5050

# API key for authentication
api_key = your-secret-key

# Enable automatic data push
push_enabled = true

# Push interval in seconds
push_interval = 5

[modes]
# Enable/disable specific modes
pager = true
sensor = true
adsb = true
ais = true
wifi = true
bluetooth = true
```

## Troubleshooting

### Agent not appearing in controller

1. Check agent is running: `curl http://agent:8020/health`
2. Verify agent is registered in `/controller/manage`
3. Check API key matches between agent config and controller registration
4. Check network connectivity between agent and controller

### Push data not arriving

1. Check agent status: `curl http://agent:8020/status`
   - Verify `push_enabled: true` and `push_connected: true`
2. Check controller logs for authentication errors
3. Verify API key matches
4. Check if mode is running and producing data

### Mode won't start on agent

1. Check capabilities: `curl http://agent:8020/capabilities`
2. Verify required tools are installed (check `tool_details`)
3. Check if SDR device is available (not in use by another process)

### No data from sensor mode

1. Verify rtl_433 is running: `ps aux | grep rtl_433`
2. Check sensor status: `curl http://agent:8020/sensor/status`
3. Note: Empty data is normal if no 433MHz devices are transmitting nearby

## Security Considerations

1. **API Keys**: Always use strong, unique API keys for each agent
2. **Network**: Consider running agents on a private network or VPN
3. **HTTPS**: For production, use HTTPS between agents and controller
4. **Firewall**: Restrict agent ports to controller IP only
5. **allowed_ips**: Use this config option to restrict agent connections

## Files

| File | Description |
|------|-------------|
| `intercept_agent.py` | Standalone agent server |
| `intercept_agent.cfg` | Agent configuration template |
| `routes/controller.py` | Controller API blueprint |
| `utils/agent_client.py` | HTTP client for agents |
| `utils/database.py` | Agent CRUD operations |
| `static/js/core/agents.js` | Frontend agent management |
| `templates/agents.html` | Agent management page |