feat: Phase 0 bootstrap — kindexr boots, migrates, serves /health
- config: koanf-based loading (defaults → YAML → KINDEXR_ env vars) - db: embedded SQLite migrations with BEGIN/END-aware statement splitter - server: chi router, GET /health returns JSON stats - cmd/kindexr: graceful SIGTERM shutdown - cmd/kindexr-cli: stub - deploy: systemd unit, example config, nginx snippet - all packages covered by race-clean tests
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
bin/
|
||||
*.db
|
||||
@@ -0,0 +1,36 @@
|
||||
.PHONY: build test clean install run lint fmt vet check
|
||||
|
||||
BINARY := kindexr
|
||||
CLI := kindexr-cli
|
||||
VERSION := $(shell git describe --tags --always --dirty 2>/dev/null || echo dev)
|
||||
LDFLAGS := -ldflags "-X main.version=$(VERSION) -s -w"
|
||||
|
||||
build:
|
||||
go build $(LDFLAGS) -o ./bin/$(BINARY) ./cmd/kindexr
|
||||
go build $(LDFLAGS) -o ./bin/$(CLI) ./cmd/kindexr-cli
|
||||
|
||||
test:
|
||||
go test -race ./...
|
||||
|
||||
lint:
|
||||
golangci-lint run
|
||||
|
||||
fmt:
|
||||
gofmt -w .
|
||||
|
||||
vet:
|
||||
go vet ./...
|
||||
|
||||
check: fmt vet test
|
||||
|
||||
install: build
|
||||
install -D -m 0755 ./bin/$(BINARY) /usr/local/bin/$(BINARY)
|
||||
install -D -m 0755 ./bin/$(CLI) /usr/local/bin/$(CLI)
|
||||
install -D -m 0644 ./deploy/kindexr.service /etc/systemd/system/kindexr.service
|
||||
install -D -m 0640 ./deploy/kindexr.example.yaml /etc/kindexr/config.yaml
|
||||
|
||||
run:
|
||||
go run ./cmd/kindexr --config ./deploy/kindexr.example.yaml
|
||||
|
||||
clean:
|
||||
rm -rf ./bin
|
||||
@@ -0,0 +1,89 @@
|
||||
# kindexr
|
||||
|
||||
A Nostr-native Torznab indexer. Bridges NIP-35 torrent events on the Nostr relay network into the Torznab API that Sonarr, Radarr, Lidarr, Readarr, and Prowlarr already speak.
|
||||
|
||||
**Same slot as Jackett or Prowlarr** — middleware that sits between Nostr and the *arr automation stack. Not a frontend, not a relay, not a downloader.
|
||||
|
||||
## Phase status
|
||||
|
||||
- [x] Phase 0 — bootstrap: daemon boots, parses config, opens DB, applies migrations, `/health` returns JSON, logs to journald in JSON
|
||||
- [ ] Phase 1 — reader, basic Torznab (subscribes to relays, indexes kind 2003 events, `t=caps` + `t=search`)
|
||||
- [ ] Phase 2 — full *arr compatibility (structured ID matching, category normalization, TMDB enrichment)
|
||||
- [ ] Phase 3 — curation (WoT filter, NIP-51 sets, per-key visibility)
|
||||
- [ ] Phase 4 — writer/publisher (NIP-46 bunker, qBittorrent integration)
|
||||
|
||||
## Quick start
|
||||
|
||||
### Build
|
||||
|
||||
```sh
|
||||
make build
|
||||
# produces ./bin/kindexr and ./bin/kindexr-cli
|
||||
```
|
||||
|
||||
### Configure
|
||||
|
||||
```sh
|
||||
sudo mkdir -p /etc/kindexr /var/lib/kindexr
|
||||
sudo cp deploy/kindexr.example.yaml /etc/kindexr/config.yaml
|
||||
sudo $EDITOR /etc/kindexr/config.yaml
|
||||
```
|
||||
|
||||
Create the service user:
|
||||
|
||||
```sh
|
||||
sudo useradd --system --no-create-home --shell /usr/sbin/nologin kindexr
|
||||
sudo chown kindexr:kindexr /var/lib/kindexr
|
||||
```
|
||||
|
||||
### Install and start
|
||||
|
||||
```sh
|
||||
sudo make install
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now kindexr
|
||||
sudo journalctl -u kindexr -f
|
||||
```
|
||||
|
||||
### Verify
|
||||
|
||||
```sh
|
||||
curl http://localhost:9117/health
|
||||
# {"status":"ok","version":"...","relays_connected":0,"events_total":0,"last_event_at":null}
|
||||
```
|
||||
|
||||
### Add to Sonarr/Radarr
|
||||
|
||||
Once Phase 1 is complete, add kindexr as a Torznab indexer:
|
||||
|
||||
- URL: `http://127.0.0.1:9117` (or your public URL behind nginx)
|
||||
- API key: generate with `kindexr-cli apikey create --label sonarr`
|
||||
|
||||
## Configuration
|
||||
|
||||
See `deploy/kindexr.example.yaml` for a fully commented configuration reference.
|
||||
|
||||
Config is loaded in order: **defaults → YAML file → environment variables** (`KINDEXR_` prefix).
|
||||
|
||||
Example env override:
|
||||
|
||||
```sh
|
||||
KINDEXR_LOGGING_LEVEL=debug kindexr --config /etc/kindexr/config.yaml
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
make test # run tests with -race
|
||||
make vet # go vet
|
||||
make fmt # gofmt
|
||||
make check # fmt + vet + test
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for the full design.
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
@@ -0,0 +1,23 @@
|
||||
// Command kindexr-cli is the admin CLI for kindexr.
|
||||
//
|
||||
// TODO (Phase 3): implement the following sub-commands:
|
||||
// - apikey create --label <name> --visibility all|wot|curated
|
||||
// - apikey list
|
||||
// - apikey revoke <key>
|
||||
// - relay add <url>
|
||||
// - relay remove <url>
|
||||
// - relay list
|
||||
// - publisher trust <pubkey>
|
||||
// - publisher block <pubkey>
|
||||
// - db stats
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
func main() {
|
||||
fmt.Println("kindexr-cli: not yet implemented")
|
||||
os.Exit(0)
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
// Command kindexr is the main daemon for the kindexr Nostr-native Torznab indexer.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"git.utn.lol/enki/kindexr/internal/config"
|
||||
"git.utn.lol/enki/kindexr/internal/db"
|
||||
"git.utn.lol/enki/kindexr/internal/server"
|
||||
)
|
||||
|
||||
// version is overridden at build time via -ldflags "-X main.version=<tag>".
|
||||
var version = "dev"
|
||||
|
||||
func main() {
|
||||
configPath := flag.String("config", "/etc/kindexr/config.yaml", "path to config file")
|
||||
flag.Parse()
|
||||
|
||||
// Load configuration.
|
||||
cfg, err := config.Load(*configPath)
|
||||
if err != nil {
|
||||
slog.Error("failed to load config", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
// Set up structured logging.
|
||||
var logLevel slog.Level
|
||||
switch cfg.Logging.Level {
|
||||
case "debug":
|
||||
logLevel = slog.LevelDebug
|
||||
case "warn", "warning":
|
||||
logLevel = slog.LevelWarn
|
||||
case "error":
|
||||
logLevel = slog.LevelError
|
||||
default:
|
||||
logLevel = slog.LevelInfo
|
||||
}
|
||||
|
||||
handlerOpts := &slog.HandlerOptions{Level: logLevel}
|
||||
var handler slog.Handler
|
||||
if cfg.Logging.Format == "json" {
|
||||
handler = slog.NewJSONHandler(os.Stderr, handlerOpts)
|
||||
} else {
|
||||
handler = slog.NewTextHandler(os.Stderr, handlerOpts)
|
||||
}
|
||||
logger := slog.New(handler)
|
||||
slog.SetDefault(logger)
|
||||
|
||||
// Open database.
|
||||
database, err := db.Open(cfg.Database.Path)
|
||||
if err != nil {
|
||||
slog.Error("failed to open database", "error", err, "path", cfg.Database.Path)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer database.Close()
|
||||
|
||||
migrationVer, err := database.MigrationVersion(context.Background())
|
||||
if err != nil {
|
||||
slog.Error("failed to query migration version", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
slog.Info("database ready", "path", cfg.Database.Path, "migration_version", migrationVer)
|
||||
|
||||
// Create HTTP server.
|
||||
srv := server.New(cfg, database, version)
|
||||
httpServer := &http.Server{
|
||||
Addr: cfg.Server.Listen,
|
||||
Handler: srv.Handler(),
|
||||
ReadTimeout: 30 * time.Second,
|
||||
WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
}
|
||||
|
||||
slog.Info("starting kindexr", "version", version, "listen", cfg.Server.Listen)
|
||||
|
||||
// Start listening in the background.
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
|
||||
go func() {
|
||||
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
slog.Error("http server error", "error", err)
|
||||
cancel()
|
||||
}
|
||||
}()
|
||||
|
||||
// Wait for shutdown signal.
|
||||
sigCh := make(chan os.Signal, 1)
|
||||
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
|
||||
|
||||
select {
|
||||
case sig := <-sigCh:
|
||||
slog.Info("received signal, shutting down", "signal", sig)
|
||||
case <-ctx.Done():
|
||||
}
|
||||
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer shutdownCancel()
|
||||
|
||||
if err := httpServer.Shutdown(shutdownCtx); err != nil {
|
||||
slog.Error("graceful shutdown failed", "error", err)
|
||||
}
|
||||
|
||||
slog.Info("kindexr stopped")
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
# kindexr config
|
||||
# Copy to /etc/kindexr/config.yaml and edit as needed.
|
||||
|
||||
server:
|
||||
listen: "127.0.0.1:9117" # bind addr; sit behind nginx for TLS
|
||||
base_url: "https://kindexr.example.com" # used in Torznab feed links
|
||||
|
||||
database:
|
||||
path: "/var/lib/kindexr/kindexr.db"
|
||||
|
||||
logging:
|
||||
level: "info" # debug|info|warn|error
|
||||
format: "json" # json|text
|
||||
|
||||
# Relays to subscribe to for NIP-35 events.
|
||||
# If empty on first run, defaults to a sane starter set.
|
||||
relays:
|
||||
- "wss://relay.damus.io"
|
||||
- "wss://nos.lol"
|
||||
- "wss://relay.primal.net"
|
||||
- "wss://nostr.mom"
|
||||
- "wss://relay.snort.social"
|
||||
- "wss://sovbit.host" # eric's own relay
|
||||
# add more
|
||||
|
||||
# Initial backfill via NIP-77 negentropy. Set false to start from "now" only.
|
||||
negentropy_bootstrap: true
|
||||
backfill_days: 365 # don't go further back than this
|
||||
|
||||
# Curation
|
||||
curation:
|
||||
# If true, only ingest events from pubkeys in your follow graph (within follow_depth hops).
|
||||
wot_only: false
|
||||
follow_depth: 2
|
||||
# Always allow these pubkeys regardless of WoT
|
||||
allowlist:
|
||||
- "npub1..."
|
||||
# Always block these
|
||||
blocklist:
|
||||
- "npub1..."
|
||||
# Auto-subscribe to these curation sets (kind 30004 naddr)
|
||||
curation_sets:
|
||||
- "naddr1..."
|
||||
|
||||
# TMDB enrichment (optional; without it, only events with imdb/tmdb i-tags are searchable by ID)
|
||||
tmdb:
|
||||
enabled: true
|
||||
api_key: "${TMDB_API_KEY}"
|
||||
cache_ttl: "168h"
|
||||
|
||||
# Health scraping (optional)
|
||||
health:
|
||||
enabled: false # off by default; rude to private trackers
|
||||
method: "dht" # dht|tracker|both
|
||||
refresh_interval: "30m"
|
||||
|
||||
# Writer side - publishing your own torrents to nostr
|
||||
publisher:
|
||||
enabled: false # off until you set it up explicitly
|
||||
signer:
|
||||
mode: "bunker" # local|ncryptsec|bunker
|
||||
bunker_uri: "bunker://..." # for NIP-46
|
||||
ncryptsec: "" # for ncryptsec mode
|
||||
nsec: "" # for local mode (NOT RECOMMENDED)
|
||||
outbox_relays:
|
||||
- "wss://sovbit.host"
|
||||
- "wss://relay.damus.io"
|
||||
- "wss://nos.lol"
|
||||
# Where to watch for completed downloads
|
||||
client:
|
||||
type: "qbittorrent" # qbittorrent|transmission|deluge|watch_dir
|
||||
qbittorrent:
|
||||
url: "http://127.0.0.1:8080"
|
||||
username: "admin"
|
||||
password: "${QBIT_PASSWORD}"
|
||||
# Only publish torrents tagged with this category (so you don't accidentally publish everything)
|
||||
category: "publish-nostr"
|
||||
watch_dir:
|
||||
path: "/var/lib/kindexr/watch"
|
||||
# Auto-enrich title parsing -> TMDB lookup before publishing
|
||||
enrich_before_publish: true
|
||||
@@ -0,0 +1,34 @@
|
||||
[Unit]
|
||||
Description=kindexr — Nostr-native Torznab indexer
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=kindexr
|
||||
Group=kindexr
|
||||
ExecStart=/usr/local/bin/kindexr --config /etc/kindexr/config.yaml
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/kindexr
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
LockPersonality=true
|
||||
MemoryDenyWriteExecute=true
|
||||
RestrictRealtime=true
|
||||
RestrictNamespaces=true
|
||||
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=kindexr
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -0,0 +1,73 @@
|
||||
# nginx reverse proxy configuration for nzbstr
|
||||
# Place this in /etc/nginx/sites-available/nzbstr and symlink to sites-enabled/.
|
||||
#
|
||||
# Prerequisites:
|
||||
# - nzbstr daemon listening on 127.0.0.1:9117
|
||||
# - TLS certificate (e.g. from Let's Encrypt via certbot)
|
||||
#
|
||||
# Usage:
|
||||
# cp nginx.conf.example /etc/nginx/sites-available/nzbstr
|
||||
# ln -s /etc/nginx/sites-available/nzbstr /etc/nginx/sites-enabled/
|
||||
# nginx -t && systemctl reload nginx
|
||||
|
||||
upstream nzbstr {
|
||||
server 127.0.0.1:9117;
|
||||
keepalive 32;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
listen [::]:80;
|
||||
server_name nzbstr.example.com;
|
||||
|
||||
# Redirect all HTTP to HTTPS.
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
listen [::]:443 ssl http2;
|
||||
server_name nzbstr.example.com;
|
||||
|
||||
# --- TLS (uncomment and adjust paths after running certbot) ---
|
||||
# ssl_certificate /etc/letsencrypt/live/nzbstr.example.com/fullchain.pem;
|
||||
# ssl_certificate_key /etc/letsencrypt/live/nzbstr.example.com/privkey.pem;
|
||||
# include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||
# ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||
# --- end TLS ---
|
||||
|
||||
# Security headers.
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
add_header X-XSS-Protection "1; mode=block" always;
|
||||
add_header Referrer-Policy "no-referrer" always;
|
||||
|
||||
# Logging.
|
||||
access_log /var/log/nginx/nzbstr_access.log;
|
||||
error_log /var/log/nginx/nzbstr_error.log;
|
||||
|
||||
# Proxy to nzbstr daemon.
|
||||
location / {
|
||||
proxy_pass http://nzbstr;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header Connection "";
|
||||
|
||||
# Torznab responses can be slow if searching a large index.
|
||||
proxy_read_timeout 60s;
|
||||
proxy_send_timeout 60s;
|
||||
}
|
||||
|
||||
# Health endpoint — allow from anywhere for monitoring.
|
||||
location /health {
|
||||
proxy_pass http://nzbstr/health;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
# Architecture
|
||||
|
||||
nzbstr is a two-halves daemon:
|
||||
|
||||
- **Reader** (Phase 1+): subscribes to Nostr relays for kind 2003 (torrent) and kind 2004 (torrent comment) events, indexes them locally in SQLite, and exposes a Torznab-compatible HTTP API.
|
||||
- **Writer** (Phase 4+): watches a download client (qBittorrent / Transmission / Deluge) for completed downloads, builds NIP-35 events signed by your npub, and publishes them to configured outbox relays.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Sonarr / Radarr / Lidarr / Readarr / Prowlarr
|
||||
|
|
||||
v Torznab HTTP/XML
|
||||
|
|
||||
+-----------+
|
||||
| nzbstr |
|
||||
| reader | <-- subscribes to relays via WebSocket
|
||||
| writer | --> publishes via WebSocket
|
||||
+-----------+
|
||||
^ ^
|
||||
| |
|
||||
v v
|
||||
Nostr relays qBittorrent / Transmission / Deluge
|
||||
(NIP-35 events) (download client)
|
||||
```
|
||||
|
||||
## Components
|
||||
|
||||
### `cmd/nzbstr` — main daemon
|
||||
|
||||
Entry point. Loads config, opens the SQLite database (applying any pending migrations), starts the HTTP server, and installs signal handlers for graceful shutdown.
|
||||
|
||||
### `internal/config`
|
||||
|
||||
koanf-based configuration loading. Load order: compiled-in defaults → YAML file → environment variables (`NZBSTR_` prefix). Missing files are silently skipped so the daemon can start with all defaults.
|
||||
|
||||
### `internal/db`
|
||||
|
||||
SQLite access via `modernc.org/sqlite` (pure Go, no CGO). Includes an embedded migration runner that applies numbered SQL files from `internal/db/migrations/` on startup. Uses WAL journal mode and a single writer connection for safe concurrent reads.
|
||||
|
||||
### `internal/server`
|
||||
|
||||
chi-based HTTP server. Phase 0 exposes only `/health`. Phase 1 adds `/api?t=caps`, `/api?t=search`, etc.
|
||||
|
||||
### `internal/nostr` (Phase 1+)
|
||||
|
||||
Relay subscription loop using `github.com/nbd-wtf/go-nostr`. NIP-77 negentropy bootstrap for efficient historical sync. Parses kind 2003 events into the internal `Torrent` struct and writes them to SQLite.
|
||||
|
||||
### `internal/torznab` (Phase 1+)
|
||||
|
||||
chi routes for the Torznab API. Auth middleware validates API keys. Returns XML in Torznab RSS format. Categories mapped from NIP-35 `i` tags to newznab integer codes.
|
||||
|
||||
### `internal/enrich` (Phase 2+)
|
||||
|
||||
Title parser (extracts season/episode/quality/source from release names) and TMDB client for backfilling IMDB/TMDB/TVDB IDs on events that lack them.
|
||||
|
||||
### `internal/wot` (Phase 3+)
|
||||
|
||||
Web-of-Trust filter. Fetches the operator's kind 3 follow list and optionally follows-of-follows to build an allowed-pubkey set. Integrates with NIP-51 curation sets.
|
||||
|
||||
### `internal/publisher` (Phase 4+)
|
||||
|
||||
qBittorrent / Transmission / watch-dir integration. Polls for completed downloads in the configured category, builds kind 2003 events, signs them via NIP-46 bunker or local key, and publishes to outbox relays.
|
||||
|
||||
## Storage
|
||||
|
||||
Single SQLite file at `/var/lib/nzbstr/nzbstr.db`. Schema managed by embedded SQL migrations in `internal/db/migrations/`. FTS5 virtual table on `(title, description)` with triggers to keep it in sync.
|
||||
|
||||
## Deployment
|
||||
|
||||
Single static binary + systemd unit. No external services required. Sits behind nginx for TLS termination. See `deploy/nzbstr.service` and `deploy/nginx.conf.example`.
|
||||
@@ -0,0 +1,111 @@
|
||||
# Publishing (Writer side)
|
||||
|
||||
nzbstr's writer side (Phase 4+) watches a download client for completed downloads and publishes them as NIP-35 kind 2003 events to configured Nostr relays.
|
||||
|
||||
## Overview
|
||||
|
||||
```
|
||||
qBittorrent / Transmission / watch_dir
|
||||
|
|
||||
v (poll completed downloads in tagged category)
|
||||
nzbstr publisher
|
||||
|
|
||||
v (build kind 2003 event)
|
||||
NIP-46 bunker signer OR local nsec
|
||||
|
|
||||
v (WebSocket publish)
|
||||
outbox_relays (e.g. wss://sovbit.host)
|
||||
|
|
||||
v (also stored locally)
|
||||
local SQLite DB
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Enable the publisher in `config.yaml`:
|
||||
|
||||
```yaml
|
||||
publisher:
|
||||
enabled: true
|
||||
signer:
|
||||
mode: "bunker"
|
||||
bunker_uri: "bunker://..."
|
||||
outbox_relays:
|
||||
- "wss://sovbit.host"
|
||||
- "wss://relay.damus.io"
|
||||
client:
|
||||
type: "qbittorrent"
|
||||
qbittorrent:
|
||||
url: "http://127.0.0.1:8080"
|
||||
username: "admin"
|
||||
password: "${QBIT_PASSWORD}"
|
||||
category: "publish-nostr"
|
||||
enrich_before_publish: true
|
||||
```
|
||||
|
||||
## Signing modes
|
||||
|
||||
### NIP-46 bunker (recommended)
|
||||
|
||||
Run a self-hosted NIP-46 bunker (e.g. [nsecbunker](https://github.com/kind-0/nsecbunker)) on a trusted machine. nzbstr connects as a client and requests signatures. Your nsec never touches the publishing machine.
|
||||
|
||||
```yaml
|
||||
signer:
|
||||
mode: "bunker"
|
||||
bunker_uri: "bunker://<pubkey>?relay=wss://relay.example.com&secret=<secret>"
|
||||
```
|
||||
|
||||
### ncryptsec (NIP-49)
|
||||
|
||||
Encrypted private key stored on disk. nzbstr prompts for the passphrase at startup.
|
||||
|
||||
```yaml
|
||||
signer:
|
||||
mode: "ncryptsec"
|
||||
ncryptsec: "ncryptsec1..."
|
||||
```
|
||||
|
||||
### Local nsec (not recommended)
|
||||
|
||||
Unencrypted private key. Only use on air-gapped or fully trusted machines.
|
||||
|
||||
```yaml
|
||||
signer:
|
||||
mode: "local"
|
||||
nsec: "nsec1..."
|
||||
```
|
||||
|
||||
Set the `nsec` value via environment variable instead of writing it to the config file:
|
||||
|
||||
```sh
|
||||
NZBSTR_PUBLISHER_SIGNER_NSEC=nsec1... nzbstr --config /etc/nzbstr/config.yaml
|
||||
```
|
||||
|
||||
## Download clients
|
||||
|
||||
### qBittorrent
|
||||
|
||||
nzbstr polls the qBittorrent Web API for torrents in the configured category (`publish-nostr` by default). Only torrents whose state is `pausedUP` (seeding completed) or `uploading` are considered.
|
||||
|
||||
Tag your torrents in qBittorrent with the publish category to opt them in. This prevents accidentally publishing everything in your client.
|
||||
|
||||
### watch_dir (Phase 4+)
|
||||
|
||||
Drop `.torrent` files into the configured watch directory. nzbstr picks them up, parses the metainfo, builds the event, and publishes. Useful for scripted workflows.
|
||||
|
||||
## Event construction
|
||||
|
||||
For each completed download:
|
||||
|
||||
1. Parse `.torrent` metainfo: extract info-hash, title, file list, tracker list, total size.
|
||||
2. If `enrich_before_publish: true`, run the title through the title parser and optionally TMDB lookup to backfill `imdb_id`, `tmdb_id`, `tvdb_id`, season, episode, quality, source.
|
||||
3. Build a kind 2003 event with the appropriate `x`, `title`, `file`, `tracker`, `i`, and `t` tags.
|
||||
4. Sign via the configured signer.
|
||||
5. Publish to all `outbox_relays`.
|
||||
6. Store in local SQLite as if ingested (so it appears in your own Torznab results immediately).
|
||||
|
||||
## Security notes
|
||||
|
||||
- **NIP-46 bunker on a public-facing seedbox** is the right answer security-wise. Run a self-hosted bunker on a trusted internal machine with nzbstr as a client. Your nsec never leaves the bunker.
|
||||
- If you use `local` mode, use `ProtectSystem=strict` in the systemd unit (already set) and ensure the config file has mode 0640 and is owned by the `nzbstr` user.
|
||||
- The `category` filter in qBittorrent config is important — do not set it to a category that contains torrents you didn't deliberately choose to publish.
|
||||
+102
@@ -0,0 +1,102 @@
|
||||
# Torznab API
|
||||
|
||||
nzbstr implements the [Torznab spec](https://torznab.github.io/spec-1.3-draft/index.html) — a superset of Newznab adapted for BitTorrent. All endpoints are under `/api`.
|
||||
|
||||
## Phase availability
|
||||
|
||||
| Endpoint | Phase |
|
||||
|---|---|
|
||||
| `GET /health` | Phase 0 |
|
||||
| `GET /api?t=caps` | Phase 1 |
|
||||
| `GET /api?t=search` | Phase 1 |
|
||||
| `GET /api?t=tvsearch` | Phase 2 |
|
||||
| `GET /api?t=movie` | Phase 2 |
|
||||
| `GET /api?t=music` | Phase 2 |
|
||||
| `GET /api?t=audio` | Phase 2 |
|
||||
| `GET /api?t=book` | Phase 2 |
|
||||
|
||||
## Authentication
|
||||
|
||||
Every endpoint except `/health` requires an `apikey` query parameter.
|
||||
|
||||
```
|
||||
GET /api?t=search&q=breaking+bad&apikey=<your-key>
|
||||
```
|
||||
|
||||
Missing or invalid key returns HTTP 200 with a Torznab error body:
|
||||
|
||||
```xml
|
||||
<error code="100" description="Incorrect user credentials"/>
|
||||
```
|
||||
|
||||
Generate keys with `nzbstr-cli apikey create --label sonarr`.
|
||||
|
||||
## `GET /api?t=caps`
|
||||
|
||||
Returns a capabilities XML document. Sonarr/Radarr call this on add to discover supported categories and search modes.
|
||||
|
||||
Supported search modes:
|
||||
- `search` — generic FTS5 over title and description
|
||||
- `tv-search` — `q`, `season`, `ep`, `imdbid`, `tvdbid`, `tmdbid`
|
||||
- `movie-search` — `q`, `imdbid`, `tmdbid`
|
||||
- `music-search` — `q`, `artist`, `album`, `year`
|
||||
- `audio-search` — `q`, `artist`, `album`, `year`
|
||||
- `book-search` — `q`, `author`, `title`
|
||||
|
||||
## `GET /api?t=search`
|
||||
|
||||
Parameters: `q`, `cat`, `limit` (default 50, max 100), `offset`, `apikey`.
|
||||
|
||||
`q` is passed directly to SQLite FTS5 against the `title` and `description` columns. Supports FTS5 query syntax (`AND`, `OR`, `NOT`, `"phrase"`, `prefix*`).
|
||||
|
||||
Returns Torznab RSS XML with `<item>` elements containing:
|
||||
- `<title>`, `<guid>`, `<link>` (magnet URL), `<pubDate>`, `<size>`, `<description>`
|
||||
- `<enclosure>` with magnet URL and file size
|
||||
- `<torznab:attr>` elements for category, infohash, magneturl, seeders, peers, upload/download factors, and ID tags
|
||||
|
||||
## `GET /api?t=tvsearch` (Phase 2+)
|
||||
|
||||
Additional parameters: `tvdbid`, `imdbid`, `tmdbid`, `season`, `ep`.
|
||||
|
||||
Structured ID parameters take priority over `q`. If `season` and/or `ep` are set, results are filtered by the parsed `season` / `episode` columns in the database.
|
||||
|
||||
## `GET /api?t=movie` (Phase 2+)
|
||||
|
||||
Additional parameters: `imdbid`, `tmdbid`.
|
||||
|
||||
## `GET /api?t=music` / `t=audio` (Phase 2+)
|
||||
|
||||
Additional parameters: `artist`, `album`, `year`.
|
||||
|
||||
## `GET /api?t=book` (Phase 2+)
|
||||
|
||||
Additional parameters: `author`, `title`.
|
||||
|
||||
## Categories (newznab codes)
|
||||
|
||||
| Code | Name |
|
||||
|---|---|
|
||||
| 2000 | Movies |
|
||||
| 2030 | Movies/SD |
|
||||
| 2040 | Movies/HD |
|
||||
| 2045 | Movies/UHD |
|
||||
| 2050 | Movies/BluRay |
|
||||
| 3000 | Audio |
|
||||
| 3010 | Audio/MP3 |
|
||||
| 3030 | Audio/Audiobook |
|
||||
| 3040 | Audio/Lossless |
|
||||
| 5000 | TV |
|
||||
| 5030 | TV/SD |
|
||||
| 5040 | TV/HD |
|
||||
| 5045 | TV/UHD |
|
||||
| 5070 | TV/Anime |
|
||||
| 7000 | Books |
|
||||
| 7020 | Books/EBook |
|
||||
| 7030 | Books/Comics |
|
||||
| 8000 | Other |
|
||||
|
||||
Categories are sourced from NIP-35 `i` tags (`newznab:NNNN` or `tcat:...`) and fall back to `8000` if unknown.
|
||||
|
||||
## XML format
|
||||
|
||||
nzbstr follows the Torznab RSS 2.0 format. The `xmlns:torznab` namespace is `http://torznab.com/schemas/2015/feed`. All `<torznab:attr>` elements use standard Newznab attribute names so existing *arr configurations work without modification.
|
||||
@@ -0,0 +1,27 @@
|
||||
module git.utn.lol/enki/kindexr
|
||||
|
||||
go 1.25.0
|
||||
|
||||
require (
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
||||
github.com/go-chi/chi/v5 v5.2.5 // indirect
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/knadh/koanf/maps v0.1.2 // indirect
|
||||
github.com/knadh/koanf/parsers/yaml v1.1.0 // indirect
|
||||
github.com/knadh/koanf/providers/env v1.1.0 // indirect
|
||||
github.com/knadh/koanf/providers/file v1.2.1 // indirect
|
||||
github.com/knadh/koanf/v2 v2.3.4 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mitchellh/copystructure v1.2.0 // indirect
|
||||
github.com/mitchellh/reflectwalk v1.0.2 // indirect
|
||||
github.com/ncruces/go-strftime v1.0.0 // indirect
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||
go.yaml.in/yaml/v3 v3.0.3 // indirect
|
||||
golang.org/x/sys v0.42.0 // indirect
|
||||
modernc.org/libc v1.72.3 // indirect
|
||||
modernc.org/mathutil v1.7.1 // indirect
|
||||
modernc.org/memory v1.11.0 // indirect
|
||||
modernc.org/sqlite v1.50.1 // indirect
|
||||
)
|
||||
@@ -0,0 +1,43 @@
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
||||
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
||||
github.com/go-chi/chi/v5 v5.2.5 h1:Eg4myHZBjyvJmAFjFvWgrqDTXFyOzjj7YIm3L3mu6Ug=
|
||||
github.com/go-chi/chi/v5 v5.2.5/go.mod h1:X7Gx4mteadT3eDOMTsXzmI4/rwUpOwBHLpAfupzFJP0=
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo=
|
||||
github.com/knadh/koanf/maps v0.1.2/go.mod h1:npD/QZY3V6ghQDdcQzl1W4ICNVTkohC8E73eI2xW4yI=
|
||||
github.com/knadh/koanf/parsers/yaml v1.1.0 h1:3ltfm9ljprAHt4jxgeYLlFPmUaunuCgu1yILuTXRdM4=
|
||||
github.com/knadh/koanf/parsers/yaml v1.1.0/go.mod h1:HHmcHXUrp9cOPcuC+2wrr44GTUB0EC+PyfN3HZD9tFg=
|
||||
github.com/knadh/koanf/providers/env v1.1.0 h1:U2VXPY0f+CsNDkvdsG8GcsnK4ah85WwWyJgef9oQMSc=
|
||||
github.com/knadh/koanf/providers/env v1.1.0/go.mod h1:QhHHHZ87h9JxJAn2czdEl6pdkNnDh/JS1Vtsyt65hTY=
|
||||
github.com/knadh/koanf/providers/file v1.2.1 h1:bEWbtQwYrA+W2DtdBrQWyXqJaJSG3KrP3AESOJYp9wM=
|
||||
github.com/knadh/koanf/providers/file v1.2.1/go.mod h1:bp1PM5f83Q+TOUu10J/0ApLBd9uIzg+n9UgthfY+nRA=
|
||||
github.com/knadh/koanf/v2 v2.3.4 h1:fnynNSDlujWE+v83hAp8wKr/cdoxHLO0629SN+U8Urc=
|
||||
github.com/knadh/koanf/v2 v2.3.4/go.mod h1:gRb40VRAbd4iJMYYD5IxZ6hfuopFcXBpc9bbQpZwo28=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
|
||||
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
|
||||
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
|
||||
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
|
||||
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
|
||||
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
|
||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
|
||||
go.yaml.in/yaml/v3 v3.0.3 h1:bXOww4E/J3f66rav3pX3m8w6jDE4knZjGOw8b5Y6iNE=
|
||||
go.yaml.in/yaml/v3 v3.0.3/go.mod h1:tBHosrYAkRZjRAOREWbDnBXUf08JOwYq++0QNwQiWzI=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
|
||||
golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
modernc.org/libc v1.72.3 h1:ZnDF4tXn4NBXFutMMQC4vtbTFSXhhKzR73fv0beZEAU=
|
||||
modernc.org/libc v1.72.3/go.mod h1:dn0dZNnnn1clLyvRxLxYExxiKRZIRENOfqQ8XEeg4Qs=
|
||||
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
|
||||
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
|
||||
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
|
||||
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
|
||||
modernc.org/sqlite v1.50.1 h1:l+cQvn0sd0zJJtfygGHuQJ5AjlrwXmWPw4KP3ZMwr9w=
|
||||
modernc.org/sqlite v1.50.1/go.mod h1:tcNzv5p84E0skkmJn038y+hWJbLQXQqEnQfeh5r2JLM=
|
||||
@@ -0,0 +1,280 @@
|
||||
// Package config provides koanf-based configuration loading for kindexr.
|
||||
// Load order: defaults → YAML file → env vars (KINDEXR_ prefix).
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/knadh/koanf/parsers/yaml"
|
||||
"github.com/knadh/koanf/providers/env"
|
||||
"github.com/knadh/koanf/providers/file"
|
||||
"github.com/knadh/koanf/v2"
|
||||
)
|
||||
|
||||
// ServerConfig holds HTTP server settings.
|
||||
type ServerConfig struct {
|
||||
Listen string `koanf:"listen"`
|
||||
BaseURL string `koanf:"base_url"`
|
||||
}
|
||||
|
||||
// DatabaseConfig holds SQLite settings.
|
||||
type DatabaseConfig struct {
|
||||
Path string `koanf:"path"`
|
||||
}
|
||||
|
||||
// LoggingConfig holds structured logging settings.
|
||||
type LoggingConfig struct {
|
||||
Level string `koanf:"level"`
|
||||
Format string `koanf:"format"`
|
||||
}
|
||||
|
||||
// CurationConfig controls ingestion filtering.
|
||||
type CurationConfig struct {
|
||||
WoTOnly bool `koanf:"wot_only"`
|
||||
FollowDepth int `koanf:"follow_depth"`
|
||||
Allowlist []string `koanf:"allowlist"`
|
||||
Blocklist []string `koanf:"blocklist"`
|
||||
CurationSets []string `koanf:"curation_sets"`
|
||||
OperatorPubkey string `koanf:"operator_pubkey"`
|
||||
}
|
||||
|
||||
// TMDBConfig controls TMDB enrichment.
|
||||
type TMDBConfig struct {
|
||||
Enabled bool `koanf:"enabled"`
|
||||
APIKey string `koanf:"api_key"`
|
||||
CacheTTL string `koanf:"cache_ttl"`
|
||||
}
|
||||
|
||||
// HealthConfig controls torrent health scraping.
|
||||
type HealthConfig struct {
|
||||
Enabled bool `koanf:"enabled"`
|
||||
Method string `koanf:"method"`
|
||||
RefreshInterval string `koanf:"refresh_interval"`
|
||||
}
|
||||
|
||||
// SignerConfig controls the Nostr signing backend.
|
||||
type SignerConfig struct {
|
||||
Mode string `koanf:"mode"`
|
||||
BunkerURI string `koanf:"bunker_uri"`
|
||||
Ncryptsec string `koanf:"ncryptsec"`
|
||||
Nsec string `koanf:"nsec"`
|
||||
}
|
||||
|
||||
// QBittorrentConfig holds qBittorrent Web API settings.
|
||||
type QBittorrentConfig struct {
|
||||
URL string `koanf:"url"`
|
||||
Username string `koanf:"username"`
|
||||
Password string `koanf:"password"`
|
||||
Category string `koanf:"category"`
|
||||
}
|
||||
|
||||
// WatchDirConfig holds watch-directory settings.
|
||||
type WatchDirConfig struct {
|
||||
Path string `koanf:"path"`
|
||||
}
|
||||
|
||||
// ClientConfig holds the download client settings.
|
||||
type ClientConfig struct {
|
||||
Type string `koanf:"type"`
|
||||
QBittorrent QBittorrentConfig `koanf:"qbittorrent"`
|
||||
WatchDir WatchDirConfig `koanf:"watch_dir"`
|
||||
}
|
||||
|
||||
// PublisherConfig controls the writer side (publishing your own events).
|
||||
type PublisherConfig struct {
|
||||
Enabled bool `koanf:"enabled"`
|
||||
Signer SignerConfig `koanf:"signer"`
|
||||
OutboxRelays []string `koanf:"outbox_relays"`
|
||||
Client ClientConfig `koanf:"client"`
|
||||
EnrichBeforePublish bool `koanf:"enrich_before_publish"`
|
||||
}
|
||||
|
||||
// Config is the top-level configuration structure.
|
||||
type Config struct {
|
||||
Server ServerConfig `koanf:"server"`
|
||||
Database DatabaseConfig `koanf:"database"`
|
||||
Logging LoggingConfig `koanf:"logging"`
|
||||
Relays []string `koanf:"relays"`
|
||||
NegentropyBootstrap bool `koanf:"negentropy_bootstrap"`
|
||||
BackfillDays int `koanf:"backfill_days"`
|
||||
Curation CurationConfig `koanf:"curation"`
|
||||
TMDB TMDBConfig `koanf:"tmdb"`
|
||||
Health HealthConfig `koanf:"health"`
|
||||
Publisher PublisherConfig `koanf:"publisher"`
|
||||
}
|
||||
|
||||
// defaultRelays is the starter set of public Nostr relays.
|
||||
var defaultRelays = []string{
|
||||
"wss://relay.damus.io",
|
||||
"wss://nos.lol",
|
||||
"wss://relay.primal.net",
|
||||
"wss://nostr.mom",
|
||||
"wss://relay.snort.social",
|
||||
"wss://sovbit.host",
|
||||
}
|
||||
|
||||
// Defaults returns a Config populated with sane default values.
|
||||
func Defaults() *Config {
|
||||
return &Config{
|
||||
Server: ServerConfig{
|
||||
Listen: "127.0.0.1:9117",
|
||||
BaseURL: "https://kindexr.example.com",
|
||||
},
|
||||
Database: DatabaseConfig{
|
||||
Path: "/var/lib/kindexr/kindexr.db",
|
||||
},
|
||||
Logging: LoggingConfig{
|
||||
Level: "info",
|
||||
Format: "json",
|
||||
},
|
||||
Relays: defaultRelays,
|
||||
NegentropyBootstrap: true,
|
||||
BackfillDays: 365,
|
||||
Curation: CurationConfig{
|
||||
WoTOnly: false,
|
||||
FollowDepth: 2,
|
||||
},
|
||||
TMDB: TMDBConfig{
|
||||
Enabled: true,
|
||||
CacheTTL: "168h",
|
||||
},
|
||||
Health: HealthConfig{
|
||||
Enabled: false,
|
||||
Method: "dht",
|
||||
RefreshInterval: "30m",
|
||||
},
|
||||
Publisher: PublisherConfig{
|
||||
Enabled: false,
|
||||
Signer: SignerConfig{
|
||||
Mode: "bunker",
|
||||
},
|
||||
OutboxRelays: []string{
|
||||
"wss://sovbit.host",
|
||||
"wss://relay.damus.io",
|
||||
"wss://nos.lol",
|
||||
},
|
||||
Client: ClientConfig{
|
||||
Type: "qbittorrent",
|
||||
QBittorrent: QBittorrentConfig{
|
||||
URL: "http://127.0.0.1:8080",
|
||||
Username: "admin",
|
||||
Category: "publish-nostr",
|
||||
},
|
||||
WatchDir: WatchDirConfig{
|
||||
Path: "/var/lib/kindexr/watch",
|
||||
},
|
||||
},
|
||||
EnrichBeforePublish: true,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Load reads config from defaults, then optionally from a YAML file, then from
|
||||
// environment variables with the NZBSTR_ prefix. If path is empty or the file
|
||||
// does not exist, the file step is silently skipped.
|
||||
func Load(path string) (*Config, error) {
|
||||
k := koanf.New(".")
|
||||
|
||||
defaults := Defaults()
|
||||
|
||||
// Apply defaults via Set calls.
|
||||
if err := k.Set("server.listen", defaults.Server.Listen); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("server.base_url", defaults.Server.BaseURL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("database.path", defaults.Database.Path); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("logging.level", defaults.Logging.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("logging.format", defaults.Logging.Format); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("relays", defaults.Relays); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("negentropy_bootstrap", defaults.NegentropyBootstrap); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("backfill_days", defaults.BackfillDays); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("curation.wot_only", defaults.Curation.WoTOnly); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("curation.follow_depth", defaults.Curation.FollowDepth); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("tmdb.enabled", defaults.TMDB.Enabled); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("tmdb.cache_ttl", defaults.TMDB.CacheTTL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("health.enabled", defaults.Health.Enabled); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("health.method", defaults.Health.Method); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("health.refresh_interval", defaults.Health.RefreshInterval); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.enabled", defaults.Publisher.Enabled); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.signer.mode", defaults.Publisher.Signer.Mode); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.outbox_relays", defaults.Publisher.OutboxRelays); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.client.type", defaults.Publisher.Client.Type); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.client.qbittorrent.url", defaults.Publisher.Client.QBittorrent.URL); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.client.qbittorrent.username", defaults.Publisher.Client.QBittorrent.Username); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.client.qbittorrent.category", defaults.Publisher.Client.QBittorrent.Category); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.client.watch_dir.path", defaults.Publisher.Client.WatchDir.Path); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := k.Set("publisher.enrich_before_publish", defaults.Publisher.EnrichBeforePublish); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Load YAML file if path is set and file exists.
|
||||
if path != "" {
|
||||
if _, err := os.Stat(path); err == nil {
|
||||
if err := k.Load(file.Provider(path), yaml.Parser()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
// If file does not exist, skip gracefully (no error).
|
||||
}
|
||||
|
||||
// Load env vars with KINDEXR_ prefix; map KINDEXR_LOGGING_LEVEL -> logging.level.
|
||||
// Ignore errors from env loading (no KINDEXR_ vars present is fine).
|
||||
_ = k.Load(env.Provider("KINDEXR_", ".", func(s string) string {
|
||||
// Strip prefix, lowercase, replace _ with .
|
||||
s = strings.TrimPrefix(s, "KINDEXR_")
|
||||
s = strings.ToLower(s)
|
||||
s = strings.ReplaceAll(s, "_", ".")
|
||||
return s
|
||||
}), nil)
|
||||
|
||||
cfg := &Config{}
|
||||
if err := k.Unmarshal("", cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"git.utn.lol/enki/kindexr/internal/config"
|
||||
)
|
||||
|
||||
func TestDefaults(t *testing.T) {
|
||||
cfg, err := config.Load("")
|
||||
if err != nil {
|
||||
t.Fatalf("Load with empty path failed: %v", err)
|
||||
}
|
||||
if cfg.Server.Listen != "127.0.0.1:9117" {
|
||||
t.Errorf("expected default listen 127.0.0.1:9117, got %q", cfg.Server.Listen)
|
||||
}
|
||||
if cfg.Logging.Format != "json" {
|
||||
t.Errorf("expected default log format json, got %q", cfg.Logging.Format)
|
||||
}
|
||||
if cfg.Logging.Level != "info" {
|
||||
t.Errorf("expected default log level info, got %q", cfg.Logging.Level)
|
||||
}
|
||||
if cfg.Database.Path != "/var/lib/kindexr/kindexr.db" {
|
||||
t.Errorf("expected default db path, got %q", cfg.Database.Path)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cfgFile := filepath.Join(dir, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
listen: "0.0.0.0:8888"
|
||||
logging:
|
||||
level: "debug"
|
||||
`
|
||||
if err := os.WriteFile(cfgFile, []byte(content), 0644); err != nil {
|
||||
t.Fatalf("failed to write temp config: %v", err)
|
||||
}
|
||||
|
||||
cfg, err := config.Load(cfgFile)
|
||||
if err != nil {
|
||||
t.Fatalf("Load from file failed: %v", err)
|
||||
}
|
||||
if cfg.Server.Listen != "0.0.0.0:8888" {
|
||||
t.Errorf("expected listen 0.0.0.0:8888, got %q", cfg.Server.Listen)
|
||||
}
|
||||
if cfg.Logging.Level != "debug" {
|
||||
t.Errorf("expected log level debug, got %q", cfg.Logging.Level)
|
||||
}
|
||||
// Non-overridden defaults should still be present.
|
||||
if cfg.Logging.Format != "json" {
|
||||
t.Errorf("expected default log format json, got %q", cfg.Logging.Format)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnvOverride(t *testing.T) {
|
||||
t.Setenv("KINDEXR_LOGGING_LEVEL", "warn")
|
||||
|
||||
cfg, err := config.Load("")
|
||||
if err != nil {
|
||||
t.Fatalf("Load failed: %v", err)
|
||||
}
|
||||
if cfg.Logging.Level != "warn" {
|
||||
t.Errorf("expected logging level warn from env, got %q", cfg.Logging.Level)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadMissingFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
nonExistent := filepath.Join(dir, "does-not-exist.yaml")
|
||||
|
||||
cfg, err := config.Load(nonExistent)
|
||||
if err != nil {
|
||||
t.Fatalf("Load with missing file should succeed, got error: %v", err)
|
||||
}
|
||||
// Should still have defaults.
|
||||
if cfg.Server.Listen != "127.0.0.1:9117" {
|
||||
t.Errorf("expected default listen, got %q", cfg.Server.Listen)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,275 @@
|
||||
// Package db provides SQLite database access and migration management for kindexr.
|
||||
package db
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"embed"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
_ "modernc.org/sqlite" // pure-Go SQLite driver, no CGO
|
||||
)
|
||||
|
||||
//go:embed migrations/*.sql
|
||||
var migrationsFS embed.FS
|
||||
|
||||
// DB wraps *sql.DB with nzbstr-specific helpers.
|
||||
type DB struct {
|
||||
*sql.DB
|
||||
}
|
||||
|
||||
// Open opens (or creates) the SQLite database at path, applies pending
|
||||
// migrations, and returns a ready-to-use *DB. The parent directory is created
|
||||
// if it does not exist.
|
||||
func Open(path string) (*DB, error) {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
||||
return nil, fmt.Errorf("db: create parent directory: %w", err)
|
||||
}
|
||||
|
||||
dsn := "file:" + path + "?_pragma=foreign_keys(1)&_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)"
|
||||
sqlDB, err := sql.Open("sqlite", dsn)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("db: open: %w", err)
|
||||
}
|
||||
|
||||
// SQLite WAL allows concurrent reads but serialises writes; one writer.
|
||||
sqlDB.SetMaxOpenConns(1)
|
||||
|
||||
if err := sqlDB.Ping(); err != nil {
|
||||
sqlDB.Close()
|
||||
return nil, fmt.Errorf("db: ping: %w", err)
|
||||
}
|
||||
|
||||
d := &DB{sqlDB}
|
||||
if err := d.migrate(); err != nil {
|
||||
sqlDB.Close()
|
||||
return nil, fmt.Errorf("db: migrate: %w", err)
|
||||
}
|
||||
return d, nil
|
||||
}
|
||||
|
||||
// migrate creates the schema_migrations table and applies any unapplied
|
||||
// migration files found in the embedded migrations/ directory.
|
||||
func (d *DB) migrate() error {
|
||||
// Ensure the migrations tracking table exists.
|
||||
_, err := d.DB.Exec(`CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
version INTEGER PRIMARY KEY,
|
||||
applied_at TEXT NOT NULL
|
||||
)`)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create schema_migrations: %w", err)
|
||||
}
|
||||
|
||||
// Find the current applied version.
|
||||
var currentVersion int
|
||||
row := d.DB.QueryRow(`SELECT COALESCE(MAX(version), 0) FROM schema_migrations`)
|
||||
if err := row.Scan(¤tVersion); err != nil {
|
||||
return fmt.Errorf("query current version: %w", err)
|
||||
}
|
||||
|
||||
// Read embedded migration files.
|
||||
entries, err := migrationsFS.ReadDir("migrations")
|
||||
if err != nil {
|
||||
return fmt.Errorf("read embedded migrations: %w", err)
|
||||
}
|
||||
|
||||
// Sort by filename so they apply in order.
|
||||
sort.Slice(entries, func(i, j int) bool {
|
||||
return entries[i].Name() < entries[j].Name()
|
||||
})
|
||||
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
name := entry.Name()
|
||||
if !strings.HasSuffix(name, ".sql") {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse version from prefix: "001_initial.sql" → 1
|
||||
parts := strings.SplitN(name, "_", 2)
|
||||
if len(parts) < 1 {
|
||||
continue
|
||||
}
|
||||
version, err := strconv.Atoi(parts[0])
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse version from %q: %w", name, err)
|
||||
}
|
||||
|
||||
if version <= currentVersion {
|
||||
continue // already applied
|
||||
}
|
||||
|
||||
sqlBytes, err := migrationsFS.ReadFile("migrations/" + name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read migration %q: %w", name, err)
|
||||
}
|
||||
|
||||
statements := splitStatements(string(sqlBytes))
|
||||
|
||||
tx, err := d.DB.Begin()
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin transaction for migration %d: %w", version, err)
|
||||
}
|
||||
|
||||
for _, stmt := range statements {
|
||||
stmt = strings.TrimSpace(stmt)
|
||||
if stmt == "" {
|
||||
continue
|
||||
}
|
||||
if _, err := tx.Exec(stmt); err != nil {
|
||||
tx.Rollback()
|
||||
return fmt.Errorf("migration %d, statement %q: %w", version, stmt[:min(len(stmt), 60)], err)
|
||||
}
|
||||
}
|
||||
|
||||
_, err = tx.Exec(
|
||||
`INSERT INTO schema_migrations(version, applied_at) VALUES (?, datetime('now'))`,
|
||||
version,
|
||||
)
|
||||
if err != nil {
|
||||
tx.Rollback()
|
||||
return fmt.Errorf("record migration %d: %w", version, err)
|
||||
}
|
||||
|
||||
if err := tx.Commit(); err != nil {
|
||||
return fmt.Errorf("commit migration %d: %w", version, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// splitStatements splits a SQL script into individual statements, handling
|
||||
// single-quoted strings, -- line comments, and BEGIN...END trigger bodies so
|
||||
// that semicolons inside them are not treated as statement boundaries.
|
||||
func splitStatements(sql string) []string {
|
||||
var statements []string
|
||||
var current strings.Builder
|
||||
var word strings.Builder
|
||||
inString := false
|
||||
depth := 0 // tracks BEGIN...END nesting for trigger bodies
|
||||
i := 0
|
||||
|
||||
flushWord := func() {
|
||||
switch strings.ToUpper(word.String()) {
|
||||
case "BEGIN":
|
||||
depth++
|
||||
case "END":
|
||||
depth--
|
||||
}
|
||||
word.Reset()
|
||||
}
|
||||
|
||||
for i < len(sql) {
|
||||
ch := sql[i]
|
||||
|
||||
if inString {
|
||||
current.WriteByte(ch)
|
||||
if ch == '\'' {
|
||||
if i+1 < len(sql) && sql[i+1] == '\'' {
|
||||
current.WriteByte(sql[i+1])
|
||||
i += 2
|
||||
continue
|
||||
}
|
||||
inString = false
|
||||
}
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// Outside a string.
|
||||
if ch == '\'' {
|
||||
flushWord()
|
||||
inString = true
|
||||
current.WriteByte(ch)
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// Line comment: skip to end of line.
|
||||
if ch == '-' && i+1 < len(sql) && sql[i+1] == '-' {
|
||||
flushWord()
|
||||
for i < len(sql) && sql[i] != '\n' {
|
||||
i++
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// Accumulate identifier/keyword characters.
|
||||
if isWordChar(ch) {
|
||||
word.WriteByte(ch)
|
||||
current.WriteByte(ch)
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
// Non-word character: finalize any pending keyword.
|
||||
flushWord()
|
||||
|
||||
if ch == ';' {
|
||||
if depth == 0 {
|
||||
stmt := strings.TrimSpace(current.String())
|
||||
if stmt != "" {
|
||||
statements = append(statements, stmt)
|
||||
}
|
||||
current.Reset()
|
||||
} else {
|
||||
current.WriteByte(ch)
|
||||
}
|
||||
i++
|
||||
continue
|
||||
}
|
||||
|
||||
current.WriteByte(ch)
|
||||
i++
|
||||
}
|
||||
|
||||
// Catch any trailing statement without a trailing semicolon.
|
||||
flushWord()
|
||||
if stmt := strings.TrimSpace(current.String()); stmt != "" {
|
||||
statements = append(statements, stmt)
|
||||
}
|
||||
|
||||
return statements
|
||||
}
|
||||
|
||||
func isWordChar(ch byte) bool {
|
||||
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || (ch >= '0' && ch <= '9') || ch == '_'
|
||||
}
|
||||
|
||||
// Stats holds aggregate statistics about the indexed content.
|
||||
type Stats struct {
|
||||
EventsTotal int64
|
||||
LastEventAt *int64
|
||||
}
|
||||
|
||||
// GetStats returns aggregate statistics from the torrents table.
|
||||
func (d *DB) GetStats(ctx context.Context) (Stats, error) {
|
||||
row := d.DB.QueryRowContext(ctx, `SELECT COUNT(*), MAX(created_at) FROM torrents`)
|
||||
var s Stats
|
||||
var lastEventAt sql.NullInt64
|
||||
if err := row.Scan(&s.EventsTotal, &lastEventAt); err != nil {
|
||||
return Stats{}, fmt.Errorf("get stats: %w", err)
|
||||
}
|
||||
if lastEventAt.Valid {
|
||||
s.LastEventAt = &lastEventAt.Int64
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// MigrationVersion returns the highest applied migration version.
|
||||
func (d *DB) MigrationVersion(ctx context.Context) (int, error) {
|
||||
var version int
|
||||
row := d.DB.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM schema_migrations`)
|
||||
if err := row.Scan(&version); err != nil {
|
||||
return 0, fmt.Errorf("migration version: %w", err)
|
||||
}
|
||||
return version, nil
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
package db_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"git.utn.lol/enki/kindexr/internal/db"
|
||||
)
|
||||
|
||||
func tempDB(t *testing.T) *db.DB {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "test.db")
|
||||
d, err := db.Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("Open failed: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { d.Close() })
|
||||
return d
|
||||
}
|
||||
|
||||
func TestOpen(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "test.db")
|
||||
d, err := db.Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("Open returned error: %v", err)
|
||||
}
|
||||
d.Close()
|
||||
}
|
||||
|
||||
func TestMigrationsApplied(t *testing.T) {
|
||||
d := tempDB(t)
|
||||
ctx := context.Background()
|
||||
version, err := d.MigrationVersion(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("MigrationVersion error: %v", err)
|
||||
}
|
||||
if version < 1 {
|
||||
t.Errorf("expected at least 1 migration applied, got version %d", version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMigrationsIdempotent(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "test.db")
|
||||
|
||||
d1, err := db.Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("first Open failed: %v", err)
|
||||
}
|
||||
d1.Close()
|
||||
|
||||
d2, err := db.Open(path)
|
||||
if err != nil {
|
||||
t.Fatalf("second Open failed (idempotency): %v", err)
|
||||
}
|
||||
d2.Close()
|
||||
}
|
||||
|
||||
func TestSchema(t *testing.T) {
|
||||
d := tempDB(t)
|
||||
|
||||
expectedTables := []string{
|
||||
"torrents", "files", "trackers", "tags",
|
||||
"publishers", "relays", "api_keys", "health", "comments",
|
||||
}
|
||||
|
||||
for _, table := range expectedTables {
|
||||
var name string
|
||||
err := d.QueryRow(
|
||||
`SELECT name FROM sqlite_master WHERE type='table' AND name=?`, table,
|
||||
).Scan(&name)
|
||||
if err == sql.ErrNoRows {
|
||||
t.Errorf("expected table %q to exist, but it does not", table)
|
||||
} else if err != nil {
|
||||
t.Errorf("querying for table %q: %v", table, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFTS5(t *testing.T) {
|
||||
d := tempDB(t)
|
||||
|
||||
// Insert a torrent row.
|
||||
_, err := d.Exec(`
|
||||
INSERT INTO torrents
|
||||
(event_id, info_hash, pubkey, created_at, ingested_at, title, description, raw_event)
|
||||
VALUES
|
||||
('evt1', 'aabbcc', 'pubkey1', 1715000000, 1715000001, 'test torrent title', 'test description', '{}')
|
||||
`)
|
||||
if err != nil {
|
||||
t.Fatalf("insert torrent: %v", err)
|
||||
}
|
||||
|
||||
// FTS5 search.
|
||||
var rowid int64
|
||||
err = d.QueryRow(`SELECT rowid FROM torrents_fts WHERE torrents_fts MATCH 'test'`).Scan(&rowid)
|
||||
if err == sql.ErrNoRows {
|
||||
t.Error("FTS5 search for 'test' returned no rows; trigger may not be working")
|
||||
} else if err != nil {
|
||||
t.Fatalf("FTS5 search error: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetStats(t *testing.T) {
|
||||
d := tempDB(t)
|
||||
ctx := context.Background()
|
||||
|
||||
stats, err := d.GetStats(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("GetStats error: %v", err)
|
||||
}
|
||||
if stats.EventsTotal != 0 {
|
||||
t.Errorf("expected EventsTotal=0, got %d", stats.EventsTotal)
|
||||
}
|
||||
if stats.LastEventAt != nil {
|
||||
t.Errorf("expected LastEventAt=nil, got %v", stats.LastEventAt)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
CREATE TABLE torrents (
|
||||
event_id TEXT PRIMARY KEY,
|
||||
info_hash TEXT NOT NULL,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
ingested_at INTEGER NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
description TEXT,
|
||||
size_bytes INTEGER,
|
||||
category TEXT,
|
||||
newznab_cat INTEGER,
|
||||
imdb_id TEXT,
|
||||
tmdb_id TEXT,
|
||||
tvdb_id TEXT,
|
||||
season INTEGER,
|
||||
episode INTEGER,
|
||||
quality TEXT,
|
||||
source TEXT,
|
||||
raw_event TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX idx_torrents_event_id ON torrents(event_id);
|
||||
CREATE INDEX idx_torrents_info_hash ON torrents(info_hash);
|
||||
CREATE INDEX idx_torrents_pubkey ON torrents(pubkey);
|
||||
CREATE INDEX idx_torrents_imdb ON torrents(imdb_id) WHERE imdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_tmdb ON torrents(tmdb_id) WHERE tmdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_tvdb ON torrents(tvdb_id) WHERE tvdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_created ON torrents(created_at DESC);
|
||||
CREATE INDEX idx_torrents_cat ON torrents(newznab_cat);
|
||||
|
||||
CREATE VIRTUAL TABLE torrents_fts USING fts5(
|
||||
title, description,
|
||||
content='torrents',
|
||||
content_rowid='rowid',
|
||||
tokenize='unicode61 remove_diacritics 2'
|
||||
);
|
||||
|
||||
CREATE TRIGGER torrents_ai AFTER INSERT ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER torrents_ad AFTER DELETE ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description);
|
||||
END;
|
||||
|
||||
CREATE TRIGGER torrents_au AFTER UPDATE ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description);
|
||||
INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description);
|
||||
END;
|
||||
|
||||
CREATE TABLE files (
|
||||
event_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
path TEXT NOT NULL,
|
||||
size_bytes INTEGER,
|
||||
PRIMARY KEY (event_id, idx),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE TABLE trackers (
|
||||
event_id TEXT NOT NULL,
|
||||
url TEXT NOT NULL,
|
||||
PRIMARY KEY (event_id, url),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE TABLE tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag TEXT NOT NULL,
|
||||
PRIMARY KEY (event_id, tag),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE INDEX idx_tags_tag ON tags(tag);
|
||||
|
||||
CREATE TABLE publishers (
|
||||
pubkey TEXT PRIMARY KEY,
|
||||
npub TEXT,
|
||||
name TEXT,
|
||||
trust REAL DEFAULT 0,
|
||||
notes TEXT,
|
||||
blocked INTEGER DEFAULT 0,
|
||||
torrents_n INTEGER DEFAULT 0,
|
||||
first_seen INTEGER,
|
||||
last_seen INTEGER
|
||||
);
|
||||
|
||||
CREATE TABLE relays (
|
||||
url TEXT PRIMARY KEY,
|
||||
enabled INTEGER DEFAULT 1,
|
||||
last_event INTEGER,
|
||||
last_sync INTEGER,
|
||||
notes TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE api_keys (
|
||||
key TEXT PRIMARY KEY,
|
||||
label TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
visibility TEXT NOT NULL DEFAULT 'all',
|
||||
curation_set TEXT,
|
||||
last_used INTEGER
|
||||
);
|
||||
|
||||
CREATE TABLE health (
|
||||
info_hash TEXT PRIMARY KEY,
|
||||
seeders INTEGER,
|
||||
leechers INTEGER,
|
||||
checked_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE comments (
|
||||
event_id TEXT PRIMARY KEY,
|
||||
torrent_event_id TEXT NOT NULL,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
content TEXT NOT NULL,
|
||||
raw_event TEXT NOT NULL,
|
||||
FOREIGN KEY (torrent_event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
@@ -0,0 +1,77 @@
|
||||
// Package server provides the HTTP server for kindexr, including the /health endpoint.
|
||||
package server
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
|
||||
"github.com/go-chi/chi/v5"
|
||||
"github.com/go-chi/chi/v5/middleware"
|
||||
"git.utn.lol/enki/kindexr/internal/config"
|
||||
"git.utn.lol/enki/kindexr/internal/db"
|
||||
)
|
||||
|
||||
// Server holds the HTTP server state.
|
||||
type Server struct {
|
||||
cfg *config.Config
|
||||
db *db.DB
|
||||
router chi.Router
|
||||
version string
|
||||
}
|
||||
|
||||
// New creates a new Server with the given config, database, and version string.
|
||||
func New(cfg *config.Config, database *db.DB, version string) *Server {
|
||||
s := &Server{
|
||||
cfg: cfg,
|
||||
db: database,
|
||||
version: version,
|
||||
}
|
||||
s.router = s.buildRouter()
|
||||
return s
|
||||
}
|
||||
|
||||
// Handler returns the root http.Handler for the server.
|
||||
func (s *Server) Handler() http.Handler {
|
||||
return s.router
|
||||
}
|
||||
|
||||
// buildRouter wires up all routes and middleware.
|
||||
func (s *Server) buildRouter() chi.Router {
|
||||
r := chi.NewRouter()
|
||||
r.Use(middleware.Logger)
|
||||
r.Use(middleware.Recoverer)
|
||||
|
||||
r.Get("/health", s.healthHandler)
|
||||
|
||||
return r
|
||||
}
|
||||
|
||||
// healthResponse is the JSON body returned by GET /health.
|
||||
type healthResponse struct {
|
||||
Status string `json:"status"`
|
||||
Version string `json:"version"`
|
||||
RelaysConnected int `json:"relays_connected"`
|
||||
EventsTotal int64 `json:"events_total"`
|
||||
LastEventAt *int64 `json:"last_event_at"`
|
||||
}
|
||||
|
||||
// healthHandler handles GET /health.
|
||||
func (s *Server) healthHandler(w http.ResponseWriter, r *http.Request) {
|
||||
stats, err := s.db.GetStats(r.Context())
|
||||
if err != nil {
|
||||
http.Error(w, `{"error":"internal server error"}`, http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
resp := healthResponse{
|
||||
Status: "ok",
|
||||
Version: s.version,
|
||||
RelaysConnected: 0, // populated in Phase 1 when relay connections are active
|
||||
EventsTotal: stats.EventsTotal,
|
||||
LastEventAt: stats.LastEventAt,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_ = json.NewEncoder(w).Encode(resp)
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package server_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.utn.lol/enki/kindexr/internal/config"
|
||||
"git.utn.lol/enki/kindexr/internal/db"
|
||||
"git.utn.lol/enki/kindexr/internal/server"
|
||||
)
|
||||
|
||||
func newTestServer(t *testing.T) *httptest.Server {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
database, err := db.Open(filepath.Join(dir, "test.db"))
|
||||
if err != nil {
|
||||
t.Fatalf("open test db: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { database.Close() })
|
||||
|
||||
cfg, err := config.Load("")
|
||||
if err != nil {
|
||||
t.Fatalf("load config: %v", err)
|
||||
}
|
||||
|
||||
srv := server.New(cfg, database, "0.1.0-test")
|
||||
return httptest.NewServer(srv.Handler())
|
||||
}
|
||||
|
||||
func TestHealthEndpoint(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
defer ts.Close()
|
||||
|
||||
resp, err := http.Get(ts.URL + "/health")
|
||||
if err != nil {
|
||||
t.Fatalf("GET /health: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Errorf("expected status 200, got %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var body map[string]interface{}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||
t.Fatalf("decode JSON: %v", err)
|
||||
}
|
||||
|
||||
if body["status"] != "ok" {
|
||||
t.Errorf("expected status=ok, got %v", body["status"])
|
||||
}
|
||||
if _, ok := body["version"]; !ok {
|
||||
t.Error("expected 'version' field in response")
|
||||
}
|
||||
if rc, ok := body["relays_connected"]; !ok {
|
||||
t.Error("expected 'relays_connected' field in response")
|
||||
} else if rc.(float64) != 0 {
|
||||
t.Errorf("expected relays_connected=0, got %v", rc)
|
||||
}
|
||||
if et, ok := body["events_total"]; !ok {
|
||||
t.Error("expected 'events_total' field in response")
|
||||
} else if et.(float64) != 0 {
|
||||
t.Errorf("expected events_total=0, got %v", et)
|
||||
}
|
||||
// last_event_at should be present (null is fine for empty DB).
|
||||
if _, ok := body["last_event_at"]; !ok {
|
||||
t.Error("expected 'last_event_at' field in response")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthContentType(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
defer ts.Close()
|
||||
|
||||
resp, err := http.Get(ts.URL + "/health")
|
||||
if err != nil {
|
||||
t.Fatalf("GET /health: %v", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
ct := resp.Header.Get("Content-Type")
|
||||
if !strings.HasPrefix(ct, "application/json") {
|
||||
t.Errorf("expected Content-Type application/json, got %q", ct)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHealthMethod(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
defer ts.Close()
|
||||
|
||||
// GET should return 200.
|
||||
resp, err := http.Get(ts.URL + "/health")
|
||||
if err != nil {
|
||||
t.Fatalf("GET /health: %v", err)
|
||||
}
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Errorf("GET /health: expected 200, got %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
// POST should return 405.
|
||||
resp, err = http.Post(ts.URL+"/health", "application/json", nil)
|
||||
if err != nil {
|
||||
t.Fatalf("POST /health: %v", err)
|
||||
}
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusMethodNotAllowed {
|
||||
t.Errorf("POST /health: expected 405, got %d", resp.StatusCode)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,744 @@
|
||||
# nzbstr — Nostr-Native Torznab Indexer
|
||||
|
||||
> Working name. Alternatives: `dtanr`, `torstr`, `arrostr`, `noindex`. Rename before v0.2 ships if you're going to.
|
||||
|
||||
## What this is
|
||||
|
||||
A daemon that bridges NIP-35 torrent events on the Nostr relay network into the Torznab API that Sonarr / Radarr / Lidarr / Readarr / Prowlarr already speak. Two halves:
|
||||
|
||||
- **Reader (Phase 1+):** subscribes to kind 2003 (torrent) and kind 2004 (torrent comment) events on configured relays, indexes them locally, exposes a Torznab-compatible HTTP API.
|
||||
- **Writer (Phase 4+):** watches a torrent client (qBittorrent / Transmission / Deluge) for completed downloads, builds and publishes NIP-35 events signed by your npub.
|
||||
|
||||
Positioning: **same slot as Jackett or Prowlarr.** Not a frontend like dtan.xyz; not a relay; not a downloader. Middleware that sits between Nostr and the *arr automation stack.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No web browse UI for humans in v1. dtan.xyz and nostrudel/torrents already do that. A browse UI may come later as a nice-to-have; it is not the product.
|
||||
- No new download protocol. BitTorrent is unchanged. Magnet links and `.torrent` files come out of nzbstr exactly as they go in.
|
||||
- No relay implementation. nzbstr is a relay *client*. If you want a relay, run strfry.
|
||||
- No replacement for Sonarr/Radarr/Lidarr. nzbstr sits underneath them.
|
||||
- No login system, no accounts, no web admin in v1. Config file + API keys.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Sonarr / Radarr / Lidarr / Readarr / Prowlarr
|
||||
|
|
||||
v Torznab HTTP/XML
|
||||
|
|
||||
+-----------+
|
||||
| nzbstr |
|
||||
| reader | <-- subscribes to relays via WebSocket
|
||||
| writer | --> publishes via WebSocket
|
||||
+-----------+
|
||||
^ ^
|
||||
| |
|
||||
v v
|
||||
Nostr relays qBittorrent / Transmission / Deluge
|
||||
(NIP-35 events) (download client)
|
||||
```
|
||||
|
||||
## Tech stack (locked)
|
||||
|
||||
These are decisions. Don't litigate them in Phase 1.
|
||||
|
||||
| Concern | Choice | Why |
|
||||
|---|---|---|
|
||||
| Language | Go 1.22+ | Mature nostr libraries, single static binary, matches existing nostr-poster pattern, systemd-friendly |
|
||||
| Nostr client | `github.com/nbd-wtf/go-nostr` | De facto Go nostr lib, NIP-77 negentropy, NIP-42 AUTH, NIP-46 bunker support |
|
||||
| Storage | SQLite + FTS5 | Single-file backup, no separate service, FTS5 handles millions of rows fine |
|
||||
| SQLite driver | `modernc.org/sqlite` | Pure Go, no CGO, simpler cross-compile |
|
||||
| HTTP router | `github.com/go-chi/chi/v5` | Lightweight, good middleware ergonomics |
|
||||
| Torrent parsing | `github.com/anacrolix/torrent/metainfo` | Reference implementation, computes info-hash correctly |
|
||||
| Config | `github.com/knadh/koanf/v2` | Cleaner than viper, supports YAML+env+flags |
|
||||
| Logging | `log/slog` (stdlib) | Structured, JSON for Loki/journald |
|
||||
| TMDB | `github.com/cyruzin/golang-tmdb` | For metadata enrichment when events lack IDs |
|
||||
| Deployment | systemd + single binary | Same shape as everything else on his boxes |
|
||||
|
||||
Postgres is explicitly *not* used in v1. SQLite for everything. Reconsider only if scale becomes a real problem (>5M events indexed or >100 concurrent Torznab clients).
|
||||
|
||||
## Repository layout
|
||||
|
||||
```
|
||||
nzbstr/
|
||||
├── cmd/
|
||||
│ ├── nzbstr/ # main daemon binary
|
||||
│ │ └── main.go
|
||||
│ └── nzbstr-cli/ # admin CLI (relay add/remove, publisher trust, db stats)
|
||||
│ └── main.go
|
||||
├── internal/
|
||||
│ ├── config/ # koanf-based config loading
|
||||
│ ├── db/ # SQLite, migrations, queries
|
||||
│ │ ├── migrations/
|
||||
│ │ └── queries.sql # sqlc-style if you go that route, or hand-rolled
|
||||
│ ├── nostr/
|
||||
│ │ ├── reader.go # relay subscription loop, NIP-77 negentropy bootstrap
|
||||
│ │ ├── writer.go # event publishing
|
||||
│ │ ├── signer.go # local nsec / NIP-49 ncryptsec / NIP-46 bunker
|
||||
│ │ └── parser.go # kind 2003 -> Torrent struct
|
||||
│ ├── torznab/
|
||||
│ │ ├── server.go # chi routes, auth middleware
|
||||
│ │ ├── caps.go # t=caps response
|
||||
│ │ ├── search.go # t=search, tvsearch, movie, music, book
|
||||
│ │ ├── xml.go # torznab XML marshalling
|
||||
│ │ └── categories.go # newznab category mapping
|
||||
│ ├── enrich/
|
||||
│ │ ├── tmdb.go # TMDB lookups for missing IDs
|
||||
│ │ └── parser.go # parse "Show.Name.S01E02.1080p" into structured fields
|
||||
│ ├── health/
|
||||
│ │ ├── tracker.go # UDP/HTTP tracker scrape (optional)
|
||||
│ │ └── dht.go # DHT peer-count scrape (optional)
|
||||
│ ├── publisher/
|
||||
│ │ ├── qbittorrent.go # qBit Web API client
|
||||
│ │ ├── transmission.go # Transmission RPC client
|
||||
│ │ └── watcher.go # event loop: on download complete -> publish
|
||||
│ └── wot/
|
||||
│ ├── follows.go # build allowed-pubkey set from kind 3 events
|
||||
│ └── trust.go # per-pubkey trust scoring
|
||||
├── deploy/
|
||||
│ ├── nzbstr.service # systemd unit
|
||||
│ ├── nzbstr.example.yaml # commented config template
|
||||
│ └── nginx.conf.example # reverse proxy with TLS
|
||||
├── docs/
|
||||
│ ├── ARCHITECTURE.md
|
||||
│ ├── TORZNAB.md # which subset of Torznab is supported
|
||||
│ └── PUBLISHING.md # how the writer side works
|
||||
├── go.mod
|
||||
├── go.sum
|
||||
├── Makefile
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Database schema
|
||||
|
||||
Single SQLite file at `/var/lib/nzbstr/nzbstr.db`. Migrations as numbered SQL files under `internal/db/migrations/`. Apply on startup; refuse to start on migration error.
|
||||
|
||||
```sql
|
||||
-- 001_initial.sql
|
||||
|
||||
CREATE TABLE torrents (
|
||||
event_id TEXT PRIMARY KEY, -- nostr event id (hex)
|
||||
info_hash TEXT NOT NULL, -- v1 info-hash (hex, lowercase)
|
||||
pubkey TEXT NOT NULL, -- publisher pubkey (hex)
|
||||
created_at INTEGER NOT NULL, -- nostr event created_at
|
||||
ingested_at INTEGER NOT NULL, -- when we saw it
|
||||
title TEXT NOT NULL,
|
||||
description TEXT,
|
||||
size_bytes INTEGER, -- sum of file sizes if no top-level size
|
||||
category TEXT, -- normalized: movie/tv/music/book/audio/xxx/other
|
||||
newznab_cat INTEGER, -- 2000/3000/5000/7000/etc
|
||||
imdb_id TEXT, -- e.g. "tt15239678"
|
||||
tmdb_id TEXT, -- "movie:693134" or "tv:1396"
|
||||
tvdb_id TEXT,
|
||||
season INTEGER, -- parsed from title or i tags
|
||||
episode INTEGER,
|
||||
quality TEXT, -- 480p/720p/1080p/2160p/SD/HD/UHD
|
||||
source TEXT, -- WEB-DL/BluRay/HDTV/REMUX
|
||||
raw_event TEXT NOT NULL -- full JSON for re-parsing
|
||||
);
|
||||
CREATE UNIQUE INDEX idx_torrents_event_id ON torrents(event_id);
|
||||
CREATE INDEX idx_torrents_info_hash ON torrents(info_hash);
|
||||
CREATE INDEX idx_torrents_pubkey ON torrents(pubkey);
|
||||
CREATE INDEX idx_torrents_imdb ON torrents(imdb_id) WHERE imdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_tmdb ON torrents(tmdb_id) WHERE tmdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_tvdb ON torrents(tvdb_id) WHERE tvdb_id IS NOT NULL;
|
||||
CREATE INDEX idx_torrents_created ON torrents(created_at DESC);
|
||||
CREATE INDEX idx_torrents_cat ON torrents(newznab_cat);
|
||||
|
||||
-- FTS index for text search
|
||||
CREATE VIRTUAL TABLE torrents_fts USING fts5(
|
||||
title, description,
|
||||
content='torrents',
|
||||
content_rowid='rowid',
|
||||
tokenize='unicode61 remove_diacritics 2'
|
||||
);
|
||||
-- triggers to keep FTS in sync (insert/update/delete)
|
||||
CREATE TRIGGER torrents_ai AFTER INSERT ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description);
|
||||
END;
|
||||
CREATE TRIGGER torrents_ad AFTER DELETE ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description);
|
||||
END;
|
||||
CREATE TRIGGER torrents_au AFTER UPDATE ON torrents BEGIN
|
||||
INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description);
|
||||
INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description);
|
||||
END;
|
||||
|
||||
CREATE TABLE files (
|
||||
event_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL, -- file order within torrent
|
||||
path TEXT NOT NULL,
|
||||
size_bytes INTEGER,
|
||||
PRIMARY KEY (event_id, idx),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE TABLE trackers (
|
||||
event_id TEXT NOT NULL,
|
||||
url TEXT NOT NULL,
|
||||
PRIMARY KEY (event_id, url),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
|
||||
CREATE TABLE tags (
|
||||
event_id TEXT NOT NULL,
|
||||
tag TEXT NOT NULL, -- e.g. "movie", "4k", "REMUX"
|
||||
PRIMARY KEY (event_id, tag),
|
||||
FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
CREATE INDEX idx_tags_tag ON tags(tag);
|
||||
|
||||
CREATE TABLE publishers (
|
||||
pubkey TEXT PRIMARY KEY, -- hex
|
||||
npub TEXT, -- bech32 cached for display
|
||||
name TEXT, -- from kind 0 if available
|
||||
trust REAL DEFAULT 0, -- -1.0 .. 1.0
|
||||
notes TEXT,
|
||||
blocked INTEGER DEFAULT 0, -- 0/1
|
||||
torrents_n INTEGER DEFAULT 0,
|
||||
first_seen INTEGER,
|
||||
last_seen INTEGER
|
||||
);
|
||||
|
||||
CREATE TABLE relays (
|
||||
url TEXT PRIMARY KEY,
|
||||
enabled INTEGER DEFAULT 1,
|
||||
last_event INTEGER, -- created_at of most recent event from this relay
|
||||
last_sync INTEGER, -- when we last successfully connected
|
||||
notes TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE api_keys (
|
||||
key TEXT PRIMARY KEY, -- random 32-byte hex
|
||||
label TEXT NOT NULL, -- "sonarr", "radarr", "friend-jane"
|
||||
created_at INTEGER NOT NULL,
|
||||
-- visibility filter: which pubkeys this key can see
|
||||
visibility TEXT NOT NULL DEFAULT 'all', -- 'all' | 'wot' | 'curated'
|
||||
curation_set TEXT, -- naddr1... of a kind 30004 set, if visibility='curated'
|
||||
last_used INTEGER
|
||||
);
|
||||
|
||||
CREATE TABLE health (
|
||||
info_hash TEXT PRIMARY KEY,
|
||||
seeders INTEGER,
|
||||
leechers INTEGER,
|
||||
checked_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
-- Comments (kind 2004) - lower priority, store but don't surface in Torznab v1
|
||||
CREATE TABLE comments (
|
||||
event_id TEXT PRIMARY KEY,
|
||||
torrent_event_id TEXT NOT NULL,
|
||||
pubkey TEXT NOT NULL,
|
||||
created_at INTEGER NOT NULL,
|
||||
content TEXT NOT NULL,
|
||||
raw_event TEXT NOT NULL,
|
||||
FOREIGN KEY (torrent_event_id) REFERENCES torrents(event_id) ON DELETE CASCADE
|
||||
);
|
||||
```
|
||||
|
||||
## Config file
|
||||
|
||||
`/etc/nzbstr/config.yaml` — load order: defaults → file → env vars (prefix `NZBSTR_`) → CLI flags.
|
||||
|
||||
```yaml
|
||||
# nzbstr config
|
||||
|
||||
server:
|
||||
listen: "127.0.0.1:9117" # bind addr; sit behind nginx for TLS
|
||||
base_url: "https://nzbstr.example.com" # used in Torznab feed links
|
||||
|
||||
database:
|
||||
path: "/var/lib/nzbstr/nzbstr.db"
|
||||
|
||||
logging:
|
||||
level: "info" # debug|info|warn|error
|
||||
format: "json" # json|text
|
||||
|
||||
# Relays to subscribe to for NIP-35 events.
|
||||
# If empty on first run, defaults to a sane starter set.
|
||||
relays:
|
||||
- "wss://relay.damus.io"
|
||||
- "wss://nos.lol"
|
||||
- "wss://relay.primal.net"
|
||||
- "wss://nostr.mom"
|
||||
- "wss://relay.snort.social"
|
||||
- "wss://sovbit.host" # eric's own relay
|
||||
# add more
|
||||
|
||||
# Initial backfill via NIP-77 negentropy. Set false to start from "now" only.
|
||||
negentropy_bootstrap: true
|
||||
backfill_days: 365 # don't go further back than this
|
||||
|
||||
# Curation
|
||||
curation:
|
||||
# If true, only ingest events from pubkeys in your follow graph (within follow_depth hops).
|
||||
wot_only: false
|
||||
follow_depth: 2
|
||||
# Always allow these pubkeys regardless of WoT
|
||||
allowlist:
|
||||
- "npub1..."
|
||||
# Always block these
|
||||
blocklist:
|
||||
- "npub1..."
|
||||
# Auto-subscribe to these curation sets (kind 30004 naddr)
|
||||
curation_sets:
|
||||
- "naddr1..."
|
||||
|
||||
# TMDB enrichment (optional; without it, only events with imdb/tmdb i-tags are searchable by ID)
|
||||
tmdb:
|
||||
enabled: true
|
||||
api_key: "${TMDB_API_KEY}"
|
||||
cache_ttl: "168h"
|
||||
|
||||
# Health scraping (optional)
|
||||
health:
|
||||
enabled: false # off by default; rude to private trackers
|
||||
method: "dht" # dht|tracker|both
|
||||
refresh_interval: "30m"
|
||||
|
||||
# Writer side - publishing your own torrents to nostr
|
||||
publisher:
|
||||
enabled: false # off until you set it up explicitly
|
||||
signer:
|
||||
mode: "bunker" # local|ncryptsec|bunker
|
||||
bunker_uri: "bunker://..." # for NIP-46
|
||||
ncryptsec: "" # for ncryptsec mode
|
||||
nsec: "" # for local mode (NOT RECOMMENDED)
|
||||
outbox_relays:
|
||||
- "wss://sovbit.host"
|
||||
- "wss://relay.damus.io"
|
||||
- "wss://nos.lol"
|
||||
# Where to watch for completed downloads
|
||||
client:
|
||||
type: "qbittorrent" # qbittorrent|transmission|deluge|watch_dir
|
||||
qbittorrent:
|
||||
url: "http://127.0.0.1:8080"
|
||||
username: "admin"
|
||||
password: "${QBIT_PASSWORD}"
|
||||
# Only publish torrents tagged with this category (so you don't accidentally publish everything)
|
||||
category: "publish-nostr"
|
||||
watch_dir:
|
||||
path: "/var/lib/nzbstr/watch"
|
||||
# Auto-enrich title parsing -> TMDB lookup before publishing
|
||||
enrich_before_publish: true
|
||||
```
|
||||
|
||||
## API keys
|
||||
|
||||
Bootstrap via CLI:
|
||||
|
||||
```
|
||||
nzbstr-cli apikey create --label sonarr --visibility wot
|
||||
# prints: <key>
|
||||
```
|
||||
|
||||
Sonarr config: paste key in. Each *arr instance gets its own.
|
||||
|
||||
## Torznab API surface
|
||||
|
||||
Minimum viable surface for full *arr compatibility. All endpoints under `/api`.
|
||||
|
||||
### Auth
|
||||
|
||||
Query param: `apikey=<key>`. Required on every endpoint except `/health`. Missing or bad key returns 401 with a Torznab `<error code="100"/>`.
|
||||
|
||||
### `GET /api?t=caps`
|
||||
|
||||
Capabilities document. Sonarr/Radarr hit this on add to learn supported categories and search modes. Return XML like:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<caps>
|
||||
<server version="0.1.0" title="nzbstr" strapline="Nostr-native Torznab"
|
||||
email="" url="https://nzbstr.example.com/" image=""/>
|
||||
<limits max="100" default="50"/>
|
||||
<searching>
|
||||
<search available="yes" supportedParams="q"/>
|
||||
<tv-search available="yes" supportedParams="q,season,ep,imdbid,tvdbid,tmdbid"/>
|
||||
<movie-search available="yes" supportedParams="q,imdbid,tmdbid"/>
|
||||
<music-search available="yes" supportedParams="q,artist,album,year"/>
|
||||
<audio-search available="yes" supportedParams="q,artist,album,year"/>
|
||||
<book-search available="yes" supportedParams="q,author,title"/>
|
||||
</searching>
|
||||
<categories>
|
||||
<category id="2000" name="Movies">
|
||||
<subcat id="2030" name="Movies/SD"/>
|
||||
<subcat id="2040" name="Movies/HD"/>
|
||||
<subcat id="2045" name="Movies/UHD"/>
|
||||
<subcat id="2050" name="Movies/BluRay"/>
|
||||
<subcat id="2060" name="Movies/3D"/>
|
||||
</category>
|
||||
<category id="3000" name="Audio">
|
||||
<subcat id="3010" name="Audio/MP3"/>
|
||||
<subcat id="3030" name="Audio/Audiobook"/>
|
||||
<subcat id="3040" name="Audio/Lossless"/>
|
||||
</category>
|
||||
<category id="5000" name="TV">
|
||||
<subcat id="5030" name="TV/SD"/>
|
||||
<subcat id="5040" name="TV/HD"/>
|
||||
<subcat id="5045" name="TV/UHD"/>
|
||||
<subcat id="5070" name="TV/Anime"/>
|
||||
</category>
|
||||
<category id="7000" name="Books">
|
||||
<subcat id="7020" name="Books/EBook"/>
|
||||
<subcat id="7030" name="Books/Comics"/>
|
||||
</category>
|
||||
</categories>
|
||||
</caps>
|
||||
```
|
||||
|
||||
Don't get creative. Copy structure from an existing Torznab indexer. Sonarr will reject the indexer outright if `<categories>` is wrong shape.
|
||||
|
||||
### `GET /api?t=search&q=<query>&cat=<id>,<id>&limit=&offset=&apikey=`
|
||||
|
||||
Generic search. `q` is FTS5 query against title+description. Returns Torznab RSS:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"
|
||||
xmlns:torznab="http://torznab.com/schemas/2015/feed">
|
||||
<channel>
|
||||
<atom:link rel="self" type="application/rss+xml"/>
|
||||
<title>nzbstr</title>
|
||||
<description>Nostr NIP-35 torrent index</description>
|
||||
<link>https://nzbstr.example.com</link>
|
||||
<language>en-us</language>
|
||||
<category>search</category>
|
||||
<item>
|
||||
<title>Some.Show.S01E02.2160p.WEB-DL.x265-GRP</title>
|
||||
<guid isPermaLink="false">nostr:nevent1...</guid>
|
||||
<link>magnet:?xt=urn:btih:HASH&dn=...&tr=...</link>
|
||||
<comments>https://nzbstr.example.com/torrent/nevent1...</comments>
|
||||
<pubDate>Tue, 12 May 2026 18:30:00 +0000</pubDate>
|
||||
<size>15234567890</size>
|
||||
<description><![CDATA[ ... ]]></description>
|
||||
<category>5045</category>
|
||||
<enclosure url="magnet:?xt=urn:btih:HASH..." length="15234567890" type="application/x-bittorrent"/>
|
||||
<torznab:attr name="category" value="5045"/>
|
||||
<torznab:attr name="category" value="5000"/>
|
||||
<torznab:attr name="size" value="15234567890"/>
|
||||
<torznab:attr name="infohash" value="HASH"/>
|
||||
<torznab:attr name="magneturl" value="magnet:?xt=urn:btih:HASH..."/>
|
||||
<torznab:attr name="seeders" value="42"/>
|
||||
<torznab:attr name="peers" value="50"/>
|
||||
<torznab:attr name="downloadvolumefactor" value="0"/>
|
||||
<torznab:attr name="uploadvolumefactor" value="1"/>
|
||||
<torznab:attr name="tvdbid" value="355567"/>
|
||||
<torznab:attr name="imdbid" value="15239678"/>
|
||||
<torznab:attr name="tmdbid" value="693134"/>
|
||||
</item>
|
||||
<!-- more items -->
|
||||
</channel>
|
||||
</rss>
|
||||
```
|
||||
|
||||
### `GET /api?t=tvsearch&q=&tvdbid=&imdbid=&tmdbid=&season=&ep=&cat=&apikey=`
|
||||
|
||||
TV search. Prefers structured ID matches (tvdbid, imdbid, tmdbid) over `q`. If `season` and/or `ep` provided, filter by parsed season/episode columns.
|
||||
|
||||
### `GET /api?t=movie&q=&imdbid=&tmdbid=&cat=&apikey=`
|
||||
|
||||
Movie search. Same shape.
|
||||
|
||||
### `GET /api?t=music&q=&artist=&album=&year=&cat=&apikey=`
|
||||
|
||||
Music search.
|
||||
|
||||
### `GET /api?t=audio&q=&artist=&album=&year=&cat=&apikey=`
|
||||
|
||||
Audio search (audiobooks etc).
|
||||
|
||||
### `GET /api?t=book&q=&author=&title=&cat=&apikey=`
|
||||
|
||||
Book search.
|
||||
|
||||
### `GET /health`
|
||||
|
||||
No auth. Returns JSON `{status, version, relays_connected, events_total, last_event_at}`. For Prometheus scrape compatibility, also expose `/metrics` (text exposition format).
|
||||
|
||||
## NIP-35 event handling
|
||||
|
||||
Reference shape (kind 2003):
|
||||
|
||||
```json
|
||||
{
|
||||
"kind": 2003,
|
||||
"pubkey": "<hex>",
|
||||
"created_at": 1715000000,
|
||||
"content": "<long description, may contain newlines>",
|
||||
"tags": [
|
||||
["title", "Some.Show.S01E02.2160p.WEB-DL.x265-GRP"],
|
||||
["x", "abcdef...0123"],
|
||||
["file", "Some.Show.S01E02.mkv", "15234567890"],
|
||||
["tracker", "udp://tracker.opentrackr.org:1337"],
|
||||
["tracker", "http://tracker.example.org/announce"],
|
||||
["i", "tcat:video,tv,4k"],
|
||||
["i", "newznab:5045"],
|
||||
["i", "imdb:tt15239678"],
|
||||
["i", "tmdb:tv:693134"],
|
||||
["i", "tvdb:355567"],
|
||||
["t", "tv"],
|
||||
["t", "4k"]
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Parser rules
|
||||
|
||||
- Reject events without a valid `["x", <40-hex>]` info-hash tag. Log + drop.
|
||||
- Reject events without at least one `["title", ...]` or fall back to `content` first line if missing.
|
||||
- `size_bytes` = sum of `file` tag sizes (parse position 2 as integer).
|
||||
- `tracker` tags accumulate into `trackers` table.
|
||||
- `i` tags drive ID matching: parse prefixes `imdb:`, `tmdb:movie:`, `tmdb:tv:`, `tvdb:`, `tcat:`, `newznab:`.
|
||||
- `newznab_cat` comes from `["i", "newznab:NNNN"]` if present, else inferred from `tcat:` path (see category map below).
|
||||
- `t` tags accumulate into `tags` table for filtering.
|
||||
- Validate signature on ingest (go-nostr handles this). Drop if invalid.
|
||||
|
||||
### Magnet construction
|
||||
|
||||
```
|
||||
magnet:?xt=urn:btih:<info_hash>
|
||||
&dn=<url-encoded title>
|
||||
&tr=<url-encoded tracker 1>
|
||||
&tr=<url-encoded tracker 2>
|
||||
...
|
||||
```
|
||||
|
||||
Always include trackers from the event plus a fixed list of public DHT trackers as fallback (configurable).
|
||||
|
||||
## Category mapping (tcat → newznab)
|
||||
|
||||
Hand-rolled map. Add entries as you encounter them. Defaults below; extend liberally.
|
||||
|
||||
| tcat path | newznab |
|
||||
|---|---|
|
||||
| `video,movie` | 2000 |
|
||||
| `video,movie,sd` | 2030 |
|
||||
| `video,movie,hd` | 2040 |
|
||||
| `video,movie,4k` or `video,movie,uhd` | 2045 |
|
||||
| `video,movie,bluray` or `video,movie,remux` | 2050 |
|
||||
| `video,tv` | 5000 |
|
||||
| `video,tv,sd` | 5030 |
|
||||
| `video,tv,hd` | 5040 |
|
||||
| `video,tv,4k` or `video,tv,uhd` | 5045 |
|
||||
| `video,tv,anime` | 5070 |
|
||||
| `audio,music` | 3000 |
|
||||
| `audio,music,lossless` | 3040 |
|
||||
| `audio,audiobook` | 3030 |
|
||||
| `book` | 7000 |
|
||||
| `book,ebook` | 7020 |
|
||||
| `book,comic` | 7030 |
|
||||
| anything else | 8000 (Other) |
|
||||
|
||||
## Title parser
|
||||
|
||||
For events lacking IMDB/TMDB/TVDB tags, parse the title to extract:
|
||||
|
||||
- Show/movie name
|
||||
- Year (4-digit pattern)
|
||||
- Season (Sxx, xx digits)
|
||||
- Episode (Exx)
|
||||
- Quality (480p|576p|720p|1080p|2160p|SD|HD|UHD)
|
||||
- Source (WEB-DL|WEBRip|BluRay|BDRip|HDTV|DVDRip|REMUX)
|
||||
- Codec (x264|x265|HEVC|AV1|XViD)
|
||||
- Release group (after final `-`)
|
||||
|
||||
Use a port of one of the existing parsers (e.g. `guessit` from Python land is the reference). For Go specifically, look at `github.com/middelink/go-parse-torrent-name` and `github.com/megalol/parsetorrentname` and fork if needed. Don't write this from scratch unless you must.
|
||||
|
||||
After parsing, optionally hit TMDB's `search/tv` or `search/movie` endpoint with the cleaned name + year to backfill `imdb_id`/`tmdb_id`/`tvdb_id`. Cache results aggressively (TMDB IDs don't change).
|
||||
|
||||
## Web of Trust filter
|
||||
|
||||
When `curation.wot_only: true`:
|
||||
|
||||
1. On startup and every N hours, fetch the operator's own kind 3 (Follow List) event.
|
||||
2. For each followed pubkey, optionally fetch *their* kind 3 (depth 2). Cache.
|
||||
3. Build `allowed_pubkeys` set: operator + follows + (follows-of-follows if depth ≥ 2) + `allowlist`, minus `blocklist`.
|
||||
4. Reject any incoming NIP-35 event from a pubkey not in the set.
|
||||
|
||||
The operator's pubkey is derived from the publisher signer config if present, or set explicitly in `curation.operator_pubkey`.
|
||||
|
||||
## Phase plan
|
||||
|
||||
Each phase has explicit acceptance criteria. Don't move to the next phase until current passes.
|
||||
|
||||
### Phase 0 — bootstrap (1 evening)
|
||||
|
||||
- [ ] Repo scaffolded with the layout above
|
||||
- [ ] `cmd/nzbstr/main.go` boots, parses config, opens DB, applies migrations
|
||||
- [ ] systemd unit installs cleanly
|
||||
- [ ] `/health` returns 200 with version
|
||||
- [ ] Logging goes to journald in JSON
|
||||
|
||||
**Acceptance:** `systemctl start nzbstr` works; `journalctl -u nzbstr` shows clean startup; `curl localhost:9117/health` returns expected JSON.
|
||||
|
||||
### Phase 1 — reader, basic Torznab (1 weekend)
|
||||
|
||||
- [ ] Subscribes to configured relays for kind 2003
|
||||
- [ ] Parses, validates, stores events into SQLite
|
||||
- [ ] Implements `t=caps` and `t=search` (generic FTS5 over title)
|
||||
- [ ] Returns valid Torznab RSS with magnet links
|
||||
- [ ] Sonarr can add nzbstr as a Torznab indexer and run a test search successfully
|
||||
|
||||
**Acceptance:** Add nzbstr to Sonarr → "Test" succeeds → manual search for a known popular release returns results with valid magnets that load in qBittorrent.
|
||||
|
||||
### Phase 2 — full *arr compatibility (1 weekend)
|
||||
|
||||
- [ ] `t=tvsearch`, `t=movie`, `t=music`, `t=audio`, `t=book` endpoints
|
||||
- [ ] Structured ID matching (imdbid/tmdbid/tvdbid)
|
||||
- [ ] Newznab category normalization complete
|
||||
- [ ] Title parser fills in season/episode/quality/source when missing
|
||||
- [ ] TMDB enrichment for events without ID tags
|
||||
|
||||
**Acceptance:** Sonarr automatically finds a known recent episode by tvdbid+season+ep with no manual intervention; Radarr finds a known movie by imdbid; Lidarr finds an album by MusicBrainz fields. End-to-end: episode airs → Sonarr queries → nzbstr returns → Sonarr sends to qBittorrent → download completes → Plex picks it up.
|
||||
|
||||
### Phase 3 — curation (1-2 weekends)
|
||||
|
||||
- [ ] WoT filter active when configured
|
||||
- [ ] NIP-32 label ingestion (kind 1985)
|
||||
- [ ] NIP-51 set subscription (kind 30004)
|
||||
- [ ] Per-API-key visibility filter
|
||||
- [ ] `nzbstr-cli` commands for publisher trust management
|
||||
- [ ] Block/mute lists honored
|
||||
|
||||
**Acceptance:** Two API keys configured, one "all", one "wot-only" → same query returns different result counts and different publishers. Subscribing to a curation set adds those events to the "curated" visibility.
|
||||
|
||||
### Phase 4 — writer / publisher (2 weekends)
|
||||
|
||||
- [ ] NIP-46 bunker connection working
|
||||
- [ ] qBittorrent integration: poll for completed torrents in tagged category
|
||||
- [ ] Build kind 2003 event from torrent metadata
|
||||
- [ ] TMDB enrichment of own publishes
|
||||
- [ ] Publish to configured outbox relays
|
||||
- [ ] Stored in local DB as if ingested
|
||||
|
||||
**Acceptance:** Add a torrent to qBittorrent with category `publish-nostr`, wait for download to complete → within 60 seconds, the event appears in your own DB *and* on dtan.xyz.
|
||||
|
||||
### Phase 5 — Usenet/Blossom binary bridge (longer horizon)
|
||||
|
||||
This is propose-a-NIP territory. Punt to a separate spec doc after Phase 4 is solid. The basic shape:
|
||||
|
||||
- Sister kind (say, 2005 — must claim, check current NIP registry first) for "Blossom binary release"
|
||||
- Same tag surface as NIP-35 minus `x`/`tracker` plus `blob` tags pointing to Blossom blob hashes on listed servers
|
||||
- Downloader sidecar that fetches blobs from configured Blossom servers, reassembles, hands to media library
|
||||
- This is the Nostr-native Usenet replacement
|
||||
|
||||
Don't start Phase 5 until Phase 4 is rock solid and ideally adopted by at least a couple of other people running nzbstr.
|
||||
|
||||
## Things to flag for Eric
|
||||
|
||||
- **NIP-46 bunker on a public-facing seedbox** is the right answer security-wise; consider running a self-hosted bunker on UTS-01 with the nzbstr daemon as a client. Don't put nsec on the seedbox.
|
||||
- **Title parsing is the part most likely to suck.** Budget more time than you think. The reference implementations all have edge cases.
|
||||
- **TMDB rate limits** are real (50 req/sec). Implement a circuit breaker.
|
||||
- **NIP-77 negentropy bootstrap** against six relays for a year of events is slow on first run. Show progress in logs. Consider a "fresh start" mode that skips backfill entirely.
|
||||
- **The Sonarr `<categories>` block in caps must be exactly right** or Sonarr silently rejects the indexer with no useful error. Copy from a working Jackett/Prowlarr indefinitely.
|
||||
- **Don't scrape private trackers for seeder counts.** That'll get a publisher's IP burned. Default `health.enabled: false`.
|
||||
- **Backup the SQLite file**, not just relay subscription state. Use SQLite's online backup API or wrap `sqlite3 .backup`.
|
||||
|
||||
## systemd unit (deploy/nzbstr.service)
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=nzbstr — Nostr-native Torznab indexer
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=nzbstr
|
||||
Group=nzbstr
|
||||
ExecStart=/usr/local/bin/nzbstr --config /etc/nzbstr/config.yaml
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
# Security hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/lib/nzbstr
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictSUIDSGID=true
|
||||
LockPersonality=true
|
||||
MemoryDenyWriteExecute=true
|
||||
RestrictRealtime=true
|
||||
RestrictNamespaces=true
|
||||
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=nzbstr
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
## Makefile (sketch)
|
||||
|
||||
```makefile
|
||||
.PHONY: build test clean install run lint
|
||||
|
||||
BINARY := nzbstr
|
||||
CLI := nzbstr-cli
|
||||
VERSION := $(shell git describe --tags --always --dirty)
|
||||
LDFLAGS := -ldflags "-X main.version=$(VERSION) -s -w"
|
||||
|
||||
build:
|
||||
go build $(LDFLAGS) -o ./bin/$(BINARY) ./cmd/nzbstr
|
||||
go build $(LDFLAGS) -o ./bin/$(CLI) ./cmd/nzbstr-cli
|
||||
|
||||
test:
|
||||
go test -race ./...
|
||||
|
||||
lint:
|
||||
golangci-lint run
|
||||
|
||||
install: build
|
||||
install -D -m 0755 ./bin/$(BINARY) /usr/local/bin/$(BINARY)
|
||||
install -D -m 0755 ./bin/$(CLI) /usr/local/bin/$(CLI)
|
||||
install -D -m 0644 ./deploy/nzbstr.service /etc/systemd/system/nzbstr.service
|
||||
install -D -m 0640 ./deploy/nzbstr.example.yaml /etc/nzbstr/config.yaml
|
||||
|
||||
run:
|
||||
go run ./cmd/nzbstr --config ./deploy/nzbstr.example.yaml
|
||||
|
||||
clean:
|
||||
rm -rf ./bin
|
||||
```
|
||||
|
||||
## Open questions for the operator
|
||||
|
||||
These should be decided before Phase 1 starts. Default in parens if punted.
|
||||
|
||||
1. Single-user or multi-tenant from day one? *(single-user; multi-tenant comes for free via API keys in Phase 3)*
|
||||
2. Embed a web browse UI eventually, or keep it pure middleware? *(keep pure; build a separate thin frontend later if needed)*
|
||||
3. Support kind 2004 comments in search results? *(no in v1, store but don't surface)*
|
||||
4. Tracker scraping for seeder counts: opt-in via DHT only? *(yes, DHT only, opt-in)*
|
||||
5. NIP-50 search relays for query offload instead of local FTS? *(no, local FTS is faster and more flexible)*
|
||||
6. Support fetching blob from Blossom for events that include `["url", ...]` non-magnet links? *(deferred to Phase 5)*
|
||||
7. Should the writer side also publish kind 1063 (NIP-94 File Metadata) for individual files, or just kind 2003? *(just 2003 in v1)*
|
||||
8. Rate limit Torznab queries per API key? *(yes, 60/min default, configurable)*
|
||||
|
||||
## Reference material
|
||||
|
||||
- NIP-35 spec: `https://github.com/nostr-protocol/nips/blob/master/35.md`
|
||||
- NIP-46 (remote signing): `https://github.com/nostr-protocol/nips/blob/master/46.md`
|
||||
- NIP-77 (negentropy sync): `https://github.com/nostr-protocol/nips/blob/master/77.md`
|
||||
- NIP-32 (labeling): `https://github.com/nostr-protocol/nips/blob/master/32.md`
|
||||
- NIP-51 (lists): `https://github.com/nostr-protocol/nips/blob/master/51.md`
|
||||
- Torznab spec: `https://torznab.github.io/spec-1.3-draft/index.html`
|
||||
- Newznab categories: `https://github.com/Prowlarr/Prowlarr/wiki/Indexer-Categories`
|
||||
- dtan source for behavior reference: `https://git.v0l.io/Kieran/dtan`
|
||||
- Example NIP-35 implementer (Jackett): `https://github.com/Jackett/Jackett/pull/16416`
|
||||
- go-nostr: `https://github.com/nbd-wtf/go-nostr`
|
||||
|
||||
## Done criteria for "v1.0"
|
||||
|
||||
- All of Phases 1-4 acceptance criteria pass
|
||||
- nzbstr has been running on a real seedbox for 30 days without crashing or needing intervention
|
||||
- At least one external user (someone other than the operator) is running an instance and reporting back
|
||||
- README is good enough that someone new can deploy in under an hour
|
||||
- A blog post / nostr long-form (kind 30023) explaining what it is and how to run it exists
|
||||
- Listed in the awesome-nostr README and the relevant *arr community resources
|
||||
Reference in New Issue
Block a user