diff --git a/spec.md b/spec.md index 305ddc3..fc8b4d4 100644 --- a/spec.md +++ b/spec.md @@ -1,108 +1,131 @@ -# nzbstr — Nostr-Native Torznab Indexer +# kindexr — Nostr-Native Torznab Indexer -> Working name. Alternatives: `dtanr`, `torstr`, `arrostr`, `noindex`. Rename before v0.2 ships if you're going to. +> Self-hosted bridge between NIP-35 torrent events on the Nostr relay network and the Torznab API spoken by Sonarr / Radarr / Lidarr / Readarr / Prowlarr. + +## Build status + +- **Phase 0** — bootstrap: ✅ done +- **Phase 1** — reader, basic Torznab: ✅ done +- **Phase 2** — full *arr compatibility: 🎯 next +- **Phase 3** — curation: planned +- **Phase 4** — writer / publisher: planned +- **Phase 5** — Blossom binary bridge: long horizon + +When handing this spec to Claude Code, start at Phase 2 unless explicitly retrofitting earlier work. ## What this is -A daemon that bridges NIP-35 torrent events on the Nostr relay network into the Torznab API that Sonarr / Radarr / Lidarr / Readarr / Prowlarr already speak. Two halves: +Two halves in one daemon: -- **Reader (Phase 1+):** subscribes to kind 2003 (torrent) and kind 2004 (torrent comment) events on configured relays, indexes them locally, exposes a Torznab-compatible HTTP API. -- **Writer (Phase 4+):** watches a torrent client (qBittorrent / Transmission / Deluge) for completed downloads, builds and publishes NIP-35 events signed by your npub. +- **Reader (Phases 1-3):** subscribes to kind 2003 (torrent) and kind 2004 (torrent comment) events on configured Nostr relays, indexes them locally in SQLite, and exposes a Torznab-compatible HTTP API so the *arr automation stack can query as if kindexr were any other indexer. +- **Writer (Phase 4+):** watches a torrent client (qBittorrent / Transmission / Deluge) for completed downloads explicitly tagged for publishing, builds NIP-35 events, signs them via a dedicated kindexer identity, and publishes them to configured relays. -Positioning: **same slot as Jackett or Prowlarr.** Not a frontend like dtan.xyz; not a relay; not a downloader. Middleware that sits between Nostr and the *arr automation stack. +Positioning: same architectural slot as Jackett or Prowlarr. Not a frontend like dtan.xyz. Not a relay. Not a downloader. Not a DHT crawler. Middleware between Nostr and *arr. + +## Deployment model + +**Single-operator, self-hosted, personal infrastructure.** Every user runs their own kindexr instance on their own hardware. There is no signup, no shared service, no multi-tenancy. The API keys exist solely to let your own Sonarr/Radarr authenticate to your own kindexr daemon. + +This is intentional. A public-service deployment would re-centralize what Nostr decentralized, take on meaningful legal exposure for an indexer of torrent content, and require operational overhead (abuse handling, billing, support) that isn't the goal. The federated-curation pattern in the Identity section below provides the "share with others" value without operating a service. ## Non-goals -- No web browse UI for humans in v1. dtan.xyz and nostrudel/torrents already do that. A browse UI may come later as a nice-to-have; it is not the product. -- No new download protocol. BitTorrent is unchanged. Magnet links and `.torrent` files come out of nzbstr exactly as they go in. -- No relay implementation. nzbstr is a relay *client*. If you want a relay, run strfry. -- No replacement for Sonarr/Radarr/Lidarr. nzbstr sits underneath them. -- No login system, no accounts, no web admin in v1. Config file + API keys. +- No DHT crawling. Kindexr does **not** join the BitTorrent DHT, harvest info-hashes, or participate in BitTorrent protocol at all. The only ingest source is signed Nostr events. This is the primary structural difference from tools like Bitmagnet. +- No website scraping. No HTML parsing of tracker sites. +- No web browse UI for humans in v1. dtan.xyz and nostrudel/torrents serve that need. A browse UI may come later but is not the product. +- No new download protocol. BitTorrent is unchanged. Magnet links come out exactly as they go in. +- No relay implementation. Kindexr is a client of relays. +- No replacement for Sonarr/Radarr. Kindexr sits underneath them in the same slot Jackett/Prowlarr occupies. +- No multi-tenant SaaS. No user accounts, no signup, no admin panel for managing other people's access. ## Architecture ``` Sonarr / Radarr / Lidarr / Readarr / Prowlarr - | - v Torznab HTTP/XML - | - +-----------+ - | nzbstr | - | reader | <-- subscribes to relays via WebSocket - | writer | --> publishes via WebSocket - +-----------+ - ^ ^ - | | - v v - Nostr relays qBittorrent / Transmission / Deluge - (NIP-35 events) (download client) + │ + │ Torznab HTTP/XML + ▼ + ┌──────────────┐ + │ kindexr │ + │ reader │ ◄── WSS ─── Nostr relays + │ writer │ ─── WSS ──► Nostr relays + └──────────────┘ + ▲ + │ HTTP API + │ + qBittorrent / Transmission / Deluge + (download client; only used by writer) ``` -## Tech stack (locked) +Reader and writer share the same daemon binary and the same SQLite database. They can be enabled independently — Phase 2/3 instances run with `publisher.enabled: false`. The reader is the default; the writer is opt-in. -These are decisions. Don't litigate them in Phase 1. +## Tech stack + +Locked. Don't re-litigate. | Concern | Choice | Why | |---|---|---| -| Language | Go 1.22+ | Mature nostr libraries, single static binary, matches existing nostr-poster pattern, systemd-friendly | -| Nostr client | `github.com/nbd-wtf/go-nostr` | De facto Go nostr lib, NIP-77 negentropy, NIP-42 AUTH, NIP-46 bunker support | -| Storage | SQLite + FTS5 | Single-file backup, no separate service, FTS5 handles millions of rows fine | +| Language | Go 1.22+ | Mature Nostr libs, single static binary, matches existing nostr-poster pattern, systemd-friendly | +| Nostr client | `github.com/nbd-wtf/go-nostr` | De facto Go Nostr lib; NIP-77 negentropy, NIP-42 AUTH, NIP-46 bunker | +| Storage | SQLite + FTS5 | Single-file backup, no separate service, FTS5 fine to several million rows | | SQLite driver | `modernc.org/sqlite` | Pure Go, no CGO, simpler cross-compile | | HTTP router | `github.com/go-chi/chi/v5` | Lightweight, good middleware ergonomics | -| Torrent parsing | `github.com/anacrolix/torrent/metainfo` | Reference implementation, computes info-hash correctly | -| Config | `github.com/knadh/koanf/v2` | Cleaner than viper, supports YAML+env+flags | -| Logging | `log/slog` (stdlib) | Structured, JSON for Loki/journald | -| TMDB | `github.com/cyruzin/golang-tmdb` | For metadata enrichment when events lack IDs | -| Deployment | systemd + single binary | Same shape as everything else on his boxes | +| Torrent parsing | `github.com/anacrolix/torrent/metainfo` | Reference implementation, correct info-hash computation | +| Config | `github.com/knadh/koanf/v2` | Cleaner than viper; YAML + env + flags | +| Logging | `log/slog` (stdlib) | Structured, JSON for journald/Loki | +| TMDB | `github.com/cyruzin/golang-tmdb` | Metadata enrichment when events lack IDs | +| Title parsing | `github.com/middelink/go-parse-torrent-name` or `github.com/megalol/parsetorrentname` (fork if needed) | Reference parsers exist; don't write from scratch | +| Deployment | systemd + single binary | Same shape as everything else on the operator's boxes | -Postgres is explicitly *not* used in v1. SQLite for everything. Reconsider only if scale becomes a real problem (>5M events indexed or >100 concurrent Torznab clients). +PostgreSQL explicitly **not** used. SQLite for everything. Revisit only at >5M events or >100 concurrent Torznab clients. ## Repository layout ``` -nzbstr/ +kindexr/ ├── cmd/ -│ ├── nzbstr/ # main daemon binary +│ ├── kindexr/ # main daemon binary │ │ └── main.go -│ └── nzbstr-cli/ # admin CLI (relay add/remove, publisher trust, db stats) +│ └── kindexr-cli/ # admin CLI │ └── main.go ├── internal/ -│ ├── config/ # koanf-based config loading -│ ├── db/ # SQLite, migrations, queries +│ ├── config/ +│ ├── db/ │ │ ├── migrations/ -│ │ └── queries.sql # sqlc-style if you go that route, or hand-rolled +│ │ └── queries.sql │ ├── nostr/ -│ │ ├── reader.go # relay subscription loop, NIP-77 negentropy bootstrap -│ │ ├── writer.go # event publishing -│ │ ├── signer.go # local nsec / NIP-49 ncryptsec / NIP-46 bunker -│ │ └── parser.go # kind 2003 -> Torrent struct +│ │ ├── reader.go # relay subscription loop, NIP-77 bootstrap +│ │ ├── writer.go # event publishing +│ │ ├── signer.go # local nsec / NIP-49 / NIP-46 bunker +│ │ └── parser.go # kind 2003 -> Torrent struct │ ├── torznab/ -│ │ ├── server.go # chi routes, auth middleware -│ │ ├── caps.go # t=caps response -│ │ ├── search.go # t=search, tvsearch, movie, music, book -│ │ ├── xml.go # torznab XML marshalling -│ │ └── categories.go # newznab category mapping +│ │ ├── server.go +│ │ ├── caps.go +│ │ ├── search.go +│ │ ├── xml.go +│ │ └── categories.go │ ├── enrich/ -│ │ ├── tmdb.go # TMDB lookups for missing IDs -│ │ └── parser.go # parse "Show.Name.S01E02.1080p" into structured fields +│ │ ├── tmdb.go +│ │ └── parser.go # parse "Show.Name.S01E02.1080p" into fields │ ├── health/ -│ │ ├── tracker.go # UDP/HTTP tracker scrape (optional) -│ │ └── dht.go # DHT peer-count scrape (optional) +│ │ ├── tracker.go # opt-in tracker scrape +│ │ └── dht.go # opt-in DHT seeder count │ ├── publisher/ -│ │ ├── qbittorrent.go # qBit Web API client -│ │ ├── transmission.go # Transmission RPC client -│ │ └── watcher.go # event loop: on download complete -> publish +│ │ ├── qbittorrent.go +│ │ ├── transmission.go +│ │ └── watcher.go │ └── wot/ -│ ├── follows.go # build allowed-pubkey set from kind 3 events -│ └── trust.go # per-pubkey trust scoring +│ ├── follows.go +│ └── trust.go ├── deploy/ -│ ├── nzbstr.service # systemd unit -│ ├── nzbstr.example.yaml # commented config template -│ └── nginx.conf.example # reverse proxy with TLS +│ ├── kindexr.service +│ ├── kindexr.example.yaml +│ └── nginx.conf.example ├── docs/ │ ├── ARCHITECTURE.md -│ ├── TORZNAB.md # which subset of Torznab is supported -│ └── PUBLISHING.md # how the writer side works +│ ├── TORZNAB.md +│ ├── PUBLISHING.md +│ └── IDENTITY.md # operator identity bootstrap walkthrough ├── go.mod ├── go.sum ├── Makefile @@ -111,30 +134,30 @@ nzbstr/ ## Database schema -Single SQLite file at `/var/lib/nzbstr/nzbstr.db`. Migrations as numbered SQL files under `internal/db/migrations/`. Apply on startup; refuse to start on migration error. +Single SQLite file at `/var/lib/kindexr/kindexr.db`. Numbered migrations under `internal/db/migrations/`. Apply on startup; refuse to start on migration failure. ```sql --- 001_initial.sql +-- 001_initial.sql (applied in Phase 0/1; here for reference) CREATE TABLE torrents ( event_id TEXT PRIMARY KEY, -- nostr event id (hex) info_hash TEXT NOT NULL, -- v1 info-hash (hex, lowercase) pubkey TEXT NOT NULL, -- publisher pubkey (hex) - created_at INTEGER NOT NULL, -- nostr event created_at - ingested_at INTEGER NOT NULL, -- when we saw it + created_at INTEGER NOT NULL, + ingested_at INTEGER NOT NULL, title TEXT NOT NULL, description TEXT, - size_bytes INTEGER, -- sum of file sizes if no top-level size - category TEXT, -- normalized: movie/tv/music/book/audio/xxx/other - newznab_cat INTEGER, -- 2000/3000/5000/7000/etc - imdb_id TEXT, -- e.g. "tt15239678" - tmdb_id TEXT, -- "movie:693134" or "tv:1396" + size_bytes INTEGER, + category TEXT, -- movie/tv/music/book/audio/xxx/other + newznab_cat INTEGER, + imdb_id TEXT, + tmdb_id TEXT, -- "movie:693134" | "tv:1396" tvdb_id TEXT, - season INTEGER, -- parsed from title or i tags + season INTEGER, episode INTEGER, - quality TEXT, -- 480p/720p/1080p/2160p/SD/HD/UHD - source TEXT, -- WEB-DL/BluRay/HDTV/REMUX - raw_event TEXT NOT NULL -- full JSON for re-parsing + quality TEXT, + source TEXT, + raw_event TEXT NOT NULL ); CREATE UNIQUE INDEX idx_torrents_event_id ON torrents(event_id); CREATE INDEX idx_torrents_info_hash ON torrents(info_hash); @@ -145,28 +168,17 @@ CREATE INDEX idx_torrents_tvdb ON torrents(tvdb_id) WHERE tvdb_id IS NOT NULL; CREATE INDEX idx_torrents_created ON torrents(created_at DESC); CREATE INDEX idx_torrents_cat ON torrents(newznab_cat); --- FTS index for text search CREATE VIRTUAL TABLE torrents_fts USING fts5( title, description, content='torrents', content_rowid='rowid', tokenize='unicode61 remove_diacritics 2' ); --- triggers to keep FTS in sync (insert/update/delete) -CREATE TRIGGER torrents_ai AFTER INSERT ON torrents BEGIN - INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description); -END; -CREATE TRIGGER torrents_ad AFTER DELETE ON torrents BEGIN - INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description); -END; -CREATE TRIGGER torrents_au AFTER UPDATE ON torrents BEGIN - INSERT INTO torrents_fts(torrents_fts, rowid, title, description) VALUES ('delete', old.rowid, old.title, old.description); - INSERT INTO torrents_fts(rowid, title, description) VALUES (new.rowid, new.title, new.description); -END; +-- (triggers ai/ad/au to keep FTS in sync — already in place from Phase 1) CREATE TABLE files ( event_id TEXT NOT NULL, - idx INTEGER NOT NULL, -- file order within torrent + idx INTEGER NOT NULL, path TEXT NOT NULL, size_bytes INTEGER, PRIMARY KEY (event_id, idx), @@ -182,19 +194,19 @@ CREATE TABLE trackers ( CREATE TABLE tags ( event_id TEXT NOT NULL, - tag TEXT NOT NULL, -- e.g. "movie", "4k", "REMUX" + tag TEXT NOT NULL, PRIMARY KEY (event_id, tag), FOREIGN KEY (event_id) REFERENCES torrents(event_id) ON DELETE CASCADE ); CREATE INDEX idx_tags_tag ON tags(tag); CREATE TABLE publishers ( - pubkey TEXT PRIMARY KEY, -- hex - npub TEXT, -- bech32 cached for display - name TEXT, -- from kind 0 if available - trust REAL DEFAULT 0, -- -1.0 .. 1.0 + pubkey TEXT PRIMARY KEY, + npub TEXT, + name TEXT, + trust REAL DEFAULT 0, notes TEXT, - blocked INTEGER DEFAULT 0, -- 0/1 + blocked INTEGER DEFAULT 0, torrents_n INTEGER DEFAULT 0, first_seen INTEGER, last_seen INTEGER @@ -203,18 +215,16 @@ CREATE TABLE publishers ( CREATE TABLE relays ( url TEXT PRIMARY KEY, enabled INTEGER DEFAULT 1, - last_event INTEGER, -- created_at of most recent event from this relay - last_sync INTEGER, -- when we last successfully connected + last_event INTEGER, + last_sync INTEGER, notes TEXT ); +-- Simple flat API keys. No per-key visibility filtering (deferred indefinitely). CREATE TABLE api_keys ( - key TEXT PRIMARY KEY, -- random 32-byte hex - label TEXT NOT NULL, -- "sonarr", "radarr", "friend-jane" + key TEXT PRIMARY KEY, + label TEXT NOT NULL, created_at INTEGER NOT NULL, - -- visibility filter: which pubkeys this key can see - visibility TEXT NOT NULL DEFAULT 'all', -- 'all' | 'wot' | 'curated' - curation_set TEXT, -- naddr1... of a kind 30004 set, if visibility='curated' last_used INTEGER ); @@ -225,237 +235,228 @@ CREATE TABLE health ( checked_at INTEGER NOT NULL ); --- Comments (kind 2004) - lower priority, store but don't surface in Torznab v1 CREATE TABLE comments ( - event_id TEXT PRIMARY KEY, - torrent_event_id TEXT NOT NULL, - pubkey TEXT NOT NULL, - created_at INTEGER NOT NULL, - content TEXT NOT NULL, - raw_event TEXT NOT NULL, + event_id TEXT PRIMARY KEY, + torrent_event_id TEXT NOT NULL, + pubkey TEXT NOT NULL, + created_at INTEGER NOT NULL, + content TEXT NOT NULL, + raw_event TEXT NOT NULL, FOREIGN KEY (torrent_event_id) REFERENCES torrents(event_id) ON DELETE CASCADE ); ``` +### Migration notes for already-completed Phase 1 + +The original spec had a `visibility` and `curation_set` column on `api_keys` for per-key filter scoping. **Drop those** — the personal-infrastructure model doesn't need them, and Phase 3 curation applies daemon-wide instead of per-key. If they exist in your current schema: + +```sql +-- 002_drop_apikey_visibility.sql +ALTER TABLE api_keys DROP COLUMN visibility; +ALTER TABLE api_keys DROP COLUMN curation_set; +``` + ## Config file -`/etc/nzbstr/config.yaml` — load order: defaults → file → env vars (prefix `NZBSTR_`) → CLI flags. +`/etc/kindexr/config.yaml`. Load order: defaults → file → env (`KINDEXR_*`) → CLI flags. ```yaml -# nzbstr config - server: - listen: "127.0.0.1:9117" # bind addr; sit behind nginx for TLS - base_url: "https://nzbstr.example.com" # used in Torznab feed links + listen: "127.0.0.1:9117" + base_url: "https://kindexr.example.com" database: - path: "/var/lib/nzbstr/nzbstr.db" + path: "/var/lib/kindexr/kindexr.db" logging: - level: "info" # debug|info|warn|error - format: "json" # json|text + level: "info" + format: "json" -# Relays to subscribe to for NIP-35 events. -# If empty on first run, defaults to a sane starter set. +# Inbox relays — where to READ NIP-35 events from relays: - "wss://relay.damus.io" - "wss://nos.lol" - "wss://relay.primal.net" - "wss://nostr.mom" - "wss://relay.snort.social" - - "wss://sovbit.host" # eric's own relay - # add more + - "wss://sovbit.host" -# Initial backfill via NIP-77 negentropy. Set false to start from "now" only. negentropy_bootstrap: true -backfill_days: 365 # don't go further back than this +backfill_days: 365 -# Curation +# Curation (Phase 3) curation: - # If true, only ingest events from pubkeys in your follow graph (within follow_depth hops). wot_only: false follow_depth: 2 - # Always allow these pubkeys regardless of WoT - allowlist: - - "npub1..." - # Always block these - blocklist: - - "npub1..." - # Auto-subscribe to these curation sets (kind 30004 naddr) - curation_sets: - - "naddr1..." + operator_pubkey: "" # the npub whose follow graph defines WoT + allowlist: [] + blocklist: [] + curation_sets: [] # naddr1... of subscribed kind 30004 sets + trust_assertions: [] # naddr1... of trusted NIP-85 assertion sources + exclude_categories: [6000, 6010, 6020, 6030, 6040, 6050, 6060, 6070, 6080, 6090] + # XXX range — always exclude regardless of WoT -# TMDB enrichment (optional; without it, only events with imdb/tmdb i-tags are searchable by ID) +# TMDB enrichment tmdb: enabled: true api_key: "${TMDB_API_KEY}" cache_ttl: "168h" -# Health scraping (optional) +# Health scraping (off by default — DHT only when enabled) health: - enabled: false # off by default; rude to private trackers - method: "dht" # dht|tracker|both + enabled: false + method: "dht" refresh_interval: "30m" -# Writer side - publishing your own torrents to nostr +# Writer side (Phase 4+) — off until explicitly configured publisher: - enabled: false # off until you set it up explicitly + enabled: false + + # Dedicated kindexr identity. NEVER your main npub. signer: - mode: "bunker" # local|ncryptsec|bunker - bunker_uri: "bunker://..." # for NIP-46 - ncryptsec: "" # for ncryptsec mode - nsec: "" # for local mode (NOT RECOMMENDED) - outbox_relays: - - "wss://sovbit.host" - - "wss://relay.damus.io" - - "wss://nos.lol" - # Where to watch for completed downloads + mode: "bunker" # bunker | ncryptsec | local + bunker_uri: "bunker://..." # NIP-46 connection string + ncryptsec: "" + nsec: "" # only if mode: local — discouraged + + # Outbox lists by content category. Different content can reach different + # audiences. Categories without an explicit list fall back to default. + outbox: + default: + - "wss://sovbit.host" + - "wss://relay.damus.io" + - "wss://nos.lol" + # narrow audience — only your relay + private: + - "wss://sovbit.host" + # wider reach for things you want public + public: + - "wss://sovbit.host" + - "wss://relay.damus.io" + - "wss://nos.lol" + - "wss://relay.primal.net" + - "wss://nostr.mom" + + # Download client integration client: - type: "qbittorrent" # qbittorrent|transmission|deluge|watch_dir + type: "qbittorrent" # qbittorrent | transmission | deluge | watch_dir qbittorrent: url: "http://127.0.0.1:8080" username: "admin" password: "${QBIT_PASSWORD}" - # Only publish torrents tagged with this category (so you don't accidentally publish everything) - category: "publish-nostr" - watch_dir: - path: "/var/lib/nzbstr/watch" - # Auto-enrich title parsing -> TMDB lookup before publishing + + # Per-category mapping: qBit category -> kindexr outbox + nostr metadata + # Only torrents in a configured category get published. Everything else + # is ignored. Default is empty — nothing publishes until you opt in. + publish_categories: + publish-public: + outbox: "public" + nostr_tags: + - ["t", "shared-by-kindexr"] + publish-private: + outbox: "private" + nostr_tags: [] + publish-books: + outbox: "public" + nostr_category: "book" + publish-foss: + outbox: "public" + nostr_category: "software" + + # Delay between download completion and publish, for sanity checking + publish_delay: "1h" + + # TMDB enrichment of own publishes before signing enrich_before_publish: true ``` -## API keys +## Identity model -Bootstrap via CLI: +This is new since the original spec and deserves its own section. -``` -nzbstr-cli apikey create --label sonarr --visibility wot -# prints: -``` +### Two distinct identities -Sonarr config: paste key in. Each *arr instance gets its own. +**Operator identity (your main npub).** Your social Nostr identity. Used by kindexr only for: +- Reading your kind 3 follow list to build the WoT +- Reading your kind 10000 mute list to honor your blocks +- Reading curation sets and labels you've published + +Kindexr never signs anything as your operator identity. + +**Kindexr identity (a dedicated nsec).** Generated specifically for this kindexr instance. Used for: +- Signing kind 2003 torrent events the writer publishes +- Signing kind 30004 curation sets ("kindexr's verified releases") +- Signing kind 1985 labels ("this pubkey is a known good UHD encoder") +- Signing kind 30382 trusted assertions about other publishers + +If the seedbox is compromised, the worst case is "someone publishes spam under the kindexr identity for a while." You rotate, your main identity is intact, your social graph is intact. + +### Bootstrapping trust for the kindexr identity + +A brand-new nsec has no followers and no presence. Other people's kindexr instances filtering by WoT will not see its publishes. The bootstrap sequence: + +1. **Generate the kindexr nsec** via NIP-49 ncryptsec or directly into the bunker. Keep the unencrypted nsec offline (paper, password manager) for recovery. Never put it on the seedbox. + +2. **Set up NIP-46 bunker** on a more locked-down host than the seedbox — UTS-01 is the natural choice. The bunker holds the actual nsec; kindexr connects to it over Nostr and requests signatures on demand. See `docs/IDENTITY.md` for the bunker walkthrough. + +3. **Publish kind 0 (profile metadata)** for the kindexr pubkey: + ```json + { + "kind": 0, + "content": "{\"name\":\"sovbit-kindexr\",\"display_name\":\"sovbit kindexr\",\"about\":\"Automated NIP-35 publisher operated by @. Linux ISOs, FOSS releases, public-domain books. Run your own: github.com//kindexr\",\"nip05\":\"kindexr@sovbit.host\",\"lud16\":\"\",\"picture\":\"https://sovbit.host/kindexr-avatar.png\"}" + } + ``` + +4. **Set up the NIP-05.** On sovbit.host's `/.well-known/nostr.json`, add an entry mapping `kindexer` to the kindexr pubkey (hex). This makes `kindexr@sovbit.host` resolve and verify. + +5. **From your main npub, follow the kindexr pubkey.** Update your kind 3 follow list to include the kindexr pubkey. Now anyone running WoT-depth-1 or 2 filtering who follows you will automatically include kindexr's publishes. + +6. **From your main npub, publish a trusted assertion (NIP-85 kind 30382) or label (NIP-32 kind 1985) explicitly vouching for the kindexr pubkey:** + ```json + { + "kind": 30382, + "tags": [ + ["d", ""], + ["rating", "1.0"], + ["operator-vouched", "kindexr-instance"] + ], + "content": "I operate this kindexr instance. Publishes are automated from my seedbox." + } + ``` + +7. **Operationally, only ever auto-publish things you'd defend.** Auto-publishing every torrent that crosses your seedbox is a bad idea regardless of identity. The `publish-categories` config in the writer section enforces this — only torrents *you deliberately tagged* get published. Use that gate. + +### Why not NIP-26 delegation? + +NIP-26 lets one key sign on behalf of another but is marked unrecommended in the current NIPs list ("adds unnecessary burden for little gain"). Use the trusted-assertion/follow approach instead. ## Torznab API surface -Minimum viable surface for full *arr compatibility. All endpoints under `/api`. +(Phase 1 already implemented `t=caps` and `t=search`. This section documents the full surface for Phase 2 completion.) -### Auth +All endpoints under `/api`. Auth via `apikey=` query param. -Query param: `apikey=`. Required on every endpoint except `/health`. Missing or bad key returns 401 with a Torznab ``. +| Endpoint | Status | Notes | +|---|---|---| +| `GET /api?t=caps` | ✅ Phase 1 | Verify the `` block matches what Sonarr/Radarr accept. Copy structure from a working Jackett indexer if unsure. | +| `GET /api?t=search&q=&cat=` | ✅ Phase 1 | Generic FTS5 search | +| `GET /api?t=tvsearch&tvdbid=&imdbid=&tmdbid=&season=&ep=&q=` | 🎯 Phase 2 | Prefer structured ID matches over `q`. | +| `GET /api?t=movie&imdbid=&tmdbid=&q=` | 🎯 Phase 2 | Movie search | +| `GET /api?t=music&artist=&album=&year=&q=` | 🎯 Phase 2 | Music search | +| `GET /api?t=audio&artist=&album=&year=&q=` | 🎯 Phase 2 | Audio (audiobooks etc) | +| `GET /api?t=book&author=&title=&q=` | 🎯 Phase 2 | Book search | +| `GET /health` | ✅ Phase 1 | No auth. JSON. | +| `GET /metrics` | Optional | Prometheus text format | -### `GET /api?t=caps` +### Response format -Capabilities document. Sonarr/Radarr hit this on add to learn supported categories and search modes. Return XML like: +Standard Torznab RSS with the `torznab` XML namespace. Mandatory `` keys per item: `category`, `size`, `infohash`, `magneturl`. Optional but strongly recommended: `seeders`, `peers`, `tvdbid`, `imdbid`, `tmdbid`. -```xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -``` +For events lacking peer counts, omit the seeders/peers attrs entirely rather than reporting 0 — Sonarr may drop "0-seeder" results unless explicitly configured otherwise. -Don't get creative. Copy structure from an existing Torznab indexer. Sonarr will reject the indexer outright if `` is wrong shape. +### Sonarr `` gotcha -### `GET /api?t=search&q=&cat=,&limit=&offset=&apikey=` - -Generic search. `q` is FTS5 query against title+description. Returns Torznab RSS: - -```xml - - - - - nzbstr - Nostr NIP-35 torrent index - https://nzbstr.example.com - en-us - search - - Some.Show.S01E02.2160p.WEB-DL.x265-GRP - nostr:nevent1... - magnet:?xt=urn:btih:HASH&dn=...&tr=... - https://nzbstr.example.com/torrent/nevent1... - Tue, 12 May 2026 18:30:00 +0000 - 15234567890 - - 5045 - - - - - - - - - - - - - - - - - -``` - -### `GET /api?t=tvsearch&q=&tvdbid=&imdbid=&tmdbid=&season=&ep=&cat=&apikey=` - -TV search. Prefers structured ID matches (tvdbid, imdbid, tmdbid) over `q`. If `season` and/or `ep` provided, filter by parsed season/episode columns. - -### `GET /api?t=movie&q=&imdbid=&tmdbid=&cat=&apikey=` - -Movie search. Same shape. - -### `GET /api?t=music&q=&artist=&album=&year=&cat=&apikey=` - -Music search. - -### `GET /api?t=audio&q=&artist=&album=&year=&cat=&apikey=` - -Audio search (audiobooks etc). - -### `GET /api?t=book&q=&author=&title=&cat=&apikey=` - -Book search. - -### `GET /health` - -No auth. Returns JSON `{status, version, relays_connected, events_total, last_event_at}`. For Prometheus scrape compatibility, also expose `/metrics` (text exposition format). +Sonarr is silently strict about the caps response. Categories must use the exact newznab numeric IDs, must include both parent and subcat structure, and must not include unrecognized IDs. If indexer testing fails in Sonarr with no obvious error, the caps response is almost always the culprit. Diff yours against a known-working Jackett `t=caps` output. ## NIP-35 event handling @@ -466,198 +467,281 @@ Reference shape (kind 2003): "kind": 2003, "pubkey": "", "created_at": 1715000000, - "content": "", + "content": "", "tags": [ ["title", "Some.Show.S01E02.2160p.WEB-DL.x265-GRP"], - ["x", "abcdef...0123"], + ["x", "<40-hex info-hash>"], ["file", "Some.Show.S01E02.mkv", "15234567890"], ["tracker", "udp://tracker.opentrackr.org:1337"], - ["tracker", "http://tracker.example.org/announce"], ["i", "tcat:video,tv,4k"], ["i", "newznab:5045"], ["i", "imdb:tt15239678"], ["i", "tmdb:tv:693134"], ["i", "tvdb:355567"], - ["t", "tv"], - ["t", "4k"] + ["t", "tv"], ["t", "4k"] ] } ``` ### Parser rules -- Reject events without a valid `["x", <40-hex>]` info-hash tag. Log + drop. -- Reject events without at least one `["title", ...]` or fall back to `content` first line if missing. -- `size_bytes` = sum of `file` tag sizes (parse position 2 as integer). -- `tracker` tags accumulate into `trackers` table. -- `i` tags drive ID matching: parse prefixes `imdb:`, `tmdb:movie:`, `tmdb:tv:`, `tvdb:`, `tcat:`, `newznab:`. -- `newznab_cat` comes from `["i", "newznab:NNNN"]` if present, else inferred from `tcat:` path (see category map below). -- `t` tags accumulate into `tags` table for filtering. -- Validate signature on ingest (go-nostr handles this). Drop if invalid. +- Reject events without `["x", <40-hex>]`. Log + drop. +- Validate signature (go-nostr handles this). Drop invalid. +- `size_bytes` = sum of `file` tag size positions (index 2). +- `trackers` table: accumulate from `tracker` tags. +- `i tag` prefixes drive ID matching: `imdb:`, `tmdb:movie:`, `tmdb:tv:`, `tvdb:`, `tcat:`, `newznab:`. +- `newznab_cat`: from `["i", "newznab:NNNN"]` if present; else inferred from `tcat:`; else fallback to title parser. +- `t` tags into `tags` table for filtering. +- Honor `curation.exclude_categories` at ingest: events with excluded newznab_cat are stored but flagged `excluded=1` (or dropped entirely — config knob, default drop). ### Magnet construction ``` magnet:?xt=urn:btih: - &dn= - &tr= - &tr= - ... + &dn= + &tr=... ``` -Always include trackers from the event plus a fixed list of public DHT trackers as fallback (configurable). +Always include event-listed trackers plus a configurable fallback list of public DHT trackers. -## Category mapping (tcat → newznab) +### Category mapping (tcat → newznab) -Hand-rolled map. Add entries as you encounter them. Defaults below; extend liberally. +Same map as the original spec. Extend as needed. | tcat path | newznab | |---|---| | `video,movie` | 2000 | -| `video,movie,sd` | 2030 | | `video,movie,hd` | 2040 | -| `video,movie,4k` or `video,movie,uhd` | 2045 | -| `video,movie,bluray` or `video,movie,remux` | 2050 | +| `video,movie,4k` / `uhd` | 2045 | +| `video,movie,bluray` / `remux` | 2050 | | `video,tv` | 5000 | -| `video,tv,sd` | 5030 | | `video,tv,hd` | 5040 | -| `video,tv,4k` or `video,tv,uhd` | 5045 | +| `video,tv,4k` / `uhd` | 5045 | | `video,tv,anime` | 5070 | | `audio,music` | 3000 | | `audio,music,lossless` | 3040 | | `audio,audiobook` | 3030 | -| `book` | 7000 | | `book,ebook` | 7020 | | `book,comic` | 7030 | -| anything else | 8000 (Other) | +| anything else | 8000 | +| anything in XXX | 6000-series — **excluded by default** | + +## Reader behavior + +### Sources + +Kindexr ingests **only** signed Nostr events from configured relays. It does not: +- Crawl the BitTorrent DHT +- Scrape websites +- Participate in BitTorrent swarms +- Listen for peer announcements + +This is the primary structural difference from Bitmagnet and similar DHT crawlers. The ingest volume and signal-to-noise ratio are categorically different — Nostr publishes are deliberate, identified, and signed; DHT announces are anonymous, automated, and dominated by garbage and porn. + +### Subscription model + +For each configured relay, kindexr opens a persistent WebSocket and submits: + +``` +["REQ", "live-2003", {"kinds": [2003], "since": }] +["REQ", "live-2004", {"kinds": [2004], "since": }] +``` + +Events arrive in real time. On disconnect, exponential backoff reconnect. + +### Resync + +Every `resync_interval` (default 6h), run NIP-77 negentropy against each relay to catch anything missed during disconnects or transient outages. On startup, negentropy bootstrap pulls everything within `backfill_days` (default 365) if `negentropy_bootstrap: true`. + +### Storage + +Even ~100k events takes <500MB SQLite including FTS. Don't preemptively optimize storage; reconsider only at >5M rows. + +## Writer behavior + +### Trigger model + +**Default: nothing publishes.** The writer is opt-in per torrent. Auto-publishing everything that crosses the seedbox is explicitly not supported. + +Two trigger modes: + +**Category-tagged auto-publish.** In qBittorrent/Transmission/Deluge, torrents get a category from `publisher.publish_categories` only when you deliberately set that category. The writer watches those categories; on download completion, it builds the NIP-35 event using the category's configured outbox list and nostr metadata, waits `publish_delay`, and publishes. + +**Manual CLI bulk publish.** `kindexr-cli publish --from-dir /media/library/foo --category publish-public` walks a directory, parses torrents, publishes events. For backfilling an existing library. + +### Publish frequency + +Exactly what you trigger. A single tagged download → one event. Could be 0/day, could be 50/day. Driven by deliberate operator action, never automatic across the whole download set. + +### Sign flow + +1. Build the kind 2003 event from torrent metainfo + optional TMDB enrichment. +2. Open NIP-46 bunker connection (or use ncryptsec/local nsec per config). +3. Send `sign_event` request to bunker over Nostr (encrypted kind 24133). +4. Receive signed event back. +5. For each relay in the resolved outbox list, open WSS and send `["EVENT", ...]`. +6. Wait for `OK` ack; log failures, don't retry forever. +7. Insert the event into local DB as if ingested. + +### Why a bunker, not a local nsec + +The seedbox is the most exposed surface in your stack. An nsec on a publicly-reachable Linux host is a persistent compromise risk. NIP-46 puts the actual signing material on a separate, locked-down host (UTS-01 or your phone running Amber) and gives kindexr a session that can be revoked instantly. Same model as a hardware wallet for Bitcoin. + +## Junk / spam defenses + +Even though Nostr is structurally cleaner than DHT, spam exists. Stacked defenses, ingest to API: + +1. **WoT filter** (`wot_only: true`): only ingest from your follow graph within N hops. Most effective single filter. + +2. **Block list**: kindexr ingests your kind 10000 mute list from configured relays and honors it. + +3. **Curation set subscription**: subscribe to kind 30004 sets from curators you trust; their endorsement boosts visibility of contained events. + +4. **Trusted assertions**: ingest NIP-85 kind 30382 events from operators you trust; aggregate their pubkey ratings. + +5. **NIP-32 labels** (kind 1985): subscribe to label sets marking known-bad pubkeys or info-hashes. + +6. **NIP-56 reports** (kind 1984): aggregate reports against pubkeys; auto-block at configurable threshold (default 5 reports from distinct trusted reporters). + +7. **Category exclusion**: `exclude_categories` drops events with newznab_cat in the excluded set at ingest. XXX range (6000-6099) excluded by default. + +8. **Info-hash dedup**: same info-hash from N different publishers collapses to one row in `torrents`; additional publishers tracked in a join table but don't multiply results. + +9. **API-layer category filter**: Sonarr asking `tvsearch&tvdbid=...&cat=5040` will never return out-of-category results regardless of WoT, because the query filters by `newznab_cat` first. + +The key insight: identity-based filtering scales. When you find a problem publisher, you mute the pubkey and they're gone permanently. Bitmagnet has no equivalent — DHT peers are anonymous and ephemeral. ## Title parser -For events lacking IMDB/TMDB/TVDB tags, parse the title to extract: +For events without `imdb`/`tmdb`/`tvdb` tags, parse the title to extract: -- Show/movie name -- Year (4-digit pattern) -- Season (Sxx, xx digits) -- Episode (Exx) +- Title/year (4-digit pattern) +- Season (Sxx) / Episode (Exx) - Quality (480p|576p|720p|1080p|2160p|SD|HD|UHD) - Source (WEB-DL|WEBRip|BluRay|BDRip|HDTV|DVDRip|REMUX) -- Codec (x264|x265|HEVC|AV1|XViD) +- Codec (x264|x265|HEVC|AV1) - Release group (after final `-`) -Use a port of one of the existing parsers (e.g. `guessit` from Python land is the reference). For Go specifically, look at `github.com/middelink/go-parse-torrent-name` and `github.com/megalol/parsetorrentname` and fork if needed. Don't write this from scratch unless you must. +After parsing, hit TMDB's `search/tv` or `search/movie` with the cleaned name + year, store the resolved IDs, cache aggressively (TMDB IDs are stable). -After parsing, optionally hit TMDB's `search/tv` or `search/movie` endpoint with the cleaned name + year to backfill `imdb_id`/`tmdb_id`/`tvdb_id`. Cache results aggressively (TMDB IDs don't change). +Use an existing Go parser as the base. Don't write from scratch unless forced. -## Web of Trust filter +## CLI -When `curation.wot_only: true`: +`kindexr-cli` for admin tasks. Each subcommand should print clear status, exit non-zero on error. -1. On startup and every N hours, fetch the operator's own kind 3 (Follow List) event. -2. For each followed pubkey, optionally fetch *their* kind 3 (depth 2). Cache. -3. Build `allowed_pubkeys` set: operator + follows + (follows-of-follows if depth ≥ 2) + `allowlist`, minus `blocklist`. -4. Reject any incoming NIP-35 event from a pubkey not in the set. +``` +kindexr-cli apikey list +kindexr-cli apikey create --label sonarr +kindexr-cli apikey revoke -The operator's pubkey is derived from the publisher signer config if present, or set explicitly in `curation.operator_pubkey`. +kindexr-cli relay list +kindexr-cli relay add wss://relay.example.com +kindexr-cli relay remove wss://relay.example.com +kindexr-cli relay enable/disable wss://... + +kindexr-cli publisher list # list known publishers, sorted by trust +kindexr-cli publisher trust <-1.0..1.0> +kindexr-cli publisher block +kindexr-cli publisher unblock + +kindexr-cli stats # events total, by category, by publisher + +kindexr-cli publish --from-dir --category publish-public +kindexr-cli publish --magnet --title --category ... + +kindexr-cli identity init # generate kindexr nsec, walk bunker setup +kindexr-cli identity profile-publish # publish kind 0 for kindexr identity +kindexr-cli identity vouch # publish NIP-85 from main npub vouching for kindexr + +kindexr-cli wot rebuild # force WoT recomputation +kindexr-cli wot status # current allowed pubkey count +``` ## Phase plan -Each phase has explicit acceptance criteria. Don't move to the next phase until current passes. +### Phase 0 — bootstrap ✅ done -### Phase 0 — bootstrap (1 evening) +Scaffolding, config loading, DB migrations, health endpoint, systemd unit. -- [ ] Repo scaffolded with the layout above -- [ ] `cmd/nzbstr/main.go` boots, parses config, opens DB, applies migrations -- [ ] systemd unit installs cleanly -- [ ] `/health` returns 200 with version -- [ ] Logging goes to journald in JSON +### Phase 1 — reader, basic Torznab ✅ done -**Acceptance:** `systemctl start nzbstr` works; `journalctl -u nzbstr` shows clean startup; `curl localhost:9117/health` returns expected JSON. +Relay subscription, NIP-35 parsing, SQLite storage, FTS search, `t=caps`, `t=search`, valid Torznab RSS, Sonarr-add-as-indexer works. -### Phase 1 — reader, basic Torznab (1 weekend) +### Phase 2 — full *arr compatibility 🎯 next -- [ ] Subscribes to configured relays for kind 2003 -- [ ] Parses, validates, stores events into SQLite -- [ ] Implements `t=caps` and `t=search` (generic FTS5 over title) -- [ ] Returns valid Torznab RSS with magnet links -- [ ] Sonarr can add nzbstr as a Torznab indexer and run a test search successfully +- [ ] `t=tvsearch` with tvdbid/imdbid/tmdbid + season/ep filters +- [ ] `t=movie` with imdbid/tmdbid +- [ ] `t=music`, `t=audio` with artist/album/year +- [ ] `t=book` with author/title +- [ ] Title parser integration (port or fork an existing one) +- [ ] TMDB enrichment with caching +- [ ] Newznab category map complete and applied at ingest +- [ ] Default-exclude XXX categories +- [ ] Sanity-check the caps `<categories>` block against a working Jackett response -**Acceptance:** Add nzbstr to Sonarr → "Test" succeeds → manual search for a known popular release returns results with valid magnets that load in qBittorrent. +**Acceptance:** Sonarr automatically finds a known recent episode by tvdbid+season+ep with no manual intervention. Radarr finds a movie by imdbid. Lidarr finds an album. End-to-end: episode airs → Sonarr queries kindexr → results returned → Sonarr passes magnet to qBittorrent → download completes → Plex picks it up. -### Phase 2 — full *arr compatibility (1 weekend) +### Phase 3 — curation -- [ ] `t=tvsearch`, `t=movie`, `t=music`, `t=audio`, `t=book` endpoints -- [ ] Structured ID matching (imdbid/tmdbid/tvdbid) -- [ ] Newznab category normalization complete -- [ ] Title parser fills in season/episode/quality/source when missing -- [ ] TMDB enrichment for events without ID tags +- [ ] WoT graph builder from operator's kind 3 + follows-of-follows +- [ ] kind 10000 mute list ingestion +- [ ] kind 30004 curation set subscription +- [ ] kind 30382 trusted assertion ingestion +- [ ] kind 1985 label ingestion +- [ ] kind 1984 report aggregation with auto-block threshold +- [ ] Trust score computation per publisher +- [ ] CLI commands for publisher trust management -**Acceptance:** Sonarr automatically finds a known recent episode by tvdbid+season+ep with no manual intervention; Radarr finds a known movie by imdbid; Lidarr finds an album by MusicBrainz fields. End-to-end: episode airs → Sonarr queries → nzbstr returns → Sonarr sends to qBittorrent → download completes → Plex picks it up. +**Acceptance:** With `wot_only: true`, ingest from a known-spammy relay yields zero events that aren't in your WoT. Subscribing to a curation set surfaces those events at higher rank. -### Phase 3 — curation (1-2 weekends) +### Phase 4 — writer / publisher -- [ ] WoT filter active when configured -- [ ] NIP-32 label ingestion (kind 1985) -- [ ] NIP-51 set subscription (kind 30004) -- [ ] Per-API-key visibility filter -- [ ] `nzbstr-cli` commands for publisher trust management -- [ ] Block/mute lists honored - -**Acceptance:** Two API keys configured, one "all", one "wot-only" → same query returns different result counts and different publishers. Subscribing to a curation set adds those events to the "curated" visibility. - -### Phase 4 — writer / publisher (2 weekends) - -- [ ] NIP-46 bunker connection working -- [ ] qBittorrent integration: poll for completed torrents in tagged category -- [ ] Build kind 2003 event from torrent metadata +- [ ] NIP-46 bunker client implementation +- [ ] kindexr identity init flow in CLI (`identity init`) +- [ ] qBittorrent webhook/poll integration +- [ ] Build kind 2003 from torrent metainfo +- [ ] Per-category outbox routing - [ ] TMDB enrichment of own publishes -- [ ] Publish to configured outbox relays -- [ ] Stored in local DB as if ingested +- [ ] Publish-delay buffer +- [ ] Insert published events into local DB +- [ ] CLI bulk publish (`publish --from-dir`) +- [ ] CLI identity profile publish + vouch flows -**Acceptance:** Add a torrent to qBittorrent with category `publish-nostr`, wait for download to complete → within 60 seconds, the event appears in your own DB *and* on dtan.xyz. +**Acceptance:** Add a torrent to qBittorrent with category `publish-public`, wait for completion → after `publish_delay` the event appears on configured outbox relays *and* in local DB. Event is fetchable on dtan.xyz. -### Phase 5 — Usenet/Blossom binary bridge (longer horizon) +### Phase 5 — Blossom binary bridge (long horizon) -This is propose-a-NIP territory. Punt to a separate spec doc after Phase 4 is solid. The basic shape: +Defer until Phase 4 is solid and ideally adopted by other operators. Requires proposing a NIP. Shape: -- Sister kind (say, 2005 — must claim, check current NIP registry first) for "Blossom binary release" -- Same tag surface as NIP-35 minus `x`/`tracker` plus `blob` tags pointing to Blossom blob hashes on listed servers -- Downloader sidecar that fetches blobs from configured Blossom servers, reassembles, hands to media library -- This is the Nostr-native Usenet replacement +- Sister kind to NIP-35 for Blossom blob releases +- Same tag surface minus `x`/`tracker`, plus `blob` tags pointing to Blossom hashes on listed servers +- Downloader sidecar that fetches blobs from Blossom servers, reassembles, hands to media library +- Nostr-native Usenet replacement -Don't start Phase 5 until Phase 4 is rock solid and ideally adopted by at least a couple of other people running nzbstr. +## systemd unit -## Things to flag for Eric - -- **NIP-46 bunker on a public-facing seedbox** is the right answer security-wise; consider running a self-hosted bunker on UTS-01 with the nzbstr daemon as a client. Don't put nsec on the seedbox. -- **Title parsing is the part most likely to suck.** Budget more time than you think. The reference implementations all have edge cases. -- **TMDB rate limits** are real (50 req/sec). Implement a circuit breaker. -- **NIP-77 negentropy bootstrap** against six relays for a year of events is slow on first run. Show progress in logs. Consider a "fresh start" mode that skips backfill entirely. -- **The Sonarr `<categories>` block in caps must be exactly right** or Sonarr silently rejects the indexer with no useful error. Copy from a working Jackett/Prowlarr indefinitely. -- **Don't scrape private trackers for seeder counts.** That'll get a publisher's IP burned. Default `health.enabled: false`. -- **Backup the SQLite file**, not just relay subscription state. Use SQLite's online backup API or wrap `sqlite3 .backup`. - -## systemd unit (deploy/nzbstr.service) +(Already deployed in Phase 0; included for completeness.) ```ini [Unit] -Description=nzbstr — Nostr-native Torznab indexer +Description=kindexr — Nostr-native Torznab indexer After=network-online.target Wants=network-online.target [Service] Type=simple -User=nzbstr -Group=nzbstr -ExecStart=/usr/local/bin/nzbstr --config /etc/nzbstr/config.yaml +User=kindexr +Group=kindexr +ExecStart=/usr/local/bin/kindexr --config /etc/kindexr/config.yaml Restart=on-failure RestartSec=5 -# Security hardening NoNewPrivileges=true PrivateTmp=true ProtectSystem=strict ProtectHome=true -ReadWritePaths=/var/lib/nzbstr +ReadWritePaths=/var/lib/kindexr ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true @@ -669,76 +753,53 @@ RestrictNamespaces=true StandardOutput=journal StandardError=journal -SyslogIdentifier=nzbstr +SyslogIdentifier=kindexr [Install] WantedBy=multi-user.target ``` -## Makefile (sketch) +## Things to flag -```makefile -.PHONY: build test clean install run lint +- **The kindexr identity should never be your main npub.** Generate a dedicated nsec, sign via NIP-46 bunker on UTS-01 (or Amber). Walk the trust-bootstrap chain in `docs/IDENTITY.md`. +- **Auto-publish should always be opt-in per torrent** via the category mechanism. Don't add an "auto-publish everything" mode. +- **Title parsing is the part most likely to suck.** Budget extra time. The reference parsers have edge cases for non-English titles, anime, scene tags. +- **TMDB rate limits** (50 req/sec). Implement a circuit breaker. +- **Sonarr `<categories>` block in caps must be exactly right** or Sonarr silently rejects with no useful error. +- **Don't scrape private trackers for seeder counts.** Default `health.enabled: false`. DHT-only when enabled. +- **Back up the SQLite file** via the online backup API or `sqlite3 .backup`. Don't naive-cp a live database. +- **NIP-77 negentropy bootstrap** against many relays is slow on first run. Show progress in logs. Consider a `--skip-bootstrap` flag. -BINARY := nzbstr -CLI := nzbstr-cli -VERSION := $(shell git describe --tags --always --dirty) -LDFLAGS := -ldflags "-X main.version=$(VERSION) -s -w" +## Open questions -build: - go build $(LDFLAGS) -o ./bin/$(BINARY) ./cmd/nzbstr - go build $(LDFLAGS) -o ./bin/$(CLI) ./cmd/nzbstr-cli +Reduced from the original spec, since several got resolved during design discussion. -test: - go test -race ./... - -lint: - golangci-lint run - -install: build - install -D -m 0755 ./bin/$(BINARY) /usr/local/bin/$(BINARY) - install -D -m 0755 ./bin/$(CLI) /usr/local/bin/$(CLI) - install -D -m 0644 ./deploy/nzbstr.service /etc/systemd/system/nzbstr.service - install -D -m 0640 ./deploy/nzbstr.example.yaml /etc/nzbstr/config.yaml - -run: - go run ./cmd/nzbstr --config ./deploy/nzbstr.example.yaml - -clean: - rm -rf ./bin -``` - -## Open questions for the operator - -These should be decided before Phase 1 starts. Default in parens if punted. - -1. Single-user or multi-tenant from day one? *(single-user; multi-tenant comes for free via API keys in Phase 3)* -2. Embed a web browse UI eventually, or keep it pure middleware? *(keep pure; build a separate thin frontend later if needed)* -3. Support kind 2004 comments in search results? *(no in v1, store but don't surface)* -4. Tracker scraping for seeder counts: opt-in via DHT only? *(yes, DHT only, opt-in)* -5. NIP-50 search relays for query offload instead of local FTS? *(no, local FTS is faster and more flexible)* -6. Support fetching blob from Blossom for events that include `["url", ...]` non-magnet links? *(deferred to Phase 5)* -7. Should the writer side also publish kind 1063 (NIP-94 File Metadata) for individual files, or just kind 2003? *(just 2003 in v1)* -8. Rate limit Torznab queries per API key? *(yes, 60/min default, configurable)* +1. **Show kind 2004 comments in Torznab description?** Default: no in v1, store but don't surface. Could add a `<description>` augmentation in Phase 3. +2. **Tracker scraping (UDP/HTTP)** in addition to DHT for seeder counts? Default: no, DHT only when enabled. +3. **NIP-50 search relays** for query offload? Default: no, local FTS is faster and more flexible. +4. **Rate limit Torznab queries per API key?** Default: yes, 60/min. ## Reference material -- NIP-35 spec: `https://github.com/nostr-protocol/nips/blob/master/35.md` -- NIP-46 (remote signing): `https://github.com/nostr-protocol/nips/blob/master/46.md` -- NIP-77 (negentropy sync): `https://github.com/nostr-protocol/nips/blob/master/77.md` -- NIP-32 (labeling): `https://github.com/nostr-protocol/nips/blob/master/32.md` -- NIP-51 (lists): `https://github.com/nostr-protocol/nips/blob/master/51.md` -- Torznab spec: `https://torznab.github.io/spec-1.3-draft/index.html` -- Newznab categories: `https://github.com/Prowlarr/Prowlarr/wiki/Indexer-Categories` -- dtan source for behavior reference: `https://git.v0l.io/Kieran/dtan` -- Example NIP-35 implementer (Jackett): `https://github.com/Jackett/Jackett/pull/16416` -- go-nostr: `https://github.com/nbd-wtf/go-nostr` +- NIP-35: https://github.com/nostr-protocol/nips/blob/master/35.md +- NIP-32 (labeling): https://github.com/nostr-protocol/nips/blob/master/32.md +- NIP-46 (remote signing): https://github.com/nostr-protocol/nips/blob/master/46.md +- NIP-49 (encrypted keys): https://github.com/nostr-protocol/nips/blob/master/49.md +- NIP-51 (lists): https://github.com/nostr-protocol/nips/blob/master/51.md +- NIP-56 (reporting): https://github.com/nostr-protocol/nips/blob/master/56.md +- NIP-77 (negentropy): https://github.com/nostr-protocol/nips/blob/master/77.md +- NIP-85 (trusted assertions): https://github.com/nostr-protocol/nips/blob/master/85.md +- Torznab spec: https://torznab.github.io/spec-1.3-draft/index.html +- Prowlarr category reference: https://github.com/Prowlarr/Prowlarr/wiki/Indexer-Categories +- dtan source (behavior reference): https://git.v0l.io/Kieran/dtan +- Jackett NIP-35 PR (reference impl): https://github.com/Jackett/Jackett/pull/16416 +- go-nostr: https://github.com/nbd-wtf/go-nostr -## Done criteria for "v1.0" +## v1.0 done criteria -- All of Phases 1-4 acceptance criteria pass -- nzbstr has been running on a real seedbox for 30 days without crashing or needing intervention -- At least one external user (someone other than the operator) is running an instance and reporting back +- Phases 1–4 acceptance criteria all pass +- kindexr has been running on the operator's seedbox for 30 days without crashing or needing intervention +- At least one external operator is running an instance - README is good enough that someone new can deploy in under an hour -- A blog post / nostr long-form (kind 30023) explaining what it is and how to run it exists -- Listed in the awesome-nostr README and the relevant *arr community resources +- A kind 30023 long-form post explaining what it is and how to run it exists, signed by the kindexr identity +- Listed in awesome-nostr and relevant *arr community resources