mirror of
https://github.com/sot-tech/mochi.git
synced 2026-07-18 06:08:10 -07:00
(tested) refactor code
* add sentinel master parameter into driver config * replace yaml double deserialization with `mapstructure` in initializers * replace struct initializers with registered functions * add torrent approval MD and a sanitize rest MDs
This commit is contained in:
+15
-5
@@ -2,12 +2,22 @@
|
||||
|
||||
### Overview
|
||||
|
||||
BitTorrent clients send Announce and Scrape requests to a _Frontend_. Frontends parse requests and write responses for
|
||||
the particular protocol they implement. The _TrackerLogic_ interface is used to generate responses for requests and
|
||||
optionally perform a task after responding to a client. A configurable chain of _PreHook_ and _PostHook_ middleware is
|
||||
used to construct an instance of TrackerLogic. PreHooks are middleware that are executed before the response has been
|
||||
BitTorrent clients send Announce and Scrape requests to a _Frontend_.
|
||||
|
||||
Frontends parse requests and write responses for
|
||||
the particular protocol they implement.
|
||||
|
||||
The _TrackerLogic_ interface is used to generate responses for requests and
|
||||
optionally perform a task after responding to a client.
|
||||
|
||||
A configurable chain of _PreHook_ and _PostHook_ middleware is
|
||||
used to construct an instance of TrackerLogic.
|
||||
|
||||
PreHooks are middleware that are executed before the response has been
|
||||
written. After all PreHooks have executed, any missing response fields that are required are filled by reading out of
|
||||
the configured implementation of the _Storage_ interface. PostHooks are asynchronous tasks that occur after a response
|
||||
the configured implementation of the _Storage_ interface.
|
||||
|
||||
PostHooks are asynchronous tasks that occur after a response
|
||||
has been delivered to the client. Because they are unnecessary to for generating a response, updates to the Storage for
|
||||
a particular request are done asynchronously in a PostHook.
|
||||
|
||||
|
||||
+3
-3
@@ -1,10 +1,10 @@
|
||||
# Frontends
|
||||
|
||||
A _Frontend_ is a component of Chihaya that serves a BitTorrent tracker on one protocol. The frontend accepts, parses
|
||||
A _Frontend_ is a component of MoChi that serves a BitTorrent tracker on one protocol. The frontend accepts, parses
|
||||
and sanitizes requests, passes them to the _Logic_ and writes responses to _Clients_.
|
||||
|
||||
This documentation first gives a high-level overview of Frontends and later goes into implementation specifics. Users of
|
||||
Chihaya are expected to just read the first part - developers should read both.
|
||||
MoChi are expected to just read the first part - developers should read both.
|
||||
|
||||
## Functionality
|
||||
|
||||
@@ -19,7 +19,7 @@ answers each of them with one response, a basic overview of the control flow is:
|
||||
|
||||
## Available Frontends
|
||||
|
||||
Chihaya ships with frontends for HTTP(S) and UDP. The HTTP frontend uses Go's `http` package. The UDP frontend
|
||||
MoChi ships with frontends for HTTP(S) and UDP. The HTTP frontend uses Go's `http` package. The UDP frontend
|
||||
implements both [old-opentracker-style] IPv6 and the IPv6 support specified in [BEP 15]. The advantage of the old
|
||||
opentracker style is that it contains a usable IPv6 `ip` field, to enable IP overrides in announces.
|
||||
|
||||
|
||||
@@ -28,10 +28,10 @@ An example config might look like this:
|
||||
|
||||
```yaml
|
||||
mochi:
|
||||
prehooks:
|
||||
- name: interval variation
|
||||
config:
|
||||
modify_response_probability: 0.2
|
||||
max_increase_delta: 60
|
||||
modify_min_interval: true
|
||||
prehooks:
|
||||
- name: interval variation
|
||||
config:
|
||||
modify_response_probability: 0.2
|
||||
max_increase_delta: 60
|
||||
modify_min_interval: true
|
||||
```
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Approved torrents list
|
||||
|
||||
Package `torrentapproval` can be used for only allow or block
|
||||
specified hashes or block specified hashes.
|
||||
|
||||
## Functionality
|
||||
|
||||
As said above, there are two modes of approval: white list and black list.
|
||||
|
||||
If mode is **white list** (`invert` set to `false`), tracker works in
|
||||
_semi-private_ mode, which means, than peers can could share and receive info
|
||||
about only specified list of torrents' hashes.
|
||||
|
||||
I.e.: if configuration contains hash `AAAA`, but peer announces hash `BBBB`
|
||||
tracker will return `unapproved torrent` message back to peer.
|
||||
|
||||
If mode is **black list** (`invert` set to `true`), tracker will allow all hashes
|
||||
**except** specified.
|
||||
|
||||
## Hash sources
|
||||
|
||||
There are two sources of hashes: `list` and `directory`.
|
||||
|
||||
Both of them used as **INITIAL** source for storing in storage.
|
||||
If storage is not `memory`, records are persisted until _somebody_
|
||||
or _something_ (different tool with access to storage) won't delete it.
|
||||
|
||||
`list` is the static set of hashes, specified in configuration file.
|
||||
|
||||
`directory` will watch for `*.torrent` files in specified path and
|
||||
append/delete records from storage. This source will parse all existing
|
||||
files at start and then watch for new files to add, or for delete events
|
||||
to remove hash from storage.
|
||||
|
||||
## Configuration
|
||||
|
||||
This middleware provides the following parameters for configuration:
|
||||
|
||||
- `initial_source` - source type: `list` or `directory`
|
||||
- `configuration` - options for specified source
|
||||
- `list`:
|
||||
- `hash_list` - list of HEX encoded hashes
|
||||
- `invert` - working mode: `true` - black list, `false` - white list
|
||||
- `storage_ctx` - name of storage _context_ where to store data.
|
||||
It may be redis hash key, DB table name etc.
|
||||
- `directory`:
|
||||
- `path` - directory to watch
|
||||
- `invert` and `storage_ctx` has the same meanins as `list`'s options
|
||||
|
||||
Configuration example:
|
||||
|
||||
An example config might look like this:
|
||||
|
||||
```yaml
|
||||
mochi:
|
||||
prehooks:
|
||||
- name: torrent approval
|
||||
options:
|
||||
initial_source: list
|
||||
configuration:
|
||||
hash_list: ["AAA", "BBB"]
|
||||
invert: false
|
||||
storage_ctx: APPROVED_HASH
|
||||
```
|
||||
+18
-21
@@ -1,17 +1,17 @@
|
||||
# Redis Storage
|
||||
|
||||
This storage implementation separates Chihaya from its storage service. Chihaya achieves HA by storing all peer data in
|
||||
Redis. Multiple instances of Chihaya can use the same redis instance concurrently. The storage service can get HA by
|
||||
clustering. If one instance of Chihaya goes down, peer data will still be available in Redis.
|
||||
This storage implementation separates MoChi from its storage service. MoChi achieves HA by storing all peer data in
|
||||
Redis. Multiple instances of MoChi can use the same redis instance concurrently. The storage service can get HA by
|
||||
clustering. If one instance of MoChi goes down, peer data will still be available in Redis.
|
||||
|
||||
The HA of storage service is not considered here. In case Redis runs as a single node, peer data will be unavailable if
|
||||
the node is down. You should consider setting up a Redis cluster for Chihaya in production.
|
||||
the node is down. You should consider setting up a Redis sentinel (or KeyDB active-active replication) for MoChi in production.
|
||||
|
||||
This storage implementation is currently orders of magnitude slower than the in-memory implementation.
|
||||
|
||||
## Use Case
|
||||
|
||||
When one instance of Chihaya is down, other instances can continue serving peers from Redis.
|
||||
When one instance of MoChi is down, other instances can continue serving peers from Redis.
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -57,27 +57,24 @@ time as value.
|
||||
Here is an example:
|
||||
|
||||
```
|
||||
- IPv4
|
||||
- IPv4_S_<infohash 1>: <modification time>
|
||||
- IPv4_L_<infohash 1>: <modification time>
|
||||
- IPv4_S_<infohash 2>: <modification time>
|
||||
- IPv4_S_<infohash 1>
|
||||
- <peer 1 key>: <modification time>
|
||||
- <peer 2 key>: <modification time>
|
||||
- IPv4_L_<infohash 1>
|
||||
- <peer 3 key>: <modification time>
|
||||
- IPv4_S_<infohash 2>
|
||||
- <peer 3 key>: <modification time>
|
||||
- CHI_4_I
|
||||
- CHI_4_S_<HASH1>
|
||||
- CHI_4_L_<HASH1>
|
||||
- CHI_4_S_<HASH1>
|
||||
- <peer 1 key>: <modification time in unix nanos>
|
||||
- <peer 2 key>: <modification time in unix nanos>
|
||||
- CHI_4_L_<HASH2>
|
||||
- <peer 3 key>: <modification time in unix nanos>
|
||||
...
|
||||
```
|
||||
|
||||
In this case, prometheus would record two swarms, three seeders, and one leecher. These three keys per address family
|
||||
are used to record the count of swarms, seeders, and leechers.
|
||||
|
||||
```
|
||||
- IPv4_infohash_count: 2
|
||||
- IPv4_S_count: 3
|
||||
- IPv4_L_count: 1
|
||||
- CHI_4_S_C: "3"
|
||||
- CHI_6_L_C: "1"
|
||||
```
|
||||
|
||||
Note: IPv4_infohash_count has a different meaning compared to the `memory` storage:
|
||||
It represents the number of infohashes reported by seeder, meaning that infohashes without seeders are not counted.
|
||||
Note: `CHI_4_I` set has a different meaning compared to the `memory` storage:
|
||||
It represents info hashes reported by seeder, meaning that info hashes without seeders are not counted.
|
||||
|
||||
Reference in New Issue
Block a user