READMEs: updated

This commit is contained in:
nym21
2025-03-17 09:55:06 +01:00
parent 07493ab0a6
commit ad761e388d
20 changed files with 347 additions and 46 deletions

2
Cargo.lock generated
View File

@@ -446,6 +446,7 @@ dependencies = [
name = "brk_exit"
version = "0.0.9"
dependencies = [
"brk_logger",
"ctrlc",
"log",
]
@@ -489,6 +490,7 @@ dependencies = [
"color-eyre",
"env_logger",
"jiff",
"log",
]
[[package]]

View File

@@ -1,16 +1,22 @@
# Bitcoin Research Kit
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk">
<img src="https://img.shields.io/crates/v/brk" alt="Version" />
</a>
<a href="https://docs.rs/brk">
<img src="https://img.shields.io/docsrs/brk" alt="Documentation" />
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<img src="https://img.shields.io/crates/size/brk" alt="Size" />
<a href="https://deps.rs/crate/brk">
<img src="https://deps.rs/crate/brk/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>

View File

@@ -1,6 +1,6 @@
[package]
name = "brk_cli"
description = "A command line interface to interact with the Bitcoin Research Kit"
description = "A command line interface to interact with the full Bitcoin Research Kit"
version.workspace = true
edition.workspace = true
license.workspace = true

View File

@@ -1,6 +1,32 @@
# BRK Cli
## Setup
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_cli">
<img src="https://img.shields.io/crates/v/brk_cli" alt="Version" />
</a>
<a href="https://docs.rs/brk_cli">
<img src="https://img.shields.io/docsrs/brk_cli" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_cli" alt="Size" />
<a href="https://deps.rs/crate/brk_cli">
<img src="https://deps.rs/crate/brk_cli/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A command line interface to interact with the full Bitcoin Research Kit. It's built on top of every other create and gives the possility to use BRK using the terminal instead of Rust.
It has 2 commandes for now (other than `help` and `version`) which are `run` and `query`. The former is used to run the processing (indexer + computer) and/or the server. The latter uses `brk_query` as its backend just like to server to be able to get datasets via the terminal instead of the API. Both commands are very costumizable by having all the parameters of their Rust counterparts ([`run`](https://github.com/bitcoinresearchkit/brk/blob/main/crates/brk_cli/src/run.rs#L91-L147), [`query`](https://github.com/bitcoinresearchkit/brk/blob/main/crates/brk_query/src/params.rs)).
## Requirements
### Hardware
@@ -16,46 +42,29 @@ To be determined
### Software
- [Bitcoin](https://bitcoin.org/en/full-node)
- [Rust](https://www.rust-lang.org/tools/install)
- Unix based operating system (Mac OS or Linux)
- Ubuntu users need to install `open-ssl` via `sudo apt install libssl-dev pkg-config`
- [Bitcoin](https://bitcoin.org/en/full-node)
- Example: `bitcoind -datadir="$HOME/.bitcoin" -blocksonly`
- [Rust](https://www.rust-lang.org/tools/install)
- Install: `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`
- Update: `rustup update`
### Install
## Install
```bash
cargo install brk_cli
cargo install brk # or `cargo install brk_cli`, the result is the same
```
If it's your first time running `brk`, it will need several information such as:
- `--bitcoindir PATH`: path to bitcoin core data directory, `???/bitcoin`
- `--outputdir PATH`: path to various outputs, if you have enough space on your main disk `~/.brk` is fine
Everything will be saved at `~/.brk/config.toml`, which will allow you to simply run `brk run` next time
If you need more options please run `brk -h` to see what parameters are available.
Here's an example
## Update
```bash
brk run --bitcoindir=~/Developer/bitcoin --outputdir=~/.brk
cargo install brk # or `cargo install-update -a` if you have `cargo-update` installed
```
## Usage
Run `brk -h` to view each available command and their respective description.
`-h` works also for commands, which mean that `brk run -h` will explain all the parameters of `brk run` for example.
Every parameter set for `brk run` will be saved at `~/.brk/config.toml`, which will allow you to simply run `brk run` next time.
Then the easiest to let others access your server is to use `cloudflared` which will also cache requests. For more information go to: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/
### Update
```bash
cargo install brk_cli
```
or
```bash
cargo install cargo-update
cargo install-update -a
```

View File

@@ -1 +1,25 @@
# BRK Computer
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_computer">
<img src="https://img.shields.io/crates/v/brk_computer" alt="Version" />
</a>
<a href="https://docs.rs/brk_computer">
<img src="https://img.shields.io/docsrs/brk_computer" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_computer" alt="Size" />
<a href="https://deps.rs/crate/brk_computer">
<img src="https://deps.rs/crate/brk_computer/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A dataset computer, built on top of `brk_indexer` and `brk_fetcher`. It computes any dataset you can think of and if it doesn't feel free to create an issue.

View File

@@ -1 +1,25 @@
# BRK Core
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_core">
<img src="https://img.shields.io/crates/v/brk_core" alt="Version" />
</a>
<a href="https://docs.rs/brk_core">
<img src="https://img.shields.io/docsrs/brk_core" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_core" alt="Size" />
<a href="https://deps.rs/crate/brk_core">
<img src="https://deps.rs/crate/brk_core/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A list of structs that are used throughout the project as units, think of `Date`, `Height`, `Sats`, `Txindex` or anything that can be either a key and/or a value of a dataset.

View File

@@ -7,5 +7,6 @@ license.workspace = true
repository.workspace = true
[dependencies]
ctrlc = "3.4.5"
brk_logger = { workspace = true }
ctrlc = { version = "3.4.5", features = ["termination"] }
log = { workspace = true }

View File

@@ -1 +1,27 @@
# BRK Exit
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_exit">
<img src="https://img.shields.io/crates/v/brk_exit" alt="Version" />
</a>
<a href="https://docs.rs/brk_exit">
<img src="https://img.shields.io/docsrs/brk_exit" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_exit" alt="Size" />
<a href="https://deps.rs/crate/brk_exit">
<img src="https://deps.rs/crate/brk_exit/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A simple crate that stops the program from exitting when blocking is activated until it is released. The purpose of that is to prevent exitting when a program is in the middle of saving data and thus prevent partial writes.
It's built on top of [ctrlc](https://crates.io/crates/ctrlc) which handles Ctrl + C (SIGINT), stopping the program using the `kill` command (SIGTERM) and closing the terminal (SIGHUP) but it doesn't support force kills (`kill -9`).

View File

@@ -0,0 +1,28 @@
use std::{path::Path, thread::sleep, time::Duration};
use brk_exit::Exit;
use log::info;
fn main() {
let exit = Exit::new();
brk_logger::init(Some(Path::new(".log")));
exit.block();
let mut i = 0;
while i < 21 {
info!("i = {i}");
sleep(Duration::from_secs(1));
i += 1;
}
exit.release();
let mut j = 0;
while j < 10 {
info!("j = {j}");
sleep(Duration::from_secs(1));
j += 1;
}
}

View File

@@ -1,4 +1,7 @@
#![doc = include_str!("../README.md")]
#![doc = "\n## Example\n\n```rust"]
#![doc = include_str!("../examples/main.rs")]
#![doc = "```"]
use std::{
process::exit,

View File

@@ -1 +1,25 @@
# BRK Fetcher
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_fetcher">
<img src="https://img.shields.io/crates/v/brk_fetcher" alt="Version" />
</a>
<a href="https://docs.rs/brk_fetcher">
<img src="https://img.shields.io/docsrs/brk_fetcher" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_fetcher" alt="Size" />
<a href="https://deps.rs/crate/brk_fetcher">
<img src="https://deps.rs/crate/brk_fetcher/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A crate that can fetch the Bitcoin price, either by date or height, from multiple APIs such Kraken, Binance and Kibo.money.

View File

@@ -1,6 +1,28 @@
# BRK Indexer
A [Bitcoin Core](https://bitcoincore.org/en/about/) node indexer which iterates over the chain (via `../brk_parser`) and creates a database of the vecs (`../brk_vec`) and key/value stores ([`fjall`](https://crates.io/crates/fjall)) that can be used in your Rust code.
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_indexer">
<img src="https://img.shields.io/crates/v/brk_indexer" alt="Version" />
</a>
<a href="https://docs.rs/brk_indexer">
<img src="https://img.shields.io/docsrs/brk_indexer" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_indexer" alt="Size" />
<a href="https://deps.rs/crate/brk_indexer">
<img src="https://deps.rs/crate/brk_indexer/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A [Bitcoin Core](https://bitcoincore.org/en/about/) node indexer which iterates over the chain (via `../brk_parser`) and creates a database of the vecs (`brk_vec`) and key/value stores ([`fjall`](https://crates.io/crates/fjall)) that can be used in your Rust code.
The crate only stores the bare minimum to be self sufficient and not have to use an RPC client (except for scripts which are not stored). If you need more data, checkout `../computer` which uses the outputs from the indexer to compute a whole range of datasets.

View File

@@ -10,3 +10,4 @@ repository.workspace = true
color-eyre = { workspace = true }
env_logger = "0.11.7"
jiff = { workspace = true }
log = { workspace = true }

View File

@@ -1 +1,27 @@
# BRK Logger
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_logger">
<img src="https://img.shields.io/crates/v/brk_logger" alt="Version" />
</a>
<a href="https://docs.rs/brk_logger">
<img src="https://img.shields.io/docsrs/brk_logger" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_logger" alt="Size" />
<a href="https://deps.rs/crate/brk_logger">
<img src="https://deps.rs/crate/brk_logger/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A simple crate built on top of [`env_logger`](https://crates.io/crates/env_logger) to display logs from the [`log`](https://crates.io/crates/log) crate in a colorful and clean format.
It can also save logs into a file if desired.

View File

@@ -1,3 +1,10 @@
use log::{debug, error, info, trace};
fn main() {
brk_logger::init(None);
info!("info");
debug!("debug");
error!("error");
trace!("trace");
}

View File

@@ -1,5 +1,27 @@
# BRK Parser
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_parser">
<img src="https://img.shields.io/crates/v/brk_parser" alt="Version" />
</a>
<a href="https://docs.rs/brk_parser">
<img src="https://img.shields.io/docsrs/brk_parser" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_parser" alt="Size" />
<a href="https://deps.rs/crate/brk_parser">
<img src="https://deps.rs/crate/brk_parser/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A very fast and simple Rust library which reads raw block files (*blkXXXXX.dat*) from Bitcoin Core node and creates an iterator over all the requested blocks in sequential order (0, 1, 2, ...).
The element returned by the iterator is a tuple which includes the:

View File

@@ -1 +1,31 @@
# BRK Query
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_query">
<img src="https://img.shields.io/crates/v/brk_query" alt="Version" />
</a>
<a href="https://docs.rs/brk_query">
<img src="https://img.shields.io/docsrs/brk_query" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_query" alt="Size" />
<a href="https://deps.rs/crate/brk_query">
<img src="https://deps.rs/crate/brk_query/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A crate that searches for datasets from either `brk_indexer` or `brk_computer` according to given parameters.
It's possible to search for one or multiple dataset if they have the same index and specify range with both the `from` and `to` being optional and supporting negative values.
The output will depend on the format choson which can be Markdown, Json, CSV or TSV and might vary if there is a one or mutiple datasets, and if one dataset one or multiple values.
In the future, it will support more features similar to a real query engine like in a Postgres databases and presets to fetch data grouped by address, transaction or blockhash/height.

View File

@@ -1,6 +1,6 @@
[package]
name = "brk_server"
description = "A server that serves Bitcoin data and swappable front-ends, built on top of brk_indexer, brk_fetcher and brk_computer"
description = "A crate that serves Bitcoin data and swappable front-ends, built on top of brk_indexer, brk_computer and brk_query"
version.workspace = true
edition.workspace = true
license.workspace = true

View File

@@ -1 +1,29 @@
# BRK Server
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_server">
<img src="https://img.shields.io/crates/v/brk_server" alt="Version" />
</a>
<a href="https://docs.rs/brk_server">
<img src="https://img.shields.io/docsrs/brk_server" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_server" alt="Size" />
<a href="https://deps.rs/crate/brk_server">
<img src="https://deps.rs/crate/brk_server/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
A crate that serves Bitcoin data and swappable front-ends, built on top of `brk_indexer`, `brk_computer` and `brk_query`.
The file handler, will serve the website specified by the user if any, which can be *no website*, *kibo.money* or *custom* (which is a blank folder for people to experiment). If a website is specified and the server is ran outside of the brk project and thus can't find the requested website, it will download the whole project with the correct version from Github and store it in `.brk` to be able to serve to website. This is due to the crate size limit on [crates.io](https://crates.io) and the various shenanigans that need to be done to have a website in a crate.
The API uses `brk_query` and so inherites all of its features including formats.

View File

@@ -1,14 +1,32 @@
# BRK Vec
A very small, fast, efficient and simple storable `vec` which uses `mmap2` for speed.
<p align="left">
<a href="https://github.com/bitcoinresearchkit/brk">
<img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
</a>
<a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
<img src="https://img.shields.io/crates/l/brk" alt="License" />
</a>
<a href="https://crates.io/crates/brk_vec">
<img src="https://img.shields.io/crates/v/brk_vec" alt="Version" />
</a>
<a href="https://docs.rs/brk_vec">
<img src="https://img.shields.io/docsrs/brk_vec" alt="Documentation" />
</a>
<img src="https://img.shields.io/crates/size/brk_vec" alt="Size" />
<a href="https://deps.rs/crate/brk_vec">
<img src="https://deps.rs/crate/brk_vec/latest/status.svg" alt="Dependency status">
</a>
<a href="https://discord.gg/EScZAYX4">
<img src="https://img.shields.io/discord/1350431684562124850" alt="Chat" />
</a>
</p>
## Features
A `Vec` (an array) that is stored on disk and thus which can be much larger than the available RAM.
- [x] Get (Rayon compatible)
- [x] Push
- [ ] Update
- [ ] Insert
- [ ] Remove
Compared to a key/value store, the data stored is raw byte interpretation of the Vec's values without any overhead which is very efficient. Additionally it uses close to no RAM when caching isn't active and up to 100 MB when it is.
Compression is also available and built on top [`zstd`](https://crates.io/crates/zstd) to save even more space (from 0 to 75%). The tradeoff being slower reading speeds, especially random reading speeds. This is due to the data being stored in compressed pages of 16 KB, which means that if you to read even one value in that page you have to uncompress the whole page.
## Disclaimer