release: v0.1.0-alpha.0

Add Docker support

.claude/prompts/changelog.md

@@ -0,0 +1,221 @@
+# Changelog Generation for Claude Code
+
+**TASK**: Update docs/CHANGELOG.md for ALL latest releases missing from the file.
+
+## ⚠️ CRITICAL FAILURE MODE TO AVOID ⚠️
+**THE #1 FAILURE**: Ignoring most changes and only documenting a few
+**ABSOLUTELY FORBIDDEN**: Skipping changes, summarizing with "and other updates", or being incomplete
+**YOU MUST DOCUMENT EVERY SINGLE MEANINGFUL CHANGE - NO EXCEPTIONS**
+
+## CORE WORKFLOW - EXECUTE EXACTLY:
+
+### Step 1: Get Release Information
+```bash
+git tag --list --sort=version:refname
+```
+
+### Step 2: Process ONE Release at a Time
+For each missing release, execute these commands to get complete information:
+
+**First, get the file list:**
+```bash
+git diff --name-only [previous-tag]..[current-tag]
+```
+
+**Then, get changes excluding Cargo.lock:**
+```bash
+git diff [previous-tag]..[current-tag] -- . ':(exclude)Cargo.lock'
+```
+
+**If output is too large, examine files individually:**
+```bash
+git diff [previous-tag]..[current-tag] -- path/to/specific/file.rs
+```
+
+**For summary of changes per file (if needed):**
+```bash
+git diff --stat [previous-tag]..[current-tag]
+```
+
+### Step 3: Analyze Before Writing
+**MANDATORY**: Before writing ANY changelog entry, analyze the diff output and explain:
+- **Use `git diff --name-only` to see ALL changed files** - this prevents truncation issues
+- **Use `git diff -- . ':(exclude)Cargo.lock'` to see actual changes** without Cargo.lock noise
+- **If output is large, examine key files individually** with `git diff [tags] -- path/to/file`
+- **Identify every functional change** - what new capabilities, fixes, or modifications were made
+- What each code change accomplishes functionally
+- The user-facing or system impact of modifications
+- **Which crate each change belongs to** (based on file paths)
+- How changes group together logically within and across crates
+- What functionality is added/removed/modified per crate
+
+**COMPLETENESS CHECK**: State "I have analyzed X files and identified Y distinct functional changes to document"
+
+### Step 4: Write Changelog Entry
+Only after analysis, update CHANGELOG.md with ONE release entry.
+
+**REQUIRED**: End each release entry with a comparison link:
+```markdown
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/vPREVIOUS...vCURRENT)
+```
+Where PREVIOUS is the previous release tag and CURRENT is the current release tag.
+
+### Step 5: Stop and Confirm
+**CRITICAL**: Process only ONE release, then ask for confirmation to continue.
+
+---
+
+## STRICT RULES
+
+### FORBIDDEN - NEVER MENTION:
+- **Cargo.lock** (ignore completely)
+- **Line counts** ("added 50 lines", "removed 200 lines")
+- **Dependency version bumps** (unless enabling major new features)
+- **Local crate version changes** (0.0.61 → 0.0.62)
+- **Vague words**: "Enhanced", "Improved", "Updated", "Refactored"
+
+### FORBIDDEN PHRASES:
+- "and other changes"
+- "various updates"
+- "along with minor improvements"
+- "among other enhancements"
+- **"along with other modifications"**
+- **"plus additional changes"**
+- **"including other updates"**
+- **"and more"**
+- **ANY phrase that suggests you're skipping changes**
+
+### REQUIRED - MUST INCLUDE:
+- **Every meaningful change** in the diff (no shortcuts)
+- **Specific functionality** descriptions
+- **Business impact** of each change
+- **Complete coverage** - if 20 changes exist, document all 20
+- **Source links** for significant changes (new features, breaking changes, major bug fixes)
+
+## WORKSPACE-SPECIFIC RULES
+
+### Crate Identification:
+- **Determine crate from file paths** in the diff (e.g., `crates/brk-core/src/lib.rs` → `brk-core`)
+- **Group all changes by their crate** before writing changelog entries
+- **Use crate name as subheading** under each change type section
+- **For root-level files**: Use `workspace` as the crate name
+
+### Cross-Crate Changes:
+- **When changes span multiple crates** for one feature, mention the relationship
+- **Example**: "Added new transaction API in `brk-core` with corresponding HTTP endpoints in `brk-api`"
+
+### Crate Naming:
+- **Use backticks** around crate names: `brk-core`, `brk-api`
+- **Use workspace structure** as shown in file paths, not display names
+
+### File Header (if missing):
+```markdown
+<!-- This changelog was generated by Claude Code -->
+```
+
+### Release Entry Format:
+```markdown
+## [vX.Y.Z](https://github.com/bitcoinresearchkit/brk/releases/tag/vX.Y.Z) - YYYY-MM-DD
+
+### Breaking Changes
+#### `crate-name`
+- Specific change with functional impact explanation ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/file.rs))
+
+### New Features
+#### `crate-name`
+- Feature description with user benefit ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/main/file.rs))
+- Another feature with implementation details
+
+#### `another-crate`
+- Feature specific to this crate
+
+### Bug Fixes
+#### `crate-name`
+- Specific problem that was resolved ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/file.rs))
+- Another bug fix with impact description
+
+### Internal Changes
+#### `crate-name`
+- Architectural improvement with purpose
+- Code organization change with benefit
+
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/vPREVIOUS...vCURRENT)
+```
+
+---
+
+## ANALYSIS REQUIREMENTS
+
+**Before writing changelog text, you MUST state:**
+
+1. **File discovery**: "Using git diff --name-only, I see X files were modified"
+2. **Cargo.lock check**: "After excluding Cargo.lock, Y files contain actual changes"
+3. **Functional changes identified**: "I identified these distinct changes: A, B, C, D..."
+4. **What each change does**: "Change A: adds new API endpoints, Change B: fixes memory leak, etc."
+5. **Which crates are affected**: "Changes span crates: A, B, C"
+6. **What this means for users**: "Users can now do X, bug Y is fixed, etc."
+7. **How changes group together**: "Changes A and B work together to implement feature W"
+8. **Cross-crate dependencies**: "Crate A's new feature requires the new API in crate B"
+9. **Any unclear changes**: "I cannot determine the purpose of change X from the diff"
+10. **FINAL CHECK**: "I will now document all X functional changes without listing files"
+
+**MANDATORY**: The changelog should describe FUNCTIONALITY, not files. But you must capture ALL functionality changes.
+
+**Only after this analysis, write the changelog entry.**
+
+---
+
+## EXAMPLES
+
+### ❌ BAD (Vague, incomplete):
+```markdown
+### New Features
+- Enhanced blockchain processing capabilities
+- Improved error handling and various other updates
+```
+
+### ✅ GOOD (Specific, complete, grouped by crate, with source links):
+```markdown
+### New Features
+#### `brk-core`
+- Added `TransactionAnalyzer` struct with fee calculation and coinbase detection methods ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-core/src/analyzer.rs))
+- Implemented in-memory caching layer for blockchain queries using HashMap storage
+
+#### `brk-api`
+- Added three new API endpoints: `/api/blocks/{hash}`, `/api/transactions/search`, and `/api/stats/network` ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-api/src/routes.rs))
+- Implemented standardized error responses with error codes and descriptions
+
+### Bug Fixes
+#### `brk-core`
+- Fixed panic when processing blocks with zero transactions by adding explicit empty block validation ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-core/src/block.rs))
+
+#### `brk-api`
+- Resolved memory leak in connection pool by implementing proper cleanup in Drop trait
+
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.107...v0.0.108)
+```
+
+---
+
+## SUCCESS CRITERIA
+
+✅ **COUNT files modified for internal verification only**
+✅ **Identify EVERY distinct functional change**
+✅ **Document ALL functionality changes (no "other changes")**
+✅ **Changelog describes WHAT users can do, not which files changed**
+✅ **Never mention Cargo.lock or line counts**
+✅ **Use specific, functional descriptions**
+✅ **Complete analysis before writing**
+✅ **Stop and ask for confirmation after each release**
+
+**FAILURE INDICATORS - If you do any of these, you FAILED:**
+❌ "and other changes"
+❌ "various updates"
+❌ "among other improvements"
+❌ **Diff shows new API endpoints but changelog doesn't mention them**
+❌ **Diff shows bug fixes but changelog misses some**
+❌ **Diff shows new structs/functions but changelog ignores them**
+❌ Missing obvious functional changes from the diff
+❌ Summarizing instead of listing each distinct functionality change
+
+**KEY PRINCIPLE**: Count files internally to ensure you don't miss changes, but write about FUNCTIONALITY for users.

.claude/prompts/readmes.md

@@ -0,0 +1,172 @@
+# README Generation Prompt
+
+Generate a professional, comprehensive README.md for each crate based SOLELY on code analysis. Use NO external documentation, commit messages, or existing READMEs.
+
+## MANDATORY PROCESS - FOLLOW EXACTLY:
+1. **IGNORE EXISTING DOCS**: Do NOT read any .md, .txt, .rst, or documentation files in the crate directory
+2. **CODE-ONLY ANALYSIS**: Examine ONLY these files:
+   - All .rs files in src/ directory and subdirectories
+   - Cargo.toml for dependencies and metadata
+   - Code structure and organization
+3. **MANDATORY CODE ANALYSIS**: Before writing ANY README content, you MUST:
+   - Examine all Rust files in src/ directory
+   - Identify the main structs, enums, traits, and functions
+   - Understand the crate's architecture and data flow
+   - Determine the crate's purpose from its implementation
+   - Map dependencies to understand external integrations
+4. **FRESH PERSPECTIVE**: Write the README as if you're the first person to document this crate
+5. Generate one complete README.md per crate
+6. Focus on one crate at a time for thorough analysis
+
+## ABSOLUTE REQUIREMENTS:
+- **SOURCE OF TRUTH**: Use ONLY the actual Rust code - no external docs, comments may provide hints but focus on implementation
+- **CRITICAL**: DO NOT read any existing README.md, CHANGELOG.md, or documentation files
+- **IGNORE ALL TEXT FILES**: .md, .txt, .rst files are FORBIDDEN sources - treat them as if they don't exist
+- **CODE ONLY**: Focus exclusively on .rs files, Cargo.toml, and code structure
+- **PROFESSIONAL GRADE**: Write as if this will be published on crates.io for other developers
+- **PROGRAMMER FOCUSED**: Assume audience knows Rust and relevant domain concepts
+- **IMPLEMENTATION-BASED**: Describe what the code actually does, not what comments claim it should do
+- **If you cannot determine functionality from code alone, state this explicitly**
+
+## README STRUCTURE (MANDATORY):
+
+### 1. CRATE HEADER
+```markdown
+# Crate Name
+
+Brief one-line description of what this crate does (max 80 chars).
+
+[![Crates.io](https://img.shields.io/crates/v/CRATE_NAME.svg)](https://crates.io/crates/CRATE_NAME)
+[![Documentation](https://docs.rs/CRATE_NAME/badge.svg)](https://docs.rs/CRATE_NAME)
+```
+
+### 2. OVERVIEW SECTION
+- **Purpose**: What problem does this crate solve?
+- **Key Features**: 3-5 bullet points of main capabilities (derived from code analysis)
+- **Target Use Cases**: Who would use this and for what?
+
+### 3. INSTALLATION
+```toml
+[dependencies]
+crate_name = "X.Y.Z"
+```
+
+### 4. QUICK START / USAGE
+- **Minimal working example** showing the primary API
+- **Common patterns** observed in the code
+- **Key structs/traits** that users will interact with
+
+### 5. API OVERVIEW
+- **Core Types**: Main structs, enums, traits with brief descriptions
+- **Key Methods**: Most important public functions
+- **Module Structure**: Brief overview of how code is organized
+
+### 6. FEATURES (if applicable)
+- Cargo features and what they enable
+- Optional dependencies and their purpose
+
+### 7. EXAMPLES
+- 2-3 practical code examples showing different use cases
+- Based on public API analysis, not existing examples
+
+## WRITING REQUIREMENTS:
+
+### TONE AND STYLE:
+- **Concise but comprehensive**: Every sentence must add value
+- **Technical precision**: Use exact terminology, avoid marketing speak
+- **Active voice**: "Provides X" not "X is provided"
+- **Present tense**: "The crate handles..." not "The crate will handle..."
+
+### FORBIDDEN PATTERNS:
+- **NEVER** use vague terms: "powerful", "flexible", "robust", "comprehensive", "advanced"
+- **NEVER** write marketing copy: "cutting-edge", "state-of-the-art", "enterprise-grade"
+- **NEVER** make claims you can't verify from code: "blazingly fast", "memory efficient"
+- **NEVER** copy-paste from existing documentation or comments
+- **NEVER** read or reference existing README.md files - pretend they don't exist
+- **NEVER** use phrases like "as mentioned in the documentation" or "according to the docs"
+- **NEVER** let existing documentation influence your analysis or writing
+
+### REQUIRED SPECIFICITY:
+- **Data structures**: Mention specific types (HashMap, Vec, etc.)
+- **Algorithms**: Reference actual implementations found in code
+- **Integration points**: Specific traits implemented, dependencies used
+- **Error handling**: How errors are represented and handled
+- **Async/sync**: Clearly state if operations are blocking or async
+
+### ANTI-BIAS PROTOCOL:
+
+### BEFORE STARTING ANY ANALYSIS:
+1. **Explicitly ignore**: Any README.md, CHANGELOG.md, docs/, documentation files
+2. **File filtering**: Only examine .rs and Cargo.toml files
+3. **Fresh eyes approach**: Analyze the code as if you've never seen this crate before
+4. **Independent thinking**: Form your own understanding purely from code inspection
+
+### IF YOU ACCIDENTALLY READ EXISTING DOCS:
+- Stop immediately and restart your analysis
+- Consciously disregard any information from documentation files
+- Base all descriptions solely on what you observe in the code
+- Ask yourself: "What would I think this code does if I had no documentation?"
+
+### VALIDATION CHECKS:
+- **Unique descriptions**: Your descriptions should differ significantly from any existing docs
+- **Code-derived insights**: Every feature mentioned must be visible in the source code
+- **Independent voice**: Write in your own technical style, not mimicking existing documentation
+- **Fresh examples**: Create new code examples based on API analysis, not existing samples
+
+## CODE ANALYSIS DEPTH:
+**You MUST analyze and understand:**
+1. **Public API surface**: All pub structs, functions, traits, modules
+2. **Core abstractions**: Main data types and their relationships
+3. **Error types**: Custom errors, Result patterns, panic conditions
+4. **Dependencies**: How external crates are integrated
+5. **Feature flags**: Conditional compilation and optional functionality
+6. **Async patterns**: Use of futures, tokio, async-std, etc.
+7. **Serialization**: Serde implementations, custom serialization
+8. **Performance characteristics**: Algorithm complexity where obvious
+
+### EXAMPLE STRUCTURE ANALYSIS OUTPUT:
+```markdown
+## Code Analysis Summary
+**Main Types**: `BlockProcessor`, `Transaction`, `ValidationError`
+**Core Trait**: `Validator` - implemented by `BasicValidator` and `StrictValidator`
+**Async Support**: All processing methods return `impl Future`
+**Error Handling**: Custom `ValidationError` enum with specific error types
+**Dependencies**: Uses `tokio` for async runtime, `serde` for serialization
+**Architecture**: Pipeline pattern with configurable validation stages
+```
+
+## EXAMPLES OF QUALITY:
+
+### ❌ BAD (VAGUE):
+```markdown
+# My Crate
+A powerful and flexible library for blockchain operations.
+
+## Features
+- Fast processing
+- Easy to use
+- Robust error handling
+```
+
+### ✅ GOOD (SPECIFIC):
+```markdown
+# brk-chain-analyzer
+Bitcoin blockchain analysis tools for transaction pattern detection.
+
+## Overview
+Provides utilities for analyzing Bitcoin transaction data, detecting address clustering patterns, and computing blockchain statistics. Built around a streaming parser that processes block data without loading entire blocks into memory.
+
+## Key Types
+- `TransactionAnalyzer`: Stateful analyzer for computing fees, detecting coinbase transactions
+- `ClusterDetector`: Implements common input ownership heuristics for address clustering
+- `BlockStream`: Async iterator over blockchain data with configurable batch sizes
+```
+
+## FINAL REQUIREMENTS:
+- **One README per crate** - don't combine multiple crates
+- **Minimum 200 words** - be thorough but concise
+- **Maximum 800 words** - stay focused and relevant
+- **Code examples must be syntactically correct** and compilable
+- **All claims must be verifiable** from the source code
+
+**PROCESS ONE CRATE AT A TIME. ANALYZE THE CODE THOROUGHLY BEFORE WRITING.**

.github/workflows/release.yml

@@ -0,0 +1,296 @@
+# This file was autogenerated by dist: https://axodotdev.github.io/cargo-dist
+#
+# Copyright 2022-2024, axodotdev
+# SPDX-License-Identifier: MIT or Apache-2.0
+#
+# CI that:
+#
+# * checks for a Git Tag that looks like a release
+# * builds artifacts with dist (archives, installers, hashes)
+# * uploads those artifacts to temporary workflow zip
+# * on success, uploads the artifacts to a GitHub Release
+#
+# Note that the GitHub Release will be created with a generated
+# title/body based on your changelogs.
+
+name: Release
+permissions:
+  "contents": "write"
+
+# This task will run whenever you push a git tag that looks like a version
+# like "1.0.0", "v0.1.0-prerelease.1", "my-app/0.1.0", "releases/v1.0.0", etc.
+# Various formats will be parsed into a VERSION and an optional PACKAGE_NAME, where
+# PACKAGE_NAME must be the name of a Cargo package in your workspace, and VERSION
+# must be a Cargo-style SemVer Version (must have at least major.minor.patch).
+#
+# If PACKAGE_NAME is specified, then the announcement will be for that
+# package (erroring out if it doesn't have the given version or isn't dist-able).
+#
+# If PACKAGE_NAME isn't specified, then the announcement will be for all
+# (dist-able) packages in the workspace with that version (this mode is
+# intended for workspaces with only one dist-able package, or with all dist-able
+# packages versioned/released in lockstep).
+#
+# If you push multiple tags at once, separate instances of this workflow will
+# spin up, creating an independent announcement for each one. However, GitHub
+# will hard limit this to 3 tags per commit, as it will assume more tags is a
+# mistake.
+#
+# If there's a prerelease-style suffix to the version, then the release(s)
+# will be marked as a prerelease.
+on:
+  pull_request:
+  push:
+    tags:
+      - '**[0-9]+.[0-9]+.[0-9]+*'
+
+jobs:
+  # Run 'dist plan' (or host) to determine what tasks we need to do
+  plan:
+    runs-on: "ubuntu-22.04"
+    outputs:
+      val: ${{ steps.plan.outputs.manifest }}
+      tag: ${{ !github.event.pull_request && github.ref_name || '' }}
+      tag-flag: ${{ !github.event.pull_request && format('--tag={0}', github.ref_name) || '' }}
+      publishing: ${{ !github.event.pull_request }}
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          submodules: recursive
+      - name: Install dist
+        # we specify bash to get pipefail; it guards against the `curl` command
+        # failing. otherwise `sh` won't catch that `curl` returned non-0
+        shell: bash
+        run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.30.2/cargo-dist-installer.sh | sh"
+      - name: Cache dist
+        uses: actions/upload-artifact@v4
+        with:
+          name: cargo-dist-cache
+          path: ~/.cargo/bin/dist
+      # sure would be cool if github gave us proper conditionals...
+      # so here's a doubly-nested ternary-via-truthiness to try to provide the best possible
+      # functionality based on whether this is a pull_request, and whether it's from a fork.
+      # (PRs run on the *source* but secrets are usually on the *target* -- that's *good*
+      # but also really annoying to build CI around when it needs secrets to work right.)
+      - id: plan
+        run: |
+          dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json
+          echo "dist ran successfully"
+          cat plan-dist-manifest.json
+          echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
+      - name: "Upload dist-manifest.json"
+        uses: actions/upload-artifact@v4
+        with:
+          name: artifacts-plan-dist-manifest
+          path: plan-dist-manifest.json
+
+  # Build and packages all the platform-specific things
+  build-local-artifacts:
+    name: build-local-artifacts (${{ join(matrix.targets, ', ') }})
+    # Let the initial task tell us to not run (currently very blunt)
+    needs:
+      - plan
+    if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
+    strategy:
+      fail-fast: false
+      # Target platforms/runners are computed by dist in create-release.
+      # Each member of the matrix has the following arguments:
+      #
+      # - runner: the github runner
+      # - dist-args: cli flags to pass to dist
+      # - install-dist: expression to run to install dist on the runner
+      #
+      # Typically there will be:
+      # - 1 "global" task that builds universal installers
+      # - N "local" tasks that build each platform's binaries and platform-specific installers
+      matrix: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix }}
+    runs-on: ${{ matrix.runner }}
+    container: ${{ matrix.container && matrix.container.image || null }}
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      BUILD_MANIFEST_NAME: target/distrib/${{ join(matrix.targets, '-') }}-dist-manifest.json
+    steps:
+      - name: enable windows longpaths
+        run: |
+          git config --global core.longpaths true
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          submodules: recursive
+      - name: Install Rust non-interactively if not already installed
+        if: ${{ matrix.container }}
+        run: |
+          if ! command -v cargo > /dev/null 2>&1; then
+            curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
+            echo "$HOME/.cargo/bin" >> $GITHUB_PATH
+          fi
+      - name: Install dist
+        run: ${{ matrix.install_dist.run }}
+      # Get the dist-manifest
+      - name: Fetch local artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: artifacts-*
+          path: target/distrib/
+          merge-multiple: true
+      - name: Install dependencies
+        run: |
+          ${{ matrix.packages_install }}
+      - name: Build artifacts
+        run: |
+          # Actually do builds and make zips and whatnot
+          dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json
+          echo "dist ran successfully"
+      - id: cargo-dist
+        name: Post-build
+        # We force bash here just because github makes it really hard to get values up
+        # to "real" actions without writing to env-vars, and writing to env-vars has
+        # inconsistent syntax between shell and powershell.
+        shell: bash
+        run: |
+          # Parse out what we just built and upload it to scratch storage
+          echo "paths<<EOF" >> "$GITHUB_OUTPUT"
+          dist print-upload-files-from-manifest --manifest dist-manifest.json >> "$GITHUB_OUTPUT"
+          echo "EOF" >> "$GITHUB_OUTPUT"
+
+          cp dist-manifest.json "$BUILD_MANIFEST_NAME"
+      - name: "Upload artifacts"
+        uses: actions/upload-artifact@v4
+        with:
+          name: artifacts-build-local-${{ join(matrix.targets, '_') }}
+          path: |
+            ${{ steps.cargo-dist.outputs.paths }}
+            ${{ env.BUILD_MANIFEST_NAME }}
+
+  # Build and package all the platform-agnostic(ish) things
+  build-global-artifacts:
+    needs:
+      - plan
+      - build-local-artifacts
+    runs-on: "ubuntu-22.04"
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          submodules: recursive
+      - name: Install cached dist
+        uses: actions/download-artifact@v4
+        with:
+          name: cargo-dist-cache
+          path: ~/.cargo/bin/
+      - run: chmod +x ~/.cargo/bin/dist
+      # Get all the local artifacts for the global tasks to use (for e.g. checksums)
+      - name: Fetch local artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: artifacts-*
+          path: target/distrib/
+          merge-multiple: true
+      - id: cargo-dist
+        shell: bash
+        run: |
+          dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json
+          echo "dist ran successfully"
+
+          # Parse out what we just built and upload it to scratch storage
+          echo "paths<<EOF" >> "$GITHUB_OUTPUT"
+          jq --raw-output ".upload_files[]" dist-manifest.json >> "$GITHUB_OUTPUT"
+          echo "EOF" >> "$GITHUB_OUTPUT"
+
+          cp dist-manifest.json "$BUILD_MANIFEST_NAME"
+      - name: "Upload artifacts"
+        uses: actions/upload-artifact@v4
+        with:
+          name: artifacts-build-global
+          path: |
+            ${{ steps.cargo-dist.outputs.paths }}
+            ${{ env.BUILD_MANIFEST_NAME }}
+  # Determines if we should publish/announce
+  host:
+    needs:
+      - plan
+      - build-local-artifacts
+      - build-global-artifacts
+    # Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
+    if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    runs-on: "ubuntu-22.04"
+    outputs:
+      val: ${{ steps.host.outputs.manifest }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          submodules: recursive
+      - name: Install cached dist
+        uses: actions/download-artifact@v4
+        with:
+          name: cargo-dist-cache
+          path: ~/.cargo/bin/
+      - run: chmod +x ~/.cargo/bin/dist
+      # Fetch artifacts from scratch-storage
+      - name: Fetch artifacts
+        uses: actions/download-artifact@v4
+        with:
+          pattern: artifacts-*
+          path: target/distrib/
+          merge-multiple: true
+      - id: host
+        shell: bash
+        run: |
+          dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json
+          echo "artifacts uploaded and released successfully"
+          cat dist-manifest.json
+          echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT"
+      - name: "Upload dist-manifest.json"
+        uses: actions/upload-artifact@v4
+        with:
+          # Overwrite the previous copy
+          name: artifacts-dist-manifest
+          path: dist-manifest.json
+      # Create a GitHub Release while uploading all files to it
+      - name: "Download GitHub Artifacts"
+        uses: actions/download-artifact@v4
+        with:
+          pattern: artifacts-*
+          path: artifacts
+          merge-multiple: true
+      - name: Cleanup
+        run: |
+          # Remove the granular manifests
+          rm -f artifacts/*-dist-manifest.json
+      - name: Create GitHub Release
+        env:
+          PRERELEASE_FLAG: "${{ fromJson(steps.host.outputs.manifest).announcement_is_prerelease && '--prerelease' || '' }}"
+          ANNOUNCEMENT_TITLE: "${{ fromJson(steps.host.outputs.manifest).announcement_title }}"
+          ANNOUNCEMENT_BODY: "${{ fromJson(steps.host.outputs.manifest).announcement_github_body }}"
+          RELEASE_COMMIT: "${{ github.sha }}"
+        run: |
+          # Write and read notes from a file to avoid quoting breaking things
+          echo "$ANNOUNCEMENT_BODY" > $RUNNER_TEMP/notes.txt
+
+          gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
+
+  announce:
+    needs:
+      - plan
+      - host
+    # use "always() && ..." to allow us to wait for all publish jobs while
+    # still allowing individual publish jobs to skip themselves (for prereleases).
+    # "host" however must run to completion, no skipping allowed!
+    if: ${{ always() && needs.host.result == 'success' }}
+    runs-on: "ubuntu-22.04"
+    env:
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          persist-credentials: false
+          submodules: recursive

.gitignore

@@ -1,59 +1,34 @@
 # Mac OS
 .DS_Store
 
-# To do
-/charts
-TODO.md
-
 # Builds
-dist
 target
-
-# I/O
-in
-out
-.log
-/datasets
-/datasets2
-/price
-*..*
-/txout_*
-/db
-
-# Sync
-.stfolder
+websites/dist
+bridge/
+/ids.txt
 
 # Copies
 *\ copy*
 
 # Ignored
-ignore
+_*
 
-# Scripts
-/start-node.sh
+# Logs
+*.log*
 
-# Editors
-.vscode
-.zed
+# Environment variables/configs
+.env
 
-# Configs
-config.toml
-
-# Flamegraph
-flamegraph/
+# Profiling
+profile.json.gz
 flamegraph.svg
+*.trace
+
+# AI
+.claude/settings*
+
+# Expand
+expand.rs
 
 # Benchmarks
-benches
-
-# Snapshots
-snapshots*/
-
-# Docker
-docker/kibo
-
-# Types
-website/scripts/types/paths.d.ts
-
-# Misc
-OPENSATS.md
+[0-9]/

.zed/settings.json

@@ -0,0 +1,22 @@
+{
+  "file_scan_exclusions": [
+    // default
+    "**/.git",
+    "**/.svn",
+    "**/.hg",
+    "**/.jj",
+    "**/CVS",
+    "**/.DS_Store",
+    "**/Thumbs.db",
+    "**/.classpath",
+    "**/.settings",
+    // custom
+    "**/lean-qr/*/index.mjs",
+    "**/modern-screenshot/*/index.mjs",
+    "**/solidjs-signals/*/dist/prod.js",
+    "uFuzzy.mjs",
+    "lightweight-charts.standalone.production.mjs"
+    // "scripts/packages",
+    // "dist"
+  ]
+}

CHANGELOG.md

@@ -1,283 +0,0 @@
-# Changelog
-
-<!--
-## v. 0.X.Y | WIP
-![Image of the kibō Web App version 0.X.Y](./assets/v0.X.Y.jpg)
--->
-
-## v. 0.5.0 | [873199](https://mempool.space/block/0000000000000000000270925aa6a565be92e13164565a3f7994ca1966e48050) - 2024/12/04
-
-![Image of the kibō Web App version 0.5.0](./assets/v0.5.0.jpg)
-
-## Datasets
-
-- Added `Sell Side Risk Ratio` to all entities
-- Added `Open`, `High` and `Low` datasets
-- Added `Satoshis Per Dollar`
-- Added `All Time High`
-- Added `All Time High Date`
-- Added `Days Since All Time High`
-- Added `Max Days Between All Time Highs`
-- Added `Max Years Between All Time Highs`
-- Added `Drawdown`
-- Added `Adjusted Value Created`, `Adjusted Value Destroyed` and `Adjusted Spent Output Profit Ratio` to all entities
-- Added `Realized Profit To Loss Ratio` to all entities
-- Added `Hash Price Min`
-- Added `Hash Price Rebound`
-- Removed all year datasets (25) in favor for epoch datasets (5), the former was too granular to be really useful
-- Removed datasets split by liquidity for all datasets **already split by any address kind**, while fun to have, they took time to compute, ram, and space to store and no one was actually checking them
-- Fixed a lot of values in split by liquidity datasets
-
-## Website
-
-- Updated the design yet again which made the website for something more minimal and easier on the eyes
-- Added a *Save In Bitcoin* (DCA) simulation page
-- ~Added a dashboard~ Added the latest values to the tree next to each option instead, while less values are visible at a time, it's much more readable and organised
-- Added a library of PDFs
-- Fixed service worker not passing 304 (not modified) response and instead serving cached responses
-- Fixed history not being properly registered
-- Fixed window being moveable on iOS when in standalone mode when it shouldn't be
-- Added `Compare` section to all groups, to compare all datasets within a group
-- Updated `Solid Signals` library, which had an important breaking change on the `createEffect` function which might bring some bugs
-- Fixed some datasets paths
-- A lot of code reorg and file splits
-- Adopted a framework like approach to load pages while still being pure JS without a build step
-- Probably more that was forgotten
-
-## Parser
-
-- Added a `/datasets/last` json file with all the latest values
-- Added `--rpcconnect` parameter to the config
-- Added handling of SIGINT and SIGTERM terminal signals which menas you can now safely CTRL+C or kill the parser while it's exporting
-- Added config print at the start of the program
-- Compressed `empty_address_data` struct to save space (should shave of between up to 50% of the `address_index_to_empty_address_data` database)
-- Doubled the number of `txid_to_tx_data` databases from 4096 to 8192
-- ~Added `--recompute_computed true` argument, to allow recomputation of computed datasets in case of a bug~ Buggy for now
-- Fixed not saved arguments, not being processed properly
-- Fixed bug in `generic_map.multi_insert_simple_average`
-- Added defragmentation option `--first-defragment true` of databases to save space (which can save up to 50%)
-- Fixed bug in the computation of averages in `GenericMap`
-- Added support and paramer for cookie files with `--rpccookiefile`, and auto find if the path is `--datadir/.cookie`
-- Increased number of retries and time between them when fetching price from exchanges APIs
-
-## Server
-
-- Fixed links in several places missing the `/api` part and thus not working
-- Fixed broken last values routes
-- Added support for the `/datasets/last` file via the `/api/last` route
-- Added support for `.json` (won't change anything) and `.csv` (will download a csv file) extension at the end of datasets routes
-- Added `all=true` query parameter to dataset routes to get to full history
-
-## Biter
-
-- Moved back to this repo
-
-## v. 0.4.0 | [861950](https://mempool.space/block/00000000000000000000530d0e30ccf7deeace122dcc99f2668a06c6dad83629) - 2024/09/19
-
-![Image of the kibō Web App version 0.4.0](./assets/v0.4.0.jpg)
-
-### Brand
-
-- **Satonomics** is now **kibō** 🎉
-
-### Website
-
-- Complete redesign of the website
-- Rewrote the whole application and removed `node`/`npm`/`pnpm` dependencies in favor for pure `HTML`/`CSS`/`Javascript`
-- Website is now served by the server
-- Added Trading View attribution link to the settings frame and file in the lightweight charts folder
-- Many other changes
-
-### Parser
-
-- Changed the block iterator from a custom version of [bitcoin-explorer](https://crates.io/crates/bitcoin-explorer) to the homemade [biter](https://crates.io/crates/biter) which allows the parser to run alongside `bitcoind`
-- Added datasets compression thanks to [zstd](https://crates.io/crates/zstd) to reduce disk usage
-- Use the Bitcoin RPC server for various calls instead of running cli commands and then parsing the JSON from the output
-- **Important database changes that will need a full rescan**:
-  - Changed databases page size from 1MB to 4KB for improved disk usage
-  - Split txid_to_tx_data database in 4096 chunks (from 256) for improved disk usage
-  - Split address_index_to_X databases to chunks of 25_000 instead of 50_000
-  - Removed local Multisig database
-- Updated the config, run with `-h` to see possible args
-- Moved outputs from `/target/outputs` to `/out` to allow to run commands like `cargo clean` without side effects
-- Various first run fixes
-- Added to `-h` which arguments are saved, which is all of them at the time of writing
-
-### Server
-
-- Updated the code to support compressed binaries
-- Added serving of the website
-- Improved `Cache-Control` behavior
-
-## v. 0.3.0 | [853930](https://mempool.space/block/00000000000000000002eb5e9a7950ca2d5d98bd1ed28fc9098aa630d417985d) - 2024/07/26
-
-![Image of the Satonomics Web App version 0.3.0](./assets/v0.3.0.jpg)
-
-### Parser
-
-- Global
-  - Improved self-hosting by:
-    - Fixing an incredibly annoying bug that made the program panic because of a wrong utxo/address durable state after a or many new datasets were added/changed after a first successful parse of the chain
-    - Fixing a bug that would crash the program if launched for the first time ever
-    - Auto fetch prices from the main Satonomics instance if missing instead of only trying Kraken's and Binance's API which are limited to the last 16 hours
-  - Merged the core of `HeightMap` and `DateMap` structs into `GenericMap`
-  - Added `Height` struct and many others
-  - Reorganized outputs of both the parser and the server for ease of use and easier sync compatibility
-- CLI
-  - Added an argument parser for improved UX with several options
-- Datasets
-  - Added the following datasets for all entities:
-    - Value destroyed
-    - Value created
-    - Spent Output Profit Ratio (SOPR)
-  - Added the following ratio datasets and their variations to all prices {realized, moving average, any cointime, etc}:
-    - Market Price to {X}
-    - Market Price to {X} Ratio
-    - Market Price to {X} Ratio 1 Week SMA
-    - Market Price to {X} Ratio 1 Month SMA
-    - Market Price to {X} Ratio 1 Year SMA
-    - Market Price to {X} Ratio 1 Year SMA Momentum Oscillator
-    - Market Price to {X} Ratio 99th Percentile
-    - Market Price to {X} Ratio 99.5th Percentile
-    - Market Price to {X} Ratio 99.9th Percentile
-    - Market Price to {X} Ratio 1st Percentile
-    - Market Price to {X} Ratio 0.5th Percentile
-    - {X} 1% Top Probability
-    - {X} 0.5% Top Probability
-    - {X} 0.1% Top Probability
-    - {X} 1% Bottom Probability
-    - {X} 0.5% Bottom Probability
-    - {X} 0.1% Bottom Probability
-  - Added block metadatasets and their variants (raw/sum/average/min/max/percentiles):
-    - Block size
-    - Block weight
-    - Block VBytes
-    - Block interval
-- Price
-  - Improved error message when price cannot be found
-
-### App
-
-- General
-  - Added chart scroll button for nice animations à la Wicked
-  - Added scale mode switch (Linear/Logarithmic) at the bottom right of all charts
-  - Added unit at the top left of all charts
-  - Added a backup API in case the main one fails or is offline
-  - Complete redesign of the datasets object
-  - Removed import of routes in JSON in favor for hardcoded typed routes in string format which resulted in:
-    - \+ A much lighter app
-    - \+ Better Lighthouse score
-    - \- Slower Typescript server
-  - Fixed datasets with null values crashing their fetch function
-  - Added a 'Go to a random chart' button in several places
-- Chart
-  - Fixed series color being set to default ones after hovering the legend
-  - Fixed chart starting showing candlesticks and quickly switching to a line when it should've started directly with the line
-  - Separated the QRCode generator library from the main chunk and made it imported on click
-  - Fixed timescale changing on small screen after changing charts
-- Folders
-  - Added the size in the "filename" of address cohorts grouped by size
-- Favorites
-  - Added a 'favorite' and 'unfavorite' button at the bottom
-- Settings
-  - Removed the horizontal scroll bar which was unintended
-
-### Server
-
-- Run file
-  - Only run with a watcher if `cargo watch` is available
-  - Removed id_to_path file in favor for only `paths.d.ts` in `app/src/types`
-
-## v. 0.2.0 | [851286](https://mempool.space/block/0000000000000000000281ca7f1bf8c50702bfca168c7af1bdc67c977c1ac8ed) - 2024/07/08
-
-![Image of the Satonomics Web App version 0.2.0](./assets/v0.2.0.jpg)
-
-### App
-
-- General
-  - Added the height version of all datasets and many optimizations to make them usable but only available on desktop and tablets for now
-  - Added a light theme
-- Charts
-  - Added split panes in order to have the vertical axis visible for all datasets
-  - Added min and max values on the charts
-  - Fixed legend hovering on mobile not resetting on touch end
-  - Added "3 months" and yearly time scale setters (from year 2009 to today)
-  - Hide scrollbar of timescale setters and instead added scroll buttons to the legend only visible on desktop
-  - Improved Share/QR Code screen
-  - Changed all Area series to Line series
-  - Fixed horizontal scrollable legend not updating on preset change
-- Performance
-  - Improved app's reactivity
-  - Added some chunk splitting for a faster initial load
-  - Global improvements that increased the Lighthouse's performance score
-- Settings
-  - Finally made a proper component where you can chose the app's theme, between a moving or static background and its text opacity
-  - Added donations section with a leaderboard
-  - Added various links that are visible on the bottom side of the strip on desktop to mobile users
-  - Added install instructions when not installed for Apple users
-- Misc
-  - Support mini window size, could be useful for embedded views
-  - Hopefully made scrollbars a little more subtle on WIndows and Linux, can't test
-  - Generale style updates
-
-### Parser
-
-- Fixed ulimit only being run in Mac OS instead of whenever the program is detected
-
-## v. 0.1.1 | [849240](https://mempool.space/block/000000000000000000002b8653988655071c07bb5f7181c038f9326bc86db741) - 2024/06/24
-
-![Image of the Satonomics Web App version 0.1.1](./assets/v0.1.1.jpg)
-
-### Parser
-
-- Fixed overflow in `Price` struct which caused many Realized Caps and Realized Prices to have completely bogus data
-- Fixed Realized Cap computation which was using rounded prices instead normal ones
-
-### Server
-
-- Added the chunk, date and time of the request to the terminal logs
-
-### App
-
-- Chart
-  - Added double click option on a legend to toggle the visibility of all other series
-  - Added highlight effect to a legend by darkening the color of all the other series on the chart while hovering it with the mouse
-  - Added an API link in the legend for each dataset where applicable (when isn't generated locally)
-  - Save fullscreen preference in local storage and url
-  - Improved resize bar on desktop
-  - Changed resize button logo
-  - Changed the share button to visible on small screen too
-  - Improved share screen
-  - Fixed time range shifting not being the one in url params or saved in local storage
-  - Fixed time range shifting on series toggling via the legend
-  - Fixed time range shifting on fullscreen
-  - Fixed time range shifting on resize of the sidebar
-  - Set default view at first load to last 6 months
-  - Added some padding around the datasets (year 1970 to 2100)
-- History
-  - Changed background for the sticky dates from blur to a solid color as it didn't appear properly in Firefox
-- Build
-  - Tried to add lazy loads to have split chunks after build, to have much faster load times and they worked great ! But they completely broke Safari on iOS, we can't have nice things
-  - Removed many libraries and did some things manually instead to improve build size
-- Strip
-  - Temporarily removed the Home button on the strip bar on desktop as there is no landing page yet
-- Settings
-  - Added version
-- PWA
-  - Fixed background update
-  - Changed update check frequency to 1 minute (~1kb to fetch every minute which is very reasonable)
-  - Added a nice banner to ask the user to install the update
-- Misc
-  - Removed tracker even though it was a very privacy friendly as it appeared to not be working properly
-
-### Price
-
-- Deleted old price datasets and their backups
-
-## v. 0.1.0 | [848642](https://mempool.space/block/000000000000000000020be5761d70751252219a9557f55e91ecdfb86c4e026a) - 2024/06/19
-
-![Image of the Satonomics Web App version 0.1.0](./assets/v0.1.0.jpg)
-
-## v. 0.0.X | [835444](https://mempool.space/block/000000000000000000009f93907a0dd83c080d5585cc7ec82c076d45f6d7c872) - 2024/03/20
-
-![Image of the Satonomics Web App version 0.0.X](./assets/v0.0.X.jpg)

CONTRIBUTING.md

@@ -1,8 +0,0 @@
-# Guidelines
-
-## Parser
-
-- Avoid floats as much as possible
-  - Use structs like `WAmount` and `Price` for calculations
-  - **Only** use `WAmount.to_btc()` when inserting or computing inside a dataset. It is **very** expensive.
-- No `Arc`, `Rc`, `Mutex` even from third party libraries, they're slower

Cargo.toml

@@ -0,0 +1,99 @@
+[workspace]
+resolver = "3"
+members = ["crates/*"]
+package.description = "The Bitcoin Research Kit is a suite of tools designed to extract, compute and display data stored on a Bitcoin Core node"
+package.license = "MIT"
+package.edition = "2024"
+package.version = "0.1.0-alpha.0"
+package.homepage = "https://bitcoinresearchkit.org"
+package.repository = "https://github.com/bitcoinresearchkit/brk"
+package.readme = "README.md"
+
+[profile.dev]
+lto = "thin"
+codegen-units = 16
+opt-level = 2
+split-debuginfo = "unpacked"
+
+[profile.release]
+lto = "fat"
+codegen-units = 1
+panic = "abort"
+strip = true
+overflow-checks = false
+
+[profile.bloaty]
+debug = true
+lto = false
+strip = false
+inherits = "release"
+
+[profile.dist]
+inherits = "release"
+
+[profile.profiling]
+inherits = "release"
+debug = true
+
+[workspace.dependencies]
+aide = { version = "0.16.0-alpha.1", features = ["axum-json", "axum-query"] }
+axum = "0.8.7"
+bitcoin = { version = "0.32.8", features = ["serde"] }
+bitcoincore-rpc = "0.19.0"
+brk_bencher = { version = "0.1.0-alpha.0", path = "crates/brk_bencher" }
+brk_binder = { version = "0.1.0-alpha.0", path = "crates/brk_binder" }
+brk_bundler = { version = "0.1.0-alpha.0", path = "crates/brk_bundler" }
+brk_cli = { version = "0.1.0-alpha.0", path = "crates/brk_cli" }
+brk_computer = { version = "0.1.0-alpha.0", path = "crates/brk_computer" }
+brk_error = { version = "0.1.0-alpha.0", path = "crates/brk_error" }
+brk_fetcher = { version = "0.1.0-alpha.0", path = "crates/brk_fetcher" }
+brk_grouper = { version = "0.1.0-alpha.0", path = "crates/brk_grouper" }
+brk_indexer = { version = "0.1.0-alpha.0", path = "crates/brk_indexer" }
+brk_query = { version = "0.1.0-alpha.0", path = "crates/brk_query", features = ["tokio"] }
+brk_iterator = { version = "0.1.0-alpha.0", path = "crates/brk_iterator" }
+brk_logger = { version = "0.1.0-alpha.0", path = "crates/brk_logger" }
+brk_mcp = { version = "0.1.0-alpha.0", path = "crates/brk_mcp" }
+brk_mempool = { version = "0.1.0-alpha.0", path = "crates/brk_mempool" }
+brk_reader = { version = "0.1.0-alpha.0", path = "crates/brk_reader" }
+brk_rpc = { version = "0.1.0-alpha.0", path = "crates/brk_rpc" }
+brk_server = { version = "0.1.0-alpha.0", path = "crates/brk_server" }
+brk_store = { version = "0.1.0-alpha.0", path = "crates/brk_store" }
+brk_types = { version = "0.1.0-alpha.0", path = "crates/brk_types" }
+brk_traversable = { version = "0.1.0-alpha.0", path = "crates/brk_traversable", features = ["pco", "derive"] }
+brk_traversable_derive = { version = "0.1.0-alpha.0", path = "crates/brk_traversable_derive" }
+byteview = "0.9.1"
+color-eyre = "0.6.5"
+derive_deref = "1.1.1"
+fjall = "3.0.0-rc.6"
+# fjall3 = { path = "../fjall3", package = "fjall" }
+# fjall3 = { git = "https://github.com/fjall-rs/fjall.git", rev = "434979ef59d8fd2b36b91e6ff759a36d19a397ee", package = "fjall" }
+jiff = "0.2.16"
+log = "0.4.29"
+mimalloc = { version = "0.1.48", features = ["v3"] }
+minreq = { version = "2.14.1", features = ["https", "serde_json"] }
+parking_lot = "0.12.5"
+rayon = "1.11.0"
+rustc-hash = "2.1.1"
+schemars = "1.1.0"
+serde = "1.0.228"
+serde_bytes = "0.11.19"
+serde_derive = "1.0.228"
+serde_json = { version = "1.0.145", features = ["float_roundtrip"] }
+smallvec = "1.15.1"
+tokio = { version = "1.48.0", features = ["rt-multi-thread"] }
+vecdb = { version = "0.4.3", features = ["derive", "serde_json", "pco"] }
+# vecdb = { path = "../anydb/crates/vecdb", features = ["derive", "serde_json", "pco"] }
+# vecdb = { git = "https://github.com/anydb-rs/anydb", features = ["derive", "serde_json", "pco"] }
+
+[workspace.metadata.release]
+shared-version = true
+tag-name = "v{{version}}"
+pre-release-commit-message = "release: v{{version}}"
+tag-message = "release: v{{version}}"
+
+[workspace.metadata.dist]
+cargo-dist-version = "0.30.2"
+ci = "github"
+allow-dirty = ["ci"]
+installers = []
+targets = ["aarch64-apple-darwin", "aarch64-unknown-linux-gnu", "x86_64-unknown-linux-gnu"]

LICENSE.md

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2024 kibō
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

README.md

@@ -1,166 +0,0 @@
-<a href="https://kibo.money" target="_blank">
-  <picture>
-    <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/kibo-money/kibo/main/assets/logo-long-text-dark.svg">
-    <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/kibo-money/kibo/main/assets/logo-long-text-light.svg">
-    <img alt="kibō" src="https://raw.githubusercontent.com/kibo-money/kibo/main/assets/logo-long-text-light.svg" width="210" height="auto">
-  </picture>
-</a>
-
-## Description
-
-[**kibō**](https://kibo.money) (_hope_ in japanese) is primarily an open source Bitcoin Core data extractor and visualizer (similar to [Glassnode](https://glassnode.com)) which goal is to empower anybody with data about Bitcoin for free.
-
-The project is split in 3 parts:
-
-- First you have the extractor (parser), which parses the block data files from your Bitcoin Core node and computes a very wide range of datasets which are stored in compressed binary files
-  > For the curious, it takes at the very least 24 hours to parse all the blocks and compute all datasets. After that it will wait for a new block and take between 1 and 3 minutes to be up to date
-- Then there is the website on which you can view, among other things, all datasets in various charts
-- Finally there is the server which serves the website and the generated data via an [API](https://github.com/kibo-money/kibo/tree/main#endpoints)
-
-Whether you're an enthusiast, a researcher, a miner, an analyst, a trader, a skeptic or just curious, there is something for everyone !
-
-This project was created out of frustration by all the alternatives that were either very expensive and thus discriminatory and against bitcoin values or just very limited and none were open-source and verifiable. So while it's not the first tool trying to solve these problems, it's the first that is completely free, open-source and self-hostable.
-
-If you are a user of [mempool.space](https://mempool.space), you'll find this to be very complimentary, as it offers a macro view of the chain over time instead of a detailed one.
-
-## Instances
-
-| URL                                              | Type   | Version                                                                                                                                                                         | Status                                                                                                                                                         | Last Height                                                                                                                                                                      | Up Time Ratio                                                                                                                        |
-| ------------------------------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
-| [kibo.money](https://kibo.money)                 | Main   | ![Version](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fkibo.money%2FCargo.toml&query=%24.package.version&style=for-the-badge&label=%20&color=%23db9e03)         | ![Status](https://img.shields.io/uptimerobot/status/m797259009-043f6b92d4cc2deef7d13f50?style=for-the-badge&label=%20&up_color=%231cb454&down_color=%23e63636) | ![Height](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fkibo.money%2Fapi%2Flast-height.json&query=%24.value&style=for-the-badge&label=%20&color=%23f26610)         | ![Ratio](https://img.shields.io/uptimerobot/ratio/m797259009-043f6b92d4cc2deef7d13f50?style=for-the-badge&label=%20&color=%232f73f1) |
-| [backup.kibo.money](https://backup.kibo.money)   | Backup | ![Version](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fbackup.kibo.money%2FCargo.toml&query=%24.package.version&style=for-the-badge&label=%20&color=%23db9e03)  | ![Status](https://img.shields.io/uptimerobot/status/m797259013-bb29a8264fab8786fb80c5ed?style=for-the-badge&label=%20&up_color=%231cb454&down_color=%23e63636) | ![Height](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fbackup.kibo.money%2Fapi%2Flast-height.json&query=%24.value&style=for-the-badge&label=%20&color=%23f26610)  | ![Ratio](https://img.shields.io/uptimerobot/ratio/m797259013-bb29a8264fab8786fb80c5ed?style=for-the-badge&label=%20&color=%232f73f1) |
-| [preview.kibo.money](https://preview.kibo.money) | Dev    | ![Version](https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fpreview.kibo.money%2FCargo.toml&query=%24.package.version&style=for-the-badge&label=%20&color=%23db9e03) | ![Status](https://img.shields.io/uptimerobot/status/m797869753-d40fc161bcb34624857a8082?style=for-the-badge&label=%20&up_color=%231cb454&down_color=%23e63636) | ![Height](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fpreview.kibo.money%2Fapi%2Flast-height.json&query=%24.value&style=for-the-badge&label=%20&color=%23f26610) | ![Ratio](https://img.shields.io/uptimerobot/ratio/m797869753-d40fc161bcb34624857a8082?style=for-the-badge&label=%20&color=%232f73f1) |
-
-Please open an issue if you want to add another instance
-
-## Endpoints
-
-> If you running locally, you can replace `https://kibo.money` by `http://localhost:3110`
-
-- [/](https://kibo.money/): Website
-- [/api](https://kibo.money/api): A JSON with all available datasets, with their respective id and endpoint, better viewed in a Firefox based browser
-- /api/TIMESCALE-to-ID: `TIMESCALE` can be `date` or `height`, and `ID` is the id with `_` replaced by `-`, let's take `date-to-close` (price at the end of each day) as an example
-  - [/api/date-to-close](https://kibo.money/api/date-to-close): current year's values in a json format
-  - [/api/date-to-close?chunk=2009](https://kibo.money/api/date-to-close?chunk=2009): values from the year 2009 in a json format
-  - [/api/date-to-close?all=true](https://kibo.money/api/date-to-close?all=true): all values in a json format
-  - You can also specify the extension to download a file, either `.json` or `.csv` to get the dataset in a CSV format; like so:
-    - [/api/date-to-close.csv](https://kibo.money/api/date-to-close.csv)
-    - [/api/date-to-close.csv?chunk=2009](https://kibo.money/api/date-to-close.csv?chunk=2009)
-    - [/api/date-to-close.csv?all=true](https://kibo.money/api/date-to-close.csv?all=true)
-
-## Roadmap
-
-- **More Datasets/Charts**
-- **Simulations**
-- **Nostr integration**
-- **API Documentation**
-- **Descriptions**
-- **Docker support**
-- **Start9 support**
-
-## Setup
-
-### Requirements
-
-- At least 16 GB of RAM
-- 1 TB of free space (will use 70% of that without defragmentation and 40% after)
-- A running instance of bitcoin-core with:
-  - `-txindex=1`
-  - `-blocksxor=0`
-  - RPC credentials
-  - Example: `bitcoind -datadir="$HOME/.bitcoin" -blocksonly -txindex=1 -blocksxor=0`
-- Git
-
-### Manual
-
-_Mac OS and Linux only, Windows is unsupported_
-
-First we need to install Rust (https://www.rust-lang.org/tools/install)
-
-```bash
-curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
-```
-
-If you already had Rust installed you could update it just in case
-
-```bash
-rustup update
-```
-
-> If you're on Ubuntu you'll probably also need to install `open-ssl` with
->
-> ```bash
-> sudo apt install libssl-dev pkg-config
-> ```
-
-Optionally, you can also install `cargo-watch` for the server to automatically restart it on file change, which will be triggered by new code and new datasets from the parser (https://github.com/watchexec/cargo-watch?tab=readme-ov-file#install)
-
-```bash
-cargo install cargo-watch --locked
-```
-
-Then you need to choose a path where all files related to **kibō** will live
-
-```bash
-cd ???
-```
-
-We can now clone the repository
-
-```bash
-git clone https://github.com/kibo-money/kibo.git
-```
-
-In a new terminal, go to the `parser`'s folder of the repository
-
-```bash
-cd ???/kibo/parser
-```
-
-Now we can finally start by running the parser, you need to use the `./run.sh` script instead of `cargo run -r` as we need to set various system variables for the program to run smoothly
-
-For the first launch, the parser will need several information such as:
-
-- `--datadir`: which is bitcoin data directory path, prefer `$HOME` to `~` as the latter might not work
-
-Optionally you can also specify:
-
-- `--rpccookiefile`: the path to the cookie file if not default
-- `--rpcuser`: the username of the RPC credentials to talk to the bitcoin server if set
-- `--rpcpassword`: the password of the RPC credentials if set
-- `--rpcconnect`: if the bitcoin core server's IP is different than `localhost`
-- `--rpcport`: if the port is different than `8332`
-
-Everything will be saved in a `config.toml` file, which will allow you to simply run `./run.sh` next time
-
-Here's an example
-
-```bash
-./run.sh --datadir=$HOME/Developer/bitcoin
-```
-
-In a **new** terminal, go to the `server`'s folder of the repository
-
-```bash
-cd ???/kibo/server
-```
-
-And start it also with the `run.sh` script instead of `cargo run -r`
-
-```bash
-./run.sh
-```
-
-Then the easiest to let others access your server is to use `cloudflared` which will also cache requests. For more information go to: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/
-
-## Donate
-
-<img width="159" alt="image" src="https://github.com/user-attachments/assets/8bbb759f-4874-46cb-b093-b30cb30f5828">
-
-[bc1q950q4ukpxxm6wjjkv6cpq8jzpazaxrrwftctkt](bitcoin:bc1q950q4ukpxxm6wjjkv6cpq8jzpazaxrrwftctkt)
-
-<img width="159" alt="image" src="https://github.com/user-attachments/assets/745e39c7-be26-4f2a-90f2-54786e62ba35">
-
-[lnurl1dp68gurn8ghj7ampd3kx2ar0veekzar0wd5xjtnrdakj7tnhv4kxctttdehhwm30d3h82unvwqhkxmmww3jkuar8d35kgetj8yuq363hv4](lightning:lnurl1dp68gurn8ghj7ampd3kx2ar0veekzar0wd5xjtnrdakj7tnhv4kxctttdehhwm30d3h82unvwqhkxmmww3jkuar8d35kgetj8yuq363hv4)
-
-[Geyser Fund](https://geyser.fund/project/kibo/)

assets/logo-dove-dark.svg

@@ -1,4 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg">
-  <path d="M 10.874 7.57 L 10.874 6.802 C 10.104 5.809 9.587 4.634 9.397 3.379 C 9.339 3.009 8.877 2.865 8.637 3.152 C 8.059 3.833 7.605 4.632 7.299 5.517 C 8.234 6.565 9.486 7.284 10.874 7.57 Z M 13.937 4.749 C 12.729 4.749 11.751 5.731 11.751 6.939 L 11.751 8.563 C 8.895 8.394 6.474 6.636 5.379 4.142 C 5.229 3.8 4.746 3.78 4.585 4.117 C 4.131 5.077 3.876 6.149 3.876 7.28 C 3.876 9.217 4.808 11.025 6.203 12.364 C 6.562 12.711 6.915 12.998 7.266 13.261 L 3.331 14.245 C 3.038 14.318 2.907 14.658 3.072 14.912 C 3.547 15.648 4.723 16.895 7.261 16.999 C 7.48 17.007 7.698 16.928 7.864 16.783 L 9.648 15.249 L 11.751 15.249 C 14.167 15.249 16.125 13.293 16.125 10.876 L 16.125 6.499 L 17 4.749 L 13.937 4.749 Z M 13.937 7.391 C 13.697 7.391 13.458 7.175 13.458 6.935 C 13.458 6.695 13.697 6.483 13.937 6.483 C 14.177 6.483 14.399 6.703 14.399 6.943 C 14.399 7.183 14.177 7.391 13.937 7.391 Z" style="fill: #12100f;"></path>
-</svg>

assets/logo-dove-light.svg

@@ -1,4 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg">
-  <path d="M 10.874 7.57 L 10.874 6.802 C 10.104 5.809 9.587 4.634 9.397 3.379 C 9.339 3.009 8.877 2.865 8.637 3.152 C 8.059 3.833 7.605 4.632 7.299 5.517 C 8.234 6.565 9.486 7.284 10.874 7.57 Z M 13.937 4.749 C 12.729 4.749 11.751 5.731 11.751 6.939 L 11.751 8.563 C 8.895 8.394 6.474 6.636 5.379 4.142 C 5.229 3.8 4.746 3.78 4.585 4.117 C 4.131 5.077 3.876 6.149 3.876 7.28 C 3.876 9.217 4.808 11.025 6.203 12.364 C 6.562 12.711 6.915 12.998 7.266 13.261 L 3.331 14.245 C 3.038 14.318 2.907 14.658 3.072 14.912 C 3.547 15.648 4.723 16.895 7.261 16.999 C 7.48 17.007 7.698 16.928 7.864 16.783 L 9.648 15.249 L 11.751 15.249 C 14.167 15.249 16.125 13.293 16.125 10.876 L 16.125 6.499 L 17 4.749 L 13.937 4.749 Z M 13.937 7.391 C 13.697 7.391 13.458 7.175 13.458 6.935 C 13.458 6.695 13.697 6.483 13.937 6.483 C 14.177 6.483 14.399 6.703 14.399 6.943 C 14.399 7.183 14.177 7.391 13.937 7.391 Z" style="fill: #fffaf6;"></path>
-</svg>

assets/logo-dove-orange.svg

@@ -1,4 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 20 20" xmlns="http://www.w3.org/2000/svg">
-  <path d="M 10.874 7.57 L 10.874 6.802 C 10.104 5.809 9.587 4.634 9.397 3.379 C 9.339 3.009 8.877 2.865 8.637 3.152 C 8.059 3.833 7.605 4.632 7.299 5.517 C 8.234 6.565 9.486 7.284 10.874 7.57 Z M 13.937 4.749 C 12.729 4.749 11.751 5.731 11.751 6.939 L 11.751 8.563 C 8.895 8.394 6.474 6.636 5.379 4.142 C 5.229 3.8 4.746 3.78 4.585 4.117 C 4.131 5.077 3.876 6.149 3.876 7.28 C 3.876 9.217 4.808 11.025 6.203 12.364 C 6.562 12.711 6.915 12.998 7.266 13.261 L 3.331 14.245 C 3.038 14.318 2.907 14.658 3.072 14.912 C 3.547 15.648 4.723 16.895 7.261 16.999 C 7.48 17.007 7.698 16.928 7.864 16.783 L 9.648 15.249 L 11.751 15.249 C 14.167 15.249 16.125 13.293 16.125 10.876 L 16.125 6.499 L 17 4.749 L 13.937 4.749 Z M 13.937 7.391 C 13.697 7.391 13.458 7.175 13.458 6.935 C 13.458 6.695 13.697 6.483 13.937 6.483 C 14.177 6.483 14.399 6.703 14.399 6.943 C 14.399 7.183 14.177 7.391 13.937 7.391 Z" style="fill: #f26610;"></path>
-</svg>

assets/logo-full-dark.svg

@@ -1,11 +0,0 @@
-<svg viewBox="0 0 720 180" xmlns="http://www.w3.org/2000/svg">
-  <defs></defs>
-  <g transform="matrix(7.5, 0, 0, 7.5, -2046.71228, -1592.744873)">
-    <ellipse style="fill: #f26610;" cx="284.895" cy="224.366" rx="12" ry="12"></ellipse>
-    <path d="M 285.769 221.936 L 285.769 221.168 C 284.999 220.175 284.482 219 284.292 217.745 C 284.234 217.375 283.772 217.231 283.532 217.518 C 282.954 218.199 282.5 218.998 282.194 219.883 C 283.129 220.931 284.381 221.65 285.769 221.936 Z M 288.832 219.115 C 287.624 219.115 286.646 220.097 286.646 221.305 L 286.646 222.929 C 283.79 222.76 281.369 221.002 280.274 218.508 C 280.124 218.166 279.641 218.146 279.48 218.483 C 279.026 219.443 278.771 220.515 278.771 221.646 C 278.771 223.583 279.703 225.391 281.098 226.73 C 281.457 227.077 281.81 227.364 282.161 227.627 L 278.226 228.611 C 277.933 228.684 277.802 229.024 277.967 229.278 C 278.442 230.014 279.618 231.261 282.156 231.365 C 282.375 231.373 282.593 231.294 282.759 231.149 L 284.543 229.615 L 286.646 229.615 C 289.062 229.615 291.02 227.659 291.02 225.242 L 291.02 220.865 L 291.895 219.115 L 288.832 219.115 Z M 288.832 221.757 C 288.592 221.757 288.353 221.541 288.353 221.301 C 288.353 221.061 288.592 220.849 288.832 220.849 C 289.072 220.849 289.294 221.069 289.294 221.309 C 289.294 221.549 289.072 221.757 288.832 221.757 Z" style="fill: #fffaf6;"></path>
-  </g>
-  <g transform="matrix(1, 0, 0, 1, -30, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: #fffaf6;"></path>
-    <path d="M 589.19 97.802 L 589.19 106.23 L 610.948 106.23 C 605.1 112.938 597.446 119.044 587.986 124.376 L 593.404 131.514 C 597.532 128.934 601.488 126.268 605.186 123.43 L 605.186 146.048 L 614.13 146.048 L 614.13 123.43 L 626.944 123.43 L 626.944 149.402 L 635.974 149.402 L 635.974 123.43 L 649.82 123.43 L 649.82 134.008 C 649.82 136.072 649.046 137.104 647.498 137.104 L 640.36 136.674 L 642.768 145.188 L 650.422 145.188 C 655.926 145.188 658.678 142.092 658.678 135.986 L 658.678 115.174 L 635.974 115.174 L 635.974 108.638 L 626.944 108.638 L 626.944 115.174 L 614.388 115.174 C 617.054 112.336 619.548 109.326 621.784 106.23 L 665.128 106.23 L 665.128 97.802 L 626.858 97.802 C 627.89 95.824 628.836 93.76 629.696 91.61 L 620.838 90.492 C 619.806 92.9 618.516 95.394 617.14 97.802 L 589.19 97.802 Z M 648.1 68.734 C 642.338 72.088 636.232 75.098 629.868 77.678 C 621.612 75.012 612.926 72.518 603.896 70.282 L 599.252 77.248 C 605.272 78.624 611.206 80.258 617.226 82.15 C 610.088 84.386 602.606 86.106 594.78 87.482 L 599.596 95.308 C 612.324 92.04 622.472 89.116 630.04 86.364 C 638.124 89.116 646.122 92.298 654.034 95.824 L 658.936 88.428 C 653.26 86.02 647.412 83.698 641.392 81.548 C 646.208 79.226 651.11 76.56 655.926 73.55 L 648.1 68.734 Z M 675.438 77.85 L 675.438 85.848 L 682.404 85.848 L 682.404 98.92 C 682.404 101.5 681.114 103.22 678.62 104.166 L 680.684 110.874 C 692.036 108.896 701.926 106.66 710.182 104.08 L 708.634 96.426 C 703.474 98.146 697.454 99.608 690.574 100.984 L 690.574 85.848 L 712.332 85.848 L 712.332 77.85 L 698.916 77.85 C 698.4 74.668 697.884 71.744 697.368 69.164 L 688.338 70.712 C 688.94 72.862 689.542 75.27 690.144 77.85 L 675.438 77.85 Z M 724.028 89.632 L 739.25 89.632 L 739.25 93.502 L 723.856 93.502 C 723.942 92.47 724.028 91.352 724.028 90.32 L 724.028 89.632 Z M 739.25 83.096 L 724.028 83.096 L 724.028 79.226 L 739.25 79.226 L 739.25 83.096 Z M 722.652 100.038 L 739.25 100.038 L 739.25 100.898 C 739.25 103.048 738.218 104.166 736.24 104.166 C 733.918 104.166 731.424 103.994 728.758 103.822 L 730.822 111.562 L 738.734 111.562 C 744.582 111.562 747.506 108.982 747.506 103.908 L 747.506 72.002 L 715.6 72.002 L 715.6 90.922 C 715.428 97.286 713.192 102.532 708.892 106.746 L 715.342 112.594 C 718.782 109.068 721.276 104.854 722.652 100.038 Z M 708.462 121.452 L 708.462 126.784 L 683.608 126.784 L 683.608 134.352 L 708.462 134.352 L 708.462 139.598 L 675.524 139.598 L 675.524 147.51 L 750 147.51 L 750 139.598 L 717.062 139.598 L 717.062 134.352 L 742.174 134.352 L 742.174 126.784 L 717.062 126.784 L 717.062 121.452 L 746.216 121.452 L 746.216 113.712 L 679.308 113.712 L 679.308 121.452 L 708.462 121.452 Z" style="fill: #68625f"></path>
-  </g>
-</svg>

assets/logo-full-light.svg

@@ -1,11 +0,0 @@
-<svg viewBox="0 0 720 180" xmlns="http://www.w3.org/2000/svg">
-  <defs></defs>
-  <g transform="matrix(7.5, 0, 0, 7.5, -2046.71228, -1592.744873)">
-    <ellipse style="fill: #f26610;" cx="284.895" cy="224.366" rx="12" ry="12"></ellipse>
-    <path d="M 285.769 221.936 L 285.769 221.168 C 284.999 220.175 284.482 219 284.292 217.745 C 284.234 217.375 283.772 217.231 283.532 217.518 C 282.954 218.199 282.5 218.998 282.194 219.883 C 283.129 220.931 284.381 221.65 285.769 221.936 Z M 288.832 219.115 C 287.624 219.115 286.646 220.097 286.646 221.305 L 286.646 222.929 C 283.79 222.76 281.369 221.002 280.274 218.508 C 280.124 218.166 279.641 218.146 279.48 218.483 C 279.026 219.443 278.771 220.515 278.771 221.646 C 278.771 223.583 279.703 225.391 281.098 226.73 C 281.457 227.077 281.81 227.364 282.161 227.627 L 278.226 228.611 C 277.933 228.684 277.802 229.024 277.967 229.278 C 278.442 230.014 279.618 231.261 282.156 231.365 C 282.375 231.373 282.593 231.294 282.759 231.149 L 284.543 229.615 L 286.646 229.615 C 289.062 229.615 291.02 227.659 291.02 225.242 L 291.02 220.865 L 291.895 219.115 L 288.832 219.115 Z M 288.832 221.757 C 288.592 221.757 288.353 221.541 288.353 221.301 C 288.353 221.061 288.592 220.849 288.832 220.849 C 289.072 220.849 289.294 221.069 289.294 221.309 C 289.294 221.549 289.072 221.757 288.832 221.757 Z" style="fill: #fffaf6;"></path>
-  </g>
-  <g transform="matrix(1, 0, 0, 1, -30, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: #12100f;"></path>
-    <path d="M 589.19 97.802 L 589.19 106.23 L 610.948 106.23 C 605.1 112.938 597.446 119.044 587.986 124.376 L 593.404 131.514 C 597.532 128.934 601.488 126.268 605.186 123.43 L 605.186 146.048 L 614.13 146.048 L 614.13 123.43 L 626.944 123.43 L 626.944 149.402 L 635.974 149.402 L 635.974 123.43 L 649.82 123.43 L 649.82 134.008 C 649.82 136.072 649.046 137.104 647.498 137.104 L 640.36 136.674 L 642.768 145.188 L 650.422 145.188 C 655.926 145.188 658.678 142.092 658.678 135.986 L 658.678 115.174 L 635.974 115.174 L 635.974 108.638 L 626.944 108.638 L 626.944 115.174 L 614.388 115.174 C 617.054 112.336 619.548 109.326 621.784 106.23 L 665.128 106.23 L 665.128 97.802 L 626.858 97.802 C 627.89 95.824 628.836 93.76 629.696 91.61 L 620.838 90.492 C 619.806 92.9 618.516 95.394 617.14 97.802 L 589.19 97.802 Z M 648.1 68.734 C 642.338 72.088 636.232 75.098 629.868 77.678 C 621.612 75.012 612.926 72.518 603.896 70.282 L 599.252 77.248 C 605.272 78.624 611.206 80.258 617.226 82.15 C 610.088 84.386 602.606 86.106 594.78 87.482 L 599.596 95.308 C 612.324 92.04 622.472 89.116 630.04 86.364 C 638.124 89.116 646.122 92.298 654.034 95.824 L 658.936 88.428 C 653.26 86.02 647.412 83.698 641.392 81.548 C 646.208 79.226 651.11 76.56 655.926 73.55 L 648.1 68.734 Z M 675.438 77.85 L 675.438 85.848 L 682.404 85.848 L 682.404 98.92 C 682.404 101.5 681.114 103.22 678.62 104.166 L 680.684 110.874 C 692.036 108.896 701.926 106.66 710.182 104.08 L 708.634 96.426 C 703.474 98.146 697.454 99.608 690.574 100.984 L 690.574 85.848 L 712.332 85.848 L 712.332 77.85 L 698.916 77.85 C 698.4 74.668 697.884 71.744 697.368 69.164 L 688.338 70.712 C 688.94 72.862 689.542 75.27 690.144 77.85 L 675.438 77.85 Z M 724.028 89.632 L 739.25 89.632 L 739.25 93.502 L 723.856 93.502 C 723.942 92.47 724.028 91.352 724.028 90.32 L 724.028 89.632 Z M 739.25 83.096 L 724.028 83.096 L 724.028 79.226 L 739.25 79.226 L 739.25 83.096 Z M 722.652 100.038 L 739.25 100.038 L 739.25 100.898 C 739.25 103.048 738.218 104.166 736.24 104.166 C 733.918 104.166 731.424 103.994 728.758 103.822 L 730.822 111.562 L 738.734 111.562 C 744.582 111.562 747.506 108.982 747.506 103.908 L 747.506 72.002 L 715.6 72.002 L 715.6 90.922 C 715.428 97.286 713.192 102.532 708.892 106.746 L 715.342 112.594 C 718.782 109.068 721.276 104.854 722.652 100.038 Z M 708.462 121.452 L 708.462 126.784 L 683.608 126.784 L 683.608 134.352 L 708.462 134.352 L 708.462 139.598 L 675.524 139.598 L 675.524 147.51 L 750 147.51 L 750 139.598 L 717.062 139.598 L 717.062 134.352 L 742.174 134.352 L 742.174 126.784 L 717.062 126.784 L 717.062 121.452 L 746.216 121.452 L 746.216 113.712 L 679.308 113.712 L 679.308 121.452 L 708.462 121.452 Z" style="fill: #b4aca9;"></path>
-  </g>
-</svg>

assets/logo-icon.svg

@@ -1,5 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg">
-  <ellipse style="fill: #f26610;" cx="12" cy="12" rx="12" ry="12"/>
-  <path d="M 12.874 9.57 L 12.874 8.802 C 12.104 7.809 11.587 6.634 11.397 5.379 C 11.339 5.009 10.877 4.865 10.637 5.152 C 10.059 5.833 9.605 6.632 9.299 7.517 C 10.234 8.565 11.486 9.284 12.874 9.57 Z M 15.937 6.749 C 14.729 6.749 13.751 7.731 13.751 8.939 L 13.751 10.563 C 10.895 10.394 8.474 8.636 7.379 6.142 C 7.229 5.8 6.746 5.78 6.585 6.117 C 6.131 7.077 5.876 8.149 5.876 9.28 C 5.876 11.217 6.808 13.025 8.203 14.364 C 8.562 14.711 8.915 14.998 9.266 15.261 L 5.331 16.245 C 5.038 16.318 4.907 16.658 5.072 16.912 C 5.547 17.648 6.723 18.895 9.261 18.999 C 9.48 19.007 9.698 18.928 9.864 18.783 L 11.648 17.249 L 13.751 17.249 C 16.167 17.249 18.125 15.293 18.125 12.876 L 18.125 8.499 L 19 6.749 L 15.937 6.749 Z M 15.937 9.391 C 15.697 9.391 15.458 9.175 15.458 8.935 C 15.458 8.695 15.697 8.483 15.937 8.483 C 16.177 8.483 16.399 8.703 16.399 8.943 C 16.399 9.183 16.177 9.391 15.937 9.391 Z" style="fill: #fffaf6;"/>
-</svg>

assets/logo-long-text-dark.svg

@@ -1,8 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 500 180" xmlns="http://www.w3.org/2000/svg">
-  <defs/>
-  <g transform="matrix(1, 0, 0, 1, -252.158997, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: #fffaf6;"/>
-    <path d="M 589.19 97.802 L 589.19 106.23 L 610.948 106.23 C 605.1 112.938 597.446 119.044 587.986 124.376 L 593.404 131.514 C 597.532 128.934 601.488 126.268 605.186 123.43 L 605.186 146.048 L 614.13 146.048 L 614.13 123.43 L 626.944 123.43 L 626.944 149.402 L 635.974 149.402 L 635.974 123.43 L 649.82 123.43 L 649.82 134.008 C 649.82 136.072 649.046 137.104 647.498 137.104 L 640.36 136.674 L 642.768 145.188 L 650.422 145.188 C 655.926 145.188 658.678 142.092 658.678 135.986 L 658.678 115.174 L 635.974 115.174 L 635.974 108.638 L 626.944 108.638 L 626.944 115.174 L 614.388 115.174 C 617.054 112.336 619.548 109.326 621.784 106.23 L 665.128 106.23 L 665.128 97.802 L 626.858 97.802 C 627.89 95.824 628.836 93.76 629.696 91.61 L 620.838 90.492 C 619.806 92.9 618.516 95.394 617.14 97.802 L 589.19 97.802 Z M 648.1 68.734 C 642.338 72.088 636.232 75.098 629.868 77.678 C 621.612 75.012 612.926 72.518 603.896 70.282 L 599.252 77.248 C 605.272 78.624 611.206 80.258 617.226 82.15 C 610.088 84.386 602.606 86.106 594.78 87.482 L 599.596 95.308 C 612.324 92.04 622.472 89.116 630.04 86.364 C 638.124 89.116 646.122 92.298 654.034 95.824 L 658.936 88.428 C 653.26 86.02 647.412 83.698 641.392 81.548 C 646.208 79.226 651.11 76.56 655.926 73.55 L 648.1 68.734 Z M 675.438 77.85 L 675.438 85.848 L 682.404 85.848 L 682.404 98.92 C 682.404 101.5 681.114 103.22 678.62 104.166 L 680.684 110.874 C 692.036 108.896 701.926 106.66 710.182 104.08 L 708.634 96.426 C 703.474 98.146 697.454 99.608 690.574 100.984 L 690.574 85.848 L 712.332 85.848 L 712.332 77.85 L 698.916 77.85 C 698.4 74.668 697.884 71.744 697.368 69.164 L 688.338 70.712 C 688.94 72.862 689.542 75.27 690.144 77.85 L 675.438 77.85 Z M 724.028 89.632 L 739.25 89.632 L 739.25 93.502 L 723.856 93.502 C 723.942 92.47 724.028 91.352 724.028 90.32 L 724.028 89.632 Z M 739.25 83.096 L 724.028 83.096 L 724.028 79.226 L 739.25 79.226 L 739.25 83.096 Z M 722.652 100.038 L 739.25 100.038 L 739.25 100.898 C 739.25 103.048 738.218 104.166 736.24 104.166 C 733.918 104.166 731.424 103.994 728.758 103.822 L 730.822 111.562 L 738.734 111.562 C 744.582 111.562 747.506 108.982 747.506 103.908 L 747.506 72.002 L 715.6 72.002 L 715.6 90.922 C 715.428 97.286 713.192 102.532 708.892 106.746 L 715.342 112.594 C 718.782 109.068 721.276 104.854 722.652 100.038 Z M 708.462 121.452 L 708.462 126.784 L 683.608 126.784 L 683.608 134.352 L 708.462 134.352 L 708.462 139.598 L 675.524 139.598 L 675.524 147.51 L 750 147.51 L 750 139.598 L 717.062 139.598 L 717.062 134.352 L 742.174 134.352 L 742.174 126.784 L 717.062 126.784 L 717.062 121.452 L 746.216 121.452 L 746.216 113.712 L 679.308 113.712 L 679.308 121.452 L 708.462 121.452 Z" style="fill: #867e7b;"/>
-  </g>
-</svg>

assets/logo-long-text-light.svg

@@ -1,8 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 500 180" xmlns="http://www.w3.org/2000/svg">
-  <defs/>
-  <g transform="matrix(1, 0, 0, 1, -252.158997, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: rgb(16, 16, 14);"/>
-    <path d="M 589.19 97.802 L 589.19 106.23 L 610.948 106.23 C 605.1 112.938 597.446 119.044 587.986 124.376 L 593.404 131.514 C 597.532 128.934 601.488 126.268 605.186 123.43 L 605.186 146.048 L 614.13 146.048 L 614.13 123.43 L 626.944 123.43 L 626.944 149.402 L 635.974 149.402 L 635.974 123.43 L 649.82 123.43 L 649.82 134.008 C 649.82 136.072 649.046 137.104 647.498 137.104 L 640.36 136.674 L 642.768 145.188 L 650.422 145.188 C 655.926 145.188 658.678 142.092 658.678 135.986 L 658.678 115.174 L 635.974 115.174 L 635.974 108.638 L 626.944 108.638 L 626.944 115.174 L 614.388 115.174 C 617.054 112.336 619.548 109.326 621.784 106.23 L 665.128 106.23 L 665.128 97.802 L 626.858 97.802 C 627.89 95.824 628.836 93.76 629.696 91.61 L 620.838 90.492 C 619.806 92.9 618.516 95.394 617.14 97.802 L 589.19 97.802 Z M 648.1 68.734 C 642.338 72.088 636.232 75.098 629.868 77.678 C 621.612 75.012 612.926 72.518 603.896 70.282 L 599.252 77.248 C 605.272 78.624 611.206 80.258 617.226 82.15 C 610.088 84.386 602.606 86.106 594.78 87.482 L 599.596 95.308 C 612.324 92.04 622.472 89.116 630.04 86.364 C 638.124 89.116 646.122 92.298 654.034 95.824 L 658.936 88.428 C 653.26 86.02 647.412 83.698 641.392 81.548 C 646.208 79.226 651.11 76.56 655.926 73.55 L 648.1 68.734 Z M 675.438 77.85 L 675.438 85.848 L 682.404 85.848 L 682.404 98.92 C 682.404 101.5 681.114 103.22 678.62 104.166 L 680.684 110.874 C 692.036 108.896 701.926 106.66 710.182 104.08 L 708.634 96.426 C 703.474 98.146 697.454 99.608 690.574 100.984 L 690.574 85.848 L 712.332 85.848 L 712.332 77.85 L 698.916 77.85 C 698.4 74.668 697.884 71.744 697.368 69.164 L 688.338 70.712 C 688.94 72.862 689.542 75.27 690.144 77.85 L 675.438 77.85 Z M 724.028 89.632 L 739.25 89.632 L 739.25 93.502 L 723.856 93.502 C 723.942 92.47 724.028 91.352 724.028 90.32 L 724.028 89.632 Z M 739.25 83.096 L 724.028 83.096 L 724.028 79.226 L 739.25 79.226 L 739.25 83.096 Z M 722.652 100.038 L 739.25 100.038 L 739.25 100.898 C 739.25 103.048 738.218 104.166 736.24 104.166 C 733.918 104.166 731.424 103.994 728.758 103.822 L 730.822 111.562 L 738.734 111.562 C 744.582 111.562 747.506 108.982 747.506 103.908 L 747.506 72.002 L 715.6 72.002 L 715.6 90.922 C 715.428 97.286 713.192 102.532 708.892 106.746 L 715.342 112.594 C 718.782 109.068 721.276 104.854 722.652 100.038 Z M 708.462 121.452 L 708.462 126.784 L 683.608 126.784 L 683.608 134.352 L 708.462 134.352 L 708.462 139.598 L 675.524 139.598 L 675.524 147.51 L 750 147.51 L 750 139.598 L 717.062 139.598 L 717.062 134.352 L 742.174 134.352 L 742.174 126.784 L 717.062 126.784 L 717.062 121.452 L 746.216 121.452 L 746.216 113.712 L 679.308 113.712 L 679.308 121.452 L 708.462 121.452 Z" style="fill: rgb(192, 192, 171);"/>
-  </g>
-</svg>

assets/logo-short-text-dark.svg

@@ -1,7 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 310 180" xmlns="http://www.w3.org/2000/svg">
-  <defs/>
-  <g transform="matrix(1, 0, 0, 1, -253.876495, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: rgb(16, 16, 14);"/>
-  </g>
-</svg>

assets/logo-short-text-light.svg

@@ -1,7 +0,0 @@
-<?xml version="1.0" encoding="utf-8"?>
-<svg viewBox="0 0 310 180" xmlns="http://www.w3.org/2000/svg">
-  <defs/>
-  <g transform="matrix(1, 0, 0, 1, -253.876495, 0)">
-    <path d="M 278.049 146.789 L 278.049 127.527 L 287.141 117.972 L 304.4 146.789 L 331.83 146.789 L 303.784 100.251 L 332.755 69.739 L 303.013 69.739 L 278.049 97.477 L 278.049 30.598 L 254.318 30.598 L 254.318 146.789 L 278.049 146.789 Z M 354.169 57.719 C 361.565 57.719 367.575 51.709 367.575 44.158 C 367.575 36.608 361.565 30.752 354.169 30.752 C 346.618 30.752 340.608 36.608 340.608 44.158 C 340.608 51.709 346.618 57.719 354.169 57.719 Z M 342.457 146.789 L 366.188 146.789 L 366.188 69.739 L 342.457 69.739 L 342.457 146.789 Z M 406.407 146.789 L 407.64 136.927 C 411.801 144.015 421.047 148.792 431.834 148.792 C 453.716 148.792 468.972 132.92 468.972 109.035 C 468.972 83.916 455.257 67.119 433.683 67.119 C 422.588 67.119 412.417 71.742 407.794 78.677 L 407.794 30.598 L 384.063 30.598 L 384.063 146.789 L 406.407 146.789 Z M 407.948 107.802 C 407.948 96.244 415.653 88.539 426.749 88.539 C 437.998 88.539 445.087 96.398 445.087 107.802 C 445.087 119.205 437.998 127.064 426.749 127.064 C 415.653 127.064 407.948 119.359 407.948 107.802 Z M 498.713 56.332 L 543.402 56.332 L 543.402 40.306 L 498.713 40.306 L 498.713 56.332 Z M 478.526 108.11 C 478.526 132.458 496.402 148.638 521.058 148.638 C 545.56 148.638 563.435 132.458 563.435 108.11 C 563.435 83.762 545.56 67.428 521.058 67.428 C 496.402 67.428 478.526 83.762 478.526 108.11 Z M 502.412 107.956 C 502.412 96.398 509.963 88.693 521.058 88.693 C 531.999 88.693 539.55 96.398 539.55 107.956 C 539.55 119.667 531.999 127.372 521.058 127.372 C 509.963 127.372 502.412 119.667 502.412 107.956 Z" style="fill: rgb(16, 16, 14);"/>
-  </g>
-</svg>

biter/Cargo.lock

@@ -1,466 +0,0 @@
-# This file is automatically @generated by Cargo.
-# It is not intended for manual editing.
-version = 3
-
-[[package]]
-name = "arrayvec"
-version = "0.7.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "96d30a06541fbafbc7f82ed10c06164cfbd2c401138f6addd8404629c4b16711"
-
-[[package]]
-name = "base58ck"
-version = "0.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2c8d66485a3a2ea485c1913c4572ce0256067a5377ac8c75c4960e1cda98605f"
-dependencies = [
- "bitcoin-internals",
- "bitcoin_hashes",
-]
-
-[[package]]
-name = "base64"
-version = "0.13.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8"
-
-[[package]]
-name = "bech32"
-version = "0.11.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d965446196e3b7decd44aa7ee49e31d630118f90ef12f97900f262eb915c951d"
-
-[[package]]
-name = "bitcoin"
-version = "0.32.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ea507acc1cd80fc084ace38544bbcf7ced7c2aa65b653b102de0ce718df668f6"
-dependencies = [
- "base58ck",
- "bech32",
- "bitcoin-internals",
- "bitcoin-io",
- "bitcoin-units",
- "bitcoin_hashes",
- "hex-conservative",
- "hex_lit",
- "secp256k1",
- "serde",
-]
-
-[[package]]
-name = "bitcoin-internals"
-version = "0.3.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "30bdbe14aa07b06e6cfeffc529a1f099e5fbe249524f8125358604df99a4bed2"
-dependencies = [
- "serde",
-]
-
-[[package]]
-name = "bitcoin-io"
-version = "0.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "340e09e8399c7bd8912f495af6aa58bea0c9214773417ffaa8f6460f93aaee56"
-
-[[package]]
-name = "bitcoin-units"
-version = "0.1.2"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5285c8bcaa25876d07f37e3d30c303f2609179716e11d688f51e8f1fe70063e2"
-dependencies = [
- "bitcoin-internals",
- "serde",
-]
-
-[[package]]
-name = "bitcoin_hashes"
-version = "0.14.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bb18c03d0db0247e147a21a6faafd5a7eb851c743db062de72018b6b7e8e4d16"
-dependencies = [
- "bitcoin-io",
- "hex-conservative",
- "serde",
-]
-
-[[package]]
-name = "bitcoincore-rpc"
-version = "0.19.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "aedd23ae0fd321affb4bbbc36126c6f49a32818dc6b979395d24da8c9d4e80ee"
-dependencies = [
- "bitcoincore-rpc-json",
- "jsonrpc",
- "log",
- "serde",
- "serde_json",
-]
-
-[[package]]
-name = "bitcoincore-rpc-json"
-version = "0.19.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d8909583c5fab98508e80ef73e5592a651c954993dc6b7739963257d19f0e71a"
-dependencies = [
- "bitcoin",
- "serde",
- "serde_json",
-]
-
-[[package]]
-name = "biter"
-version = "0.1.1"
-dependencies = [
- "bitcoin",
- "bitcoincore-rpc",
- "crossbeam",
- "derived-deref",
- "rayon",
- "serde",
- "serde_json",
-]
-
-[[package]]
-name = "byteorder"
-version = "1.5.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b"
-
-[[package]]
-name = "cc"
-version = "1.1.6"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2aba8f4e9906c7ce3c73463f62a7f0c65183ada1a2d47e397cc8810827f9694f"
-
-[[package]]
-name = "cfg-if"
-version = "1.0.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "baf1de4339761588bc0619e3cbc0120ee582ebb74b53b4efbf79117bd2da40fd"
-
-[[package]]
-name = "crossbeam"
-version = "0.8.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1137cd7e7fc0fb5d3c5a8678be38ec56e819125d8d7907411fe24ccb943faca8"
-dependencies = [
- "crossbeam-channel",
- "crossbeam-deque",
- "crossbeam-epoch",
- "crossbeam-queue",
- "crossbeam-utils",
-]
-
-[[package]]
-name = "crossbeam-channel"
-version = "0.5.13"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "33480d6946193aa8033910124896ca395333cae7e2d1113d1fef6c3272217df2"
-dependencies = [
- "crossbeam-utils",
-]
-
-[[package]]
-name = "crossbeam-deque"
-version = "0.8.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "613f8cc01fe9cf1a3eb3d7f488fd2fa8388403e97039e2f73692932e291a770d"
-dependencies = [
- "crossbeam-epoch",
- "crossbeam-utils",
-]
-
-[[package]]
-name = "crossbeam-epoch"
-version = "0.9.18"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5b82ac4a3c2ca9c3460964f020e1402edd5753411d7737aa39c3714ad1b5420e"
-dependencies = [
- "crossbeam-utils",
-]
-
-[[package]]
-name = "crossbeam-queue"
-version = "0.3.11"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "df0346b5d5e76ac2fe4e327c5fd1118d6be7c51dfb18f9b7922923f287471e35"
-dependencies = [
- "crossbeam-utils",
-]
-
-[[package]]
-name = "crossbeam-utils"
-version = "0.8.20"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "22ec99545bb0ed0ea7bb9b8e1e9122ea386ff8a48c0922e43f36d45ab09e0e80"
-
-[[package]]
-name = "derived-deref"
-version = "2.1.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "805ef2023ccd65425743a91ecd11fc020979a0b01921db3104fb606d18a7b43e"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn",
-]
-
-[[package]]
-name = "either"
-version = "1.13.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "60b1af1c220855b6ceac025d3f6ecdd2b7c4894bfe9cd9bda4fbb4bc7c0d4cf0"
-
-[[package]]
-name = "getrandom"
-version = "0.2.15"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "c4567c8db10ae91089c99af84c68c38da3ec2f087c3f82960bcdbf3656b6f4d7"
-dependencies = [
- "cfg-if",
- "libc",
- "wasi",
-]
-
-[[package]]
-name = "hex-conservative"
-version = "0.2.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5313b072ce3c597065a808dbf612c4c8e8590bdbf8b579508bf7a762c5eae6cd"
-dependencies = [
- "arrayvec",
-]
-
-[[package]]
-name = "hex_lit"
-version = "0.1.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3011d1213f159867b13cfd6ac92d2cd5f1345762c63be3554e84092d85a50bbd"
-
-[[package]]
-name = "itoa"
-version = "1.0.11"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "49f1f14873335454500d59611f1cf4a4b0f786f9ac11f4312a78e4cf2566695b"
-
-[[package]]
-name = "jsonrpc"
-version = "0.18.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3662a38d341d77efecb73caf01420cfa5aa63c0253fd7bc05289ef9f6616e1bf"
-dependencies = [
- "base64",
- "minreq",
- "serde",
- "serde_json",
-]
-
-[[package]]
-name = "libc"
-version = "0.2.155"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "97b3888a4aecf77e811145cadf6eef5901f4782c53886191b2f693f24761847c"
-
-[[package]]
-name = "log"
-version = "0.4.22"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a7a70ba024b9dc04c27ea2f0c0548feb474ec5c54bba33a7f72f873a39d07b24"
-
-[[package]]
-name = "memchr"
-version = "2.7.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "78ca9ab1a0babb1e7d5695e3530886289c18cf2f87ec19a575a0abdce112e3a3"
-
-[[package]]
-name = "minreq"
-version = "2.12.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "763d142cdff44aaadd9268bebddb156ef6c65a0e13486bb81673cf2d8739f9b0"
-dependencies = [
- "log",
- "serde",
- "serde_json",
-]
-
-[[package]]
-name = "ppv-lite86"
-version = "0.2.18"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dee4364d9f3b902ef14fab8a1ddffb783a1cb6b4bba3bfc1fa3922732c7de97f"
-dependencies = [
- "zerocopy",
-]
-
-[[package]]
-name = "proc-macro2"
-version = "1.0.86"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "5e719e8df665df0d1c8fbfd238015744736151d4445ec0836b8e628aae103b77"
-dependencies = [
- "unicode-ident",
-]
-
-[[package]]
-name = "quote"
-version = "1.0.36"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "0fa76aaf39101c457836aec0ce2316dbdc3ab723cdda1c6bd4e6ad4208acaca7"
-dependencies = [
- "proc-macro2",
-]
-
-[[package]]
-name = "rand"
-version = "0.8.5"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
-dependencies = [
- "libc",
- "rand_chacha",
- "rand_core",
-]
-
-[[package]]
-name = "rand_chacha"
-version = "0.3.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
-dependencies = [
- "ppv-lite86",
- "rand_core",
-]
-
-[[package]]
-name = "rand_core"
-version = "0.6.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
-dependencies = [
- "getrandom",
-]
-
-[[package]]
-name = "rayon"
-version = "1.10.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "b418a60154510ca1a002a752ca9714984e21e4241e804d32555251faf8b78ffa"
-dependencies = [
- "either",
- "rayon-core",
-]
-
-[[package]]
-name = "rayon-core"
-version = "1.12.1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1465873a3dfdaa8ae7cb14b4383657caab0b3e8a0aa9ae8e04b044854c8dfce2"
-dependencies = [
- "crossbeam-deque",
- "crossbeam-utils",
-]
-
-[[package]]
-name = "ryu"
-version = "1.0.18"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "f3cb5ba0dc43242ce17de99c180e96db90b235b8a9fdc9543c96d2209116bd9f"
-
-[[package]]
-name = "secp256k1"
-version = "0.29.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "0e0cc0f1cf93f4969faf3ea1c7d8a9faed25918d96affa959720823dfe86d4f3"
-dependencies = [
- "bitcoin_hashes",
- "rand",
- "secp256k1-sys",
- "serde",
-]
-
-[[package]]
-name = "secp256k1-sys"
-version = "0.10.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "1433bd67156263443f14d603720b082dd3121779323fce20cba2aa07b874bc1b"
-dependencies = [
- "cc",
-]
-
-[[package]]
-name = "serde"
-version = "1.0.204"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "bc76f558e0cbb2a839d37354c575f1dc3fdc6546b5be373ba43d95f231bf7c12"
-dependencies = [
- "serde_derive",
-]
-
-[[package]]
-name = "serde_derive"
-version = "1.0.204"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "e0cd7e117be63d3c3678776753929474f3b04a43a080c744d6b0ae2a8c28e222"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn",
-]
-
-[[package]]
-name = "serde_json"
-version = "1.0.122"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "784b6203951c57ff748476b126ccb5e8e2959a5c19e5c617ab1956be3dbc68da"
-dependencies = [
- "itoa",
- "memchr",
- "ryu",
- "serde",
-]
-
-[[package]]
-name = "syn"
-version = "2.0.72"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "dc4b9b9bf2add8093d3f2c0204471e951b2285580335de42f9d2534f3ae7a8af"
-dependencies = [
- "proc-macro2",
- "quote",
- "unicode-ident",
-]
-
-[[package]]
-name = "unicode-ident"
-version = "1.0.12"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "3354b9ac3fae1ff6755cb6db53683adb661634f67557942dea4facebec0fee4b"
-
-[[package]]
-name = "wasi"
-version = "0.11.0+wasi-snapshot-preview1"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9c8d87e72b64a3b4db28d11ce29237c246188f4f51057d65a7eab63b7987e423"
-
-[[package]]
-name = "zerocopy"
-version = "0.6.6"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "854e949ac82d619ee9a14c66a1b674ac730422372ccb759ce0c39cabcf2bf8e6"
-dependencies = [
- "byteorder",
- "zerocopy-derive",
-]
-
-[[package]]
-name = "zerocopy-derive"
-version = "0.6.6"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "125139de3f6b9d625c39e2efdd73d41bdac468ccd556556440e322be0e1bbd91"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn",
-]

biter/Cargo.toml

@@ -1,19 +0,0 @@
-[package]
-name = "biter"
-description = "A very fast Bitcoin block iterator"
-version = "0.1.1"
-license = "MIT"
-repository = "https://github.com/kibo-money/kibo/tree/main/biter"
-keywords = ["bitcoin", "block", "iterator"]
-categories = ["cryptography::cryptocurrencies", "encoding"]
-edition = "2021"
-
-[dependencies]
-bitcoin = { version = "0.32.2", features = ["serde"] }
-rayon = "1.10.0"
-crossbeam = { version = "0.8.4", features = ["crossbeam-channel"] }
-serde = { version = "1.0.204", features = ["derive"] }
-serde_json = "1.0.122"
-derived-deref = "2.1.0"
-bitcoincore-rpc = "0.19.0"
-# tokio = { version = "1.39.2", features = ["rt-multi-thread"] }

biter/README.md

@@ -1,62 +0,0 @@
-# Biter
-
-Biter (Bitcoin Block Iterator) is a very fast and simple Rust library which reads raw block files (*blkXXXXX.dat*) from Bitcoin Core Node and creates an iterator over all the requested blocks in sequential order (0, 1, 2, ...).
-
-The element returned by the iterator is a tuple which includes the:
-- Height: `usize`
-- Block: `Block` (from `bitcoin-rust`)
-- Block's Hash: `BlockHash` (also from `bitcoin-rust`)
-
-## Example
-
-```rust
-use bitcoincore_rpc::{Auth, Client};
-
-fn main() {
-    let i = std::time::Instant::now();
-
-    // Path to the Bitcoin data directory
-    let data_dir = "../../bitcoin";
-
-    // Path to the export directory where a mini blk indexer will be exported
-    let export_dir = "./target";
-
-    // Inclusive starting height of the blocks received, `None` for 0
-    let start = Some(850_000);
-
-    // Inclusive ending height of the blocks received, `None` for the last one
-    let end = None;
-
-    // RPC client to filter out forks
-    let url = "http://localhost:8332";
-    let auth = Auth::UserPass("satoshi".to_string(), "nakamoto".to_string());
-    let rpc = Client::new(url, auth).unwrap();
-
-    // Create channel receiver then iterate over the blocks
-    biter::new(data_dir, export_dir, start, end, rpc)
-        .iter()
-        .for_each(|(height, _block, hash)| {
-            println!("{height}: {hash}");
-        });
-
-    dbg!(i.elapsed());
-}
-
-```
-
-## Requirements
-
-Even though it reads *blkXXXXX.dat* files, it **needs** `bitcoind` to run with the RPC server to filter out block forks.
-
-Peak memory should be around 500MB.
-
-## Comparaison
-
-|  | [biter](https://crates.io/crates/biter) | [bitcoin-explorer](https://crates.io/crates/bitcoin-explorer) | [blocks_iterator](https://crates.io/crates/blocks_iterator) |
-| --- | --- | --- | --- |
-| Run **with** `bitcoind` | Yes ✅ | No ❌ | Yes ✅ |
-| Run **without** `bitcoind` | No ❌ | Yes ✅ | Yes ✅ |
-| `0..=855_000` | 16mn40s | 17mn 46s | > 2h |
-| `800_000..=855_000` | 2mn 53s (16mn40s if first run) | 3mn 2s | > 2h |
-
-*Benchmarked on a Macbook Pro M3 Pro*

biter/src/blk_index_to_blk_recap.rs

@@ -1,105 +0,0 @@
-use std::{
-    cmp::Ordering,
-    collections::BTreeMap,
-    fs::{self, File},
-    io::{BufReader, BufWriter},
-    path::PathBuf,
-};
-
-use derived_deref::{Deref, DerefMut};
-
-use crate::{blk_recap::BlkRecap, BlkMetadataAndBlock};
-
-#[derive(Deref, DerefMut, Debug)]
-pub struct BlkIndexToBlkRecap {
-    path: String,
-    #[target]
-    tree: BTreeMap<usize, BlkRecap>,
-}
-
-impl BlkIndexToBlkRecap {
-    pub fn import(blocks_dir: &BTreeMap<usize, PathBuf>, export_dir: &str) -> Self {
-        let path = format!("{export_dir}/blk_index_to_blk_recap.json");
-
-        let tree = {
-            fs::create_dir_all(export_dir).unwrap();
-
-            if let Ok(file) = File::open(&path) {
-                let reader = BufReader::new(file);
-                serde_json::from_reader(reader).unwrap_or_default()
-            } else {
-                BTreeMap::default()
-            }
-        };
-
-        let mut this = Self { path, tree };
-
-        this.clean_outdated(blocks_dir);
-
-        this
-    }
-
-    pub fn clean_outdated(&mut self, blocks_dir: &BTreeMap<usize, PathBuf>) {
-        blocks_dir.iter().for_each(|(blk_index, blk_path)| {
-            if let Some(blk_recap) = self.get(blk_index) {
-                if blk_recap.has_different_modified_time(blk_path) {
-                    self.remove(blk_index);
-                }
-            }
-        });
-    }
-
-    pub fn get_start_recap(&self, start: Option<usize>) -> Option<(usize, BlkRecap)> {
-        if let Some(start) = start {
-            let (last_key, last_value) = self.last_key_value()?;
-
-            if last_value.height() < start {
-                return Some((*last_key, *last_value));
-            } else if let Some((blk_index, _)) = self
-                .iter()
-                .find(|(_, blk_recap)| blk_recap.is_younger_than(start))
-            {
-                if *blk_index != 0 {
-                    let blk_index = *blk_index - 1;
-                    return Some((blk_index, *self.get(&blk_index).unwrap()));
-                }
-            }
-        }
-
-        None
-    }
-
-    pub fn update(&mut self, blk_metadata_and_block: &BlkMetadataAndBlock, height: usize) {
-        let blk_index = blk_metadata_and_block.blk_metadata.index;
-
-        if let Some(last_entry) = self.last_entry() {
-            // if last_entry.get().is_older_than(height) {
-            match last_entry.key().cmp(&blk_index) {
-                Ordering::Greater => {
-                    last_entry.remove_entry();
-                }
-                Ordering::Less => {
-                    self.insert(blk_index, BlkRecap::from(height, blk_metadata_and_block));
-                }
-                Ordering::Equal => {}
-            };
-            // }
-        } else {
-            if blk_index != 0 || height != 0 {
-                // dbg!(blk_index, height);
-                unreachable!();
-            }
-
-            self.insert(blk_index, BlkRecap::first(blk_metadata_and_block));
-        }
-    }
-
-    pub fn export(&self) {
-        let file = File::create(&self.path).unwrap_or_else(|_| {
-            dbg!(&self.path);
-            panic!("No such file or directory")
-        });
-
-        serde_json::to_writer_pretty(&mut BufWriter::new(file), &self.tree).unwrap();
-    }
-}

biter/src/blk_metadata.rs

@@ -1,18 +0,0 @@
-use std::path::PathBuf;
-
-use crate::path_to_modified_time;
-
-#[derive(Clone, Copy)]
-pub struct BlkMetadata {
-    pub index: usize,
-    pub modified_time: u64,
-}
-
-impl BlkMetadata {
-    pub fn new(index: usize, path: &PathBuf) -> Self {
-        Self {
-            index,
-            modified_time: path_to_modified_time(path),
-        }
-    }
-}

biter/src/blk_metadata_and_block.rs

@@ -1,17 +0,0 @@
-use bitcoin::Block;
-
-use crate::BlkMetadata;
-
-pub struct BlkMetadataAndBlock {
-    pub blk_metadata: BlkMetadata,
-    pub block: Block,
-}
-
-impl BlkMetadataAndBlock {
-    pub fn new(blk_metadata: BlkMetadata, block: Block) -> Self {
-        Self {
-            blk_metadata,
-            block,
-        }
-    }
-}

biter/src/blk_recap.rs

@@ -1,47 +0,0 @@
-use std::path::PathBuf;
-
-use bitcoin::{hashes::Hash, BlockHash};
-use serde::{Deserialize, Serialize};
-
-use crate::{path_to_modified_time, BlkMetadataAndBlock};
-
-#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
-pub struct BlkRecap {
-    min_continuous_height: usize,
-    min_continuous_prev_hash: BlockHash,
-    modified_time: u64,
-}
-
-impl BlkRecap {
-    pub fn first(blk_metadata_and_block: &BlkMetadataAndBlock) -> Self {
-        Self {
-            min_continuous_height: 0,
-            min_continuous_prev_hash: BlockHash::all_zeros(),
-            modified_time: blk_metadata_and_block.blk_metadata.modified_time,
-        }
-    }
-
-    pub fn from(height: usize, blk_metadata_and_block: &BlkMetadataAndBlock) -> Self {
-        Self {
-            min_continuous_height: height,
-            min_continuous_prev_hash: blk_metadata_and_block.block.header.prev_blockhash,
-            modified_time: blk_metadata_and_block.blk_metadata.modified_time,
-        }
-    }
-
-    pub fn has_different_modified_time(&self, blk_path: &PathBuf) -> bool {
-        self.modified_time != path_to_modified_time(blk_path)
-    }
-
-    pub fn is_younger_than(&self, height: usize) -> bool {
-        self.min_continuous_height > height
-    }
-
-    pub fn height(&self) -> usize {
-        self.min_continuous_height
-    }
-
-    pub fn prev_hash(&self) -> &BlockHash {
-        &self.min_continuous_prev_hash
-    }
-}

biter/src/lib.rs

@@ -1,384 +0,0 @@
-use std::{
-    collections::{BTreeMap, BTreeSet, VecDeque},
-    fs::{self},
-    ops::ControlFlow,
-    thread,
-};
-
-use bitcoin::{
-    consensus::{Decodable, ReadExt},
-    hashes::Hash,
-    io::{Cursor, Read},
-    Block, BlockHash,
-};
-use bitcoincore_rpc::RpcApi;
-use crossbeam::channel::{bounded, Receiver};
-use rayon::prelude::*;
-
-pub use bitcoin;
-pub use bitcoincore_rpc;
-
-mod blk_index_to_blk_recap;
-mod blk_metadata;
-mod blk_metadata_and_block;
-mod blk_recap;
-mod utils;
-
-use blk_index_to_blk_recap::*;
-use blk_metadata::*;
-use blk_metadata_and_block::*;
-use utils::*;
-
-pub const NUMBER_OF_UNSAFE_BLOCKS: usize = 100;
-const MAGIC_BYTES: [u8; 4] = [249, 190, 180, 217];
-const BOUND_CAP: usize = 210;
-
-enum BlockState {
-    Raw(Vec<u8>),
-    Decoded(Block),
-}
-
-///
-/// Returns a crossbeam channel receiver that receives `(usize, Block, BlockHash)` tuples (with `usize` being the height) in sequential order.
-///
-/// # Arguments
-///
-/// * `data_dir` - Path to the Bitcoin data directory
-/// * `export_dir` - Path to the export directory where a mini blk indexer will be exported
-/// * `start` - Inclusive starting height of the blocks received, `None` for 0
-/// * `end` - Inclusive ending height of the blocks received, `None` for the last one
-/// * `rpc` - RPC client to filter out forks
-///
-/// # Example
-///
-/// ```rust
-/// use bitcoincore_rpc::{Auth, Client};
-///
-/// fn main() {
-///     let i = std::time::Instant::now();
-///
-///     let url = "http://localhost:8332";
-///     let auth = Auth::UserPass("satoshi".to_string(), "nakamoto".to_string());
-///     let rpc = Client::new(url, auth).unwrap();
-///
-///     let data_dir = "../../bitcoin";
-///     let export_dir = "./target";
-///     let start = Some(850_000);
-///     let end = None;
-///
-///     biter::new(data_dir, export_dir, start, end, rpc)
-///         .iter()
-///         .for_each(|(height, _block, hash)| {
-///             println!("{height}: {hash}");
-///         });
-///
-///     dbg!(i.elapsed());
-///}
-/// ```
-///
-pub fn new(
-    data_dir: &str,
-    export_dir: &str,
-    start: Option<usize>,
-    end: Option<usize>,
-    rpc: bitcoincore_rpc::Client,
-) -> Receiver<(usize, Block, BlockHash)> {
-    let (send_block_reader, recv_block_reader) = bounded(BOUND_CAP);
-    let (send_block, recv_block) = bounded(BOUND_CAP);
-    let (send_height_block_hash, recv_height_block_hash) = bounded(BOUND_CAP);
-
-    let blocks_dir = scan_blocks_dir(data_dir);
-
-    let mut blk_index_to_blk_recap = BlkIndexToBlkRecap::import(&blocks_dir, export_dir);
-
-    let start_recap = blk_index_to_blk_recap.get_start_recap(start);
-    let starting_blk_index = start_recap.as_ref().map_or(0, |(index, _)| *index);
-
-    thread::spawn(move || {
-        blocks_dir
-            .into_iter()
-            .filter(|(blk_index, _)| blk_index >= &starting_blk_index)
-            .try_for_each(move |(blk_index, blk_path)| {
-                let blk_metadata = BlkMetadata::new(blk_index, &blk_path);
-
-                let blk_bytes = fs::read(&blk_path).unwrap();
-                let blk_bytes_len = blk_bytes.len() as u64;
-
-                let mut cursor = Cursor::new(blk_bytes.as_slice());
-
-                let mut current_4bytes = [0; 4];
-
-                'parent: loop {
-                    if cursor.position() == blk_bytes_len {
-                        break;
-                    }
-
-                    // Read until we find a valid suite of MAGIC_BYTES
-                    loop {
-                        current_4bytes.rotate_left(1);
-
-                        if let Ok(byte) = cursor.read_u8() {
-                            current_4bytes[3] = byte;
-                        } else {
-                            break 'parent;
-                        }
-
-                        if current_4bytes == MAGIC_BYTES {
-                            break;
-                        }
-                    }
-
-                    let block_size = cursor.read_u32().unwrap();
-
-                    let mut raw_block = vec![0u8; block_size as usize];
-
-                    cursor.read_exact(&mut raw_block).unwrap();
-
-                    if send_block_reader
-                        .send((blk_metadata, BlockState::Raw(raw_block)))
-                        .is_err()
-                    {
-                        return ControlFlow::Break(());
-                    }
-                }
-
-                ControlFlow::Continue(())
-            })
-    });
-
-    // thread::spawn(move || {
-    //     recv_block_reader.iter().par_bridge().try_for_each(
-    //         move |(blk_metadata, mut block_state)| {
-    //             let raw_block = match block_state {
-    //                 BlockState::Raw(vec) => vec,
-    //                 _ => unreachable!(),
-    //             };
-
-    //             let mut cursor = Cursor::new(raw_block);
-
-    //             block_state = BlockState::Decoded(Block::consensus_decode(&mut cursor).unwrap());
-
-    //             if send_block
-    //                 .send(BlkMetadataAndBlock::new(
-    //                     blk_metadata,
-    //                     match block_state {
-    //                         BlockState::Decoded(block) => block,
-    //                         _ => unreachable!(),
-    //                     },
-    //                 ))
-    //                 .is_err()
-    //             {
-    //                 return ControlFlow::Break(());
-    //             }
-
-    //             ControlFlow::Continue(())
-    //         },
-    //     );
-    // });
-
-    // Can't use the previous code because .send() blocks all the threads if full
-    // And other .par_iter() are also stuck because of that
-    thread::spawn(move || {
-        let mut bulk = vec![];
-
-        let drain_and_send = |bulk: &mut Vec<_>| {
-            // Using a vec and sending after to not end up with stuck threads in par iter
-            bulk.par_iter_mut().for_each(|(_, block_state)| {
-                let raw_block = match block_state {
-                    BlockState::Raw(vec) => vec,
-                    _ => unreachable!(),
-                };
-
-                let mut cursor = Cursor::new(raw_block);
-
-                *block_state = BlockState::Decoded(Block::consensus_decode(&mut cursor).unwrap());
-            });
-
-            bulk.drain(..).try_for_each(|(blk_metadata, block_state)| {
-                let block = match block_state {
-                    BlockState::Decoded(block) => block,
-                    _ => unreachable!(),
-                };
-
-                if send_block
-                    .send(BlkMetadataAndBlock::new(blk_metadata, block))
-                    .is_err()
-                {
-                    return ControlFlow::Break(());
-                }
-
-                ControlFlow::Continue(())
-            })
-        };
-
-        recv_block_reader.iter().try_for_each(|tuple| {
-            bulk.push(tuple);
-
-            if bulk.len() < BOUND_CAP / 2 {
-                return ControlFlow::Continue(());
-            }
-
-            drain_and_send(&mut bulk)
-        });
-
-        drain_and_send(&mut bulk)
-    });
-
-    // Tokio version: 1022s
-    // Slighlty slower than rayon version
-    // thread::spawn(move || {
-    //     let rt = tokio::runtime::Runtime::new().unwrap();
-    //     let _guard = rt.enter();
-
-    //     let mut tasks = VecDeque::with_capacity(BOUND);
-
-    //     recv_block_reader
-    //         .iter()
-    //         .try_for_each(move |(blk_metadata, block_state)| {
-    //             let raw_block = match block_state {
-    //                 BlockState::Raw(vec) => vec,
-    //                 _ => unreachable!(),
-    //             };
-
-    //             tasks.push_back(tokio::task::spawn(async move {
-    //                 let block = Block::consensus_decode(&mut Cursor::new(raw_block)).unwrap();
-
-    //                 (blk_metadata, block)
-    //             }));
-
-    //             while tasks.len() > BOUND {
-    //                 let (blk_metadata, block) = rt.block_on(tasks.pop_front().unwrap()).unwrap();
-
-    //                 if send_block
-    //                     .send(BlkMetadataAndBlock::new(blk_metadata, block))
-    //                     .is_err()
-    //                 {
-    //                     return ControlFlow::Break(());
-    //                 }
-    //             }
-
-    //             ControlFlow::Continue(())
-    //         });
-    //
-    // todo!("Send the rest")
-    // });
-
-    thread::spawn(move || {
-        let mut height = start_recap.map_or(0, |(_, recap)| recap.height());
-
-        let mut future_blocks = BTreeMap::default();
-        let mut recent_chain: VecDeque<(BlockHash, BlkMetadataAndBlock)> = VecDeque::default();
-        let mut recent_hashes: BTreeSet<BlockHash> = BTreeSet::default();
-
-        let mut prev_hash =
-            start_recap.map_or_else(BlockHash::all_zeros, |(_, recap)| *recap.prev_hash());
-
-        let mut prepare_and_send = |(hash, tuple): (BlockHash, BlkMetadataAndBlock)| {
-            blk_index_to_blk_recap.update(&tuple, height);
-
-            if start.map_or(true, |start| start <= height) {
-                send_height_block_hash
-                    .send((height, tuple.block, hash))
-                    .unwrap();
-            }
-
-            if end.map_or(false, |end| height == end) {
-                return ControlFlow::Break(());
-            }
-
-            height += 1;
-
-            ControlFlow::Continue(())
-        };
-
-        let mut update_tip = |prev_hash: &mut BlockHash,
-                              recent_hashes: &mut BTreeSet<BlockHash>,
-                              recent_chain: &mut VecDeque<(BlockHash, BlkMetadataAndBlock)>,
-                              future_blocks: &mut BTreeMap<BlockHash, BlkMetadataAndBlock>,
-                              tuple: BlkMetadataAndBlock| {
-            let mut tuple = Some(tuple);
-
-            while let Some(tuple) = tuple.take().or_else(|| future_blocks.remove(prev_hash)) {
-                let hash = tuple.block.block_hash();
-
-                *prev_hash = hash;
-                recent_hashes.insert(hash);
-                recent_chain.push_back((hash, tuple));
-            }
-
-            while recent_chain.len() > NUMBER_OF_UNSAFE_BLOCKS {
-                let (hash, tuple) = recent_chain.pop_front().unwrap();
-
-                recent_hashes.remove(&hash);
-
-                if prepare_and_send((hash, tuple)).is_break() {
-                    return ControlFlow::Break(());
-                }
-            }
-
-            ControlFlow::Continue(())
-        };
-
-        let flow = recv_block.iter().try_for_each(|tuple| {
-            // block isn't next after current tip
-            if prev_hash != tuple.block.header.prev_blockhash {
-                let is_block_active =
-                    |hash| rpc.get_block_header_info(hash).unwrap().confirmations > 0;
-
-                // block prev has already been processed
-                if recent_hashes.contains(&tuple.block.header.prev_blockhash) {
-                    let hash = tuple.block.block_hash();
-
-                    if is_block_active(&hash) {
-                        let prev_index = recent_chain
-                            .iter()
-                            .position(|(hash, ..)| hash == &tuple.block.header.prev_blockhash)
-                            .unwrap();
-
-                        let bad_index_start = prev_index + 1;
-
-                        recent_chain.drain(bad_index_start..).for_each(|(hash, _)| {
-                            recent_hashes.remove(&hash);
-                        });
-
-                        return update_tip(
-                            &mut prev_hash,
-                            &mut recent_hashes,
-                            &mut recent_chain,
-                            &mut future_blocks,
-                            tuple,
-                        );
-                    }
-                // Check if there was already a future block with the same prev hash
-                } else if let Some(prev_tuple) =
-                    future_blocks.insert(tuple.block.header.prev_blockhash, tuple)
-                {
-                    // If the previous was the active one
-                    if is_block_active(&prev_tuple.block.block_hash()) {
-                        // Rollback the insert
-                        future_blocks.insert(prev_tuple.block.header.prev_blockhash, prev_tuple);
-                    }
-                }
-            } else {
-                return update_tip(
-                    &mut prev_hash,
-                    &mut recent_hashes,
-                    &mut recent_chain,
-                    &mut future_blocks,
-                    tuple,
-                );
-            }
-
-            ControlFlow::Continue(())
-        });
-
-        if flow.is_continue() {
-            // Send the last (up to 100) blocks
-            recent_chain.into_iter().try_for_each(prepare_and_send);
-        }
-
-        blk_index_to_blk_recap.export();
-    });
-
-    recv_height_block_hash
-}

biter/src/main.rs

@@ -1,22 +0,0 @@
-use bitcoincore_rpc::{Auth, Client};
-
-fn main() {
-    let i = std::time::Instant::now();
-
-    let url = "http://localhost:8332";
-    let auth = Auth::UserPass("satoshi".to_string(), "nakamoto".to_string());
-    let rpc = Client::new(url, auth).unwrap();
-
-    let data_dir = "../bitcoin";
-    let export_dir = "./target";
-    let start = None;
-    let end = None;
-
-    biter::new(data_dir, export_dir, start, end, rpc)
-        .iter()
-        .for_each(|(height, _block, hash)| {
-            println!("{height}: {hash}");
-        });
-
-    dbg!(i.elapsed());
-}

biter/src/utils.rs

@@ -1,43 +0,0 @@
-use std::{collections::BTreeMap, fs, path::PathBuf, time::UNIX_EPOCH};
-
-const BLK: &str = "blk";
-const DAT: &str = ".dat";
-
-pub fn scan_blocks_dir(data_dir_path: &str) -> BTreeMap<usize, PathBuf> {
-    let blocks_dir_path = &format!("{data_dir_path}/blocks");
-
-    fs::read_dir(blocks_dir_path)
-        .unwrap()
-        .map(|entry| entry.unwrap().path())
-        .filter(|path| {
-            let is_file = path.is_file();
-
-            if is_file {
-                let file_name = path.file_name().unwrap().to_str().unwrap();
-
-                file_name.starts_with(BLK) && file_name.ends_with(DAT)
-            } else {
-                false
-            }
-        })
-        .map(|path| {
-            let file_name = path.file_name().unwrap().to_str().unwrap();
-
-            let blk_index = file_name[BLK.len()..(file_name.len() - DAT.len())]
-                .parse::<usize>()
-                .unwrap();
-
-            (blk_index, path)
-        })
-        .collect::<BTreeMap<_, _>>()
-}
-
-pub fn path_to_modified_time(path: &PathBuf) -> u64 {
-    fs::metadata(path)
-        .unwrap()
-        .modified()
-        .unwrap()
-        .duration_since(UNIX_EPOCH)
-        .unwrap()
-        .as_secs()
-}

crates/brk/Cargo.toml

@@ -0,0 +1,75 @@
+[package]
+name = "brk"
+description.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+edition.workspace = true
+version.workspace = true
+build = "build.rs"
+
+[features]
+full = [
+    "bencher",
+    "binder",
+    "bundler",
+    "computer",
+    "error",
+    "fetcher",
+    "grouper",
+    "indexer",
+    "iterator",
+    "logger",
+    "mcp",
+    "mempool",
+    "query",
+    "reader",
+    "rpc",
+    "server",
+    "store",
+    "traversable",
+    "types",
+]
+bencher = ["brk_bencher"]
+binder = ["brk_binder"]
+bundler = ["brk_bundler"]
+computer = ["brk_computer"]
+error = ["brk_error"]
+fetcher = ["brk_fetcher"]
+grouper = ["brk_grouper"]
+indexer = ["brk_indexer"]
+iterator = ["brk_iterator"]
+logger = ["brk_logger"]
+mcp = ["brk_mcp"]
+mempool = ["brk_mempool"]
+query = ["brk_query"]
+reader = ["brk_reader"]
+rpc = ["brk_rpc"]
+server = ["brk_server"]
+store = ["brk_store"]
+traversable = ["brk_traversable"]
+types = ["brk_types"]
+
+[dependencies]
+brk_bencher = { workspace = true, optional = true }
+brk_binder = { workspace = true, optional = true }
+brk_bundler = { workspace = true, optional = true }
+brk_computer = { workspace = true, optional = true }
+brk_error = { workspace = true, optional = true }
+brk_fetcher = { workspace = true, optional = true }
+brk_grouper = { workspace = true, optional = true }
+brk_indexer = { workspace = true, optional = true }
+brk_iterator = { workspace = true, optional = true }
+brk_logger = { workspace = true, optional = true }
+brk_mcp = { workspace = true, optional = true }
+brk_mempool = { workspace = true, optional = true }
+brk_query = { workspace = true, optional = true }
+brk_reader = { workspace = true, optional = true }
+brk_rpc = { workspace = true, optional = true }
+brk_server = { workspace = true, optional = true }
+brk_store = { workspace = true, optional = true }
+brk_traversable = { workspace = true, optional = true }
+brk_types = { workspace = true, optional = true }
+
+[package.metadata.docs.rs]
+all-features = true

crates/brk/README.md

@@ -0,0 +1,43 @@
+# brk
+
+Umbrella crate for the Bitcoin Research Kit.
+
+## What It Enables
+
+Single dependency to access any BRK component. Enable only what you need via feature flags.
+
+## Usage
+
+```toml
+[dependencies]
+brk = { version = "0.x", features = ["query", "types"] }
+```
+
+```rust,ignore
+use brk::query::Query;
+use brk::types::Height;
+```
+
+## Feature Flags
+
+| Feature | Crate | Description |
+|---------|-------|-------------|
+| `bencher` | `brk_bencher` | Resource monitoring |
+| `binder` | `brk_binder` | Client code generation |
+| `bundler` | `brk_bundler` | JS bundling |
+| `computer` | `brk_computer` | Metric computation |
+| `error` | `brk_error` | Error types |
+| `fetcher` | `brk_fetcher` | Price data fetching |
+| `grouper` | `brk_grouper` | Cohort filtering |
+| `indexer` | `brk_indexer` | Blockchain indexing |
+| `iterator` | `brk_iterator` | Block iteration |
+| `logger` | `brk_logger` | Logging setup |
+| `mcp` | `brk_mcp` | MCP server |
+| `mempool` | `brk_mempool` | Mempool monitoring |
+| `query` | `brk_query` | Query interface |
+| `reader` | `brk_reader` | Raw block reading |
+| `rpc` | `brk_rpc` | Bitcoin RPC client |
+| `server` | `brk_server` | HTTP API server |
+| `store` | `brk_store` | Key-value storage |
+| `traversable` | `brk_traversable` | Data traversal |
+| `types` | `brk_types` | Domain types |

crates/brk/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk/flamegraph.sh

@@ -0,0 +1 @@
+sudo cargo flamegraph --profile profiling --root

crates/brk/samply.sh

@@ -0,0 +1,2 @@
+cargo build --profile profiling
+samply record ../../target/profiling/brk

crates/brk/src/lib.rs

@@ -0,0 +1,77 @@
+#![doc = include_str!("../README.md")]
+
+#[cfg(feature = "bencher")]
+#[doc(inline)]
+pub use brk_bencher as bencher;
+
+#[cfg(feature = "binder")]
+#[doc(inline)]
+pub use brk_binder as binder;
+
+#[cfg(feature = "bundler")]
+#[doc(inline)]
+pub use brk_bundler as bundler;
+
+#[cfg(feature = "computer")]
+#[doc(inline)]
+pub use brk_computer as computer;
+
+#[cfg(feature = "error")]
+#[doc(inline)]
+pub use brk_error as error;
+
+#[cfg(feature = "fetcher")]
+#[doc(inline)]
+pub use brk_fetcher as fetcher;
+
+#[cfg(feature = "grouper")]
+#[doc(inline)]
+pub use brk_grouper as grouper;
+
+#[cfg(feature = "indexer")]
+#[doc(inline)]
+pub use brk_indexer as indexer;
+
+#[cfg(feature = "iterator")]
+#[doc(inline)]
+pub use brk_iterator as iterator;
+
+#[cfg(feature = "logger")]
+#[doc(inline)]
+pub use brk_logger as logger;
+
+#[cfg(feature = "mcp")]
+#[doc(inline)]
+pub use brk_mcp as mcp;
+
+#[cfg(feature = "mempool")]
+#[doc(inline)]
+pub use brk_mempool as mempool;
+
+#[cfg(feature = "query")]
+#[doc(inline)]
+pub use brk_query as query;
+
+#[cfg(feature = "reader")]
+#[doc(inline)]
+pub use brk_reader as reader;
+
+#[cfg(feature = "rpc")]
+#[doc(inline)]
+pub use brk_rpc as rpc;
+
+#[cfg(feature = "server")]
+#[doc(inline)]
+pub use brk_server as server;
+
+#[cfg(feature = "store")]
+#[doc(inline)]
+pub use brk_store as store;
+
+#[cfg(feature = "traversable")]
+#[doc(inline)]
+pub use brk_traversable as traversable;
+
+#[cfg(feature = "types")]
+#[doc(inline)]
+pub use brk_types as types;

crates/brk_bencher/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "brk_bencher"
+description = "A simple benchmarker for testing other crates."
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+brk_error = { workspace = true }
+brk_logger = { workspace = true }
+parking_lot = { workspace = true }
+
+[target.'cfg(target_os = "macos")'.dependencies]
+libproc = "0.14"

crates/brk_bencher/README.md

@@ -0,0 +1,43 @@
+# brk_bencher
+
+Resource monitoring for long-running Bitcoin indexing operations.
+
+## What It Enables
+
+Track disk usage, memory consumption (current + peak), and I/O throughput during indexing runs. Progress tracking hooks into brk_logger to record processing milestones automatically.
+
+## Key Features
+
+- **Multi-metric monitoring**: Disk, memory (RSS + peak), I/O read/write
+- **Progress tracking**: Integrates with logging to capture block heights as they're processed
+- **Run comparison**: Outputs timestamped CSVs for comparing multiple runs
+- **macOS optimized**: Uses libproc for accurate process metrics on macOS
+- **Non-blocking**: Monitors in background thread with 5-second sample interval
+
+## Core API
+
+```rust,ignore
+let mut bencher = Bencher::from_cargo_env("brk_indexer", &data_path)?;
+bencher.start()?;
+
+// ... run indexing ...
+
+bencher.stop()?;
+```
+
+## Output Structure
+
+```
+benches/
+└── brk_indexer/
+    └── 1703001234/
+        ├── disk.csv      # timestamp_ms, bytes
+        ├── memory.csv    # timestamp_ms, current, peak
+        ├── io.csv        # timestamp_ms, read, written
+        └── progress.csv  # timestamp_ms, height
+```
+
+## Built On
+
+- `brk_error` for error handling
+- `brk_logger` for progress hook integration

crates/brk_bencher/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bencher/src/disk.rs

@@ -0,0 +1,64 @@
+use std::collections::HashMap;
+use std::fs::{self, File};
+use std::io::{self, Write};
+use std::os::unix::fs::MetadataExt;
+use std::path::{Path, PathBuf};
+use std::time::SystemTime;
+
+pub struct DiskMonitor {
+    cache: HashMap<PathBuf, (u64, SystemTime)>, // path -> (bytes_used, mtime)
+    monitored_path: PathBuf,
+    writer: File,
+}
+
+impl DiskMonitor {
+    pub fn new(monitored_path: &Path, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,disk_usage")?;
+
+        Ok(Self {
+            cache: HashMap::new(),
+            monitored_path: monitored_path.to_path_buf(),
+            writer,
+        })
+    }
+
+    /// Record disk usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok(bytes) = self.scan_recursive(&self.monitored_path.clone()) {
+            writeln!(self.writer, "{},{}", elapsed_ms, bytes)?;
+        }
+        Ok(())
+    }
+
+    fn scan_recursive(&mut self, path: &Path) -> io::Result<u64> {
+        let mut total = 0;
+
+        for entry in fs::read_dir(path)? {
+            let entry = entry?;
+            let path = entry.path();
+            let metadata = entry.metadata()?;
+
+            if metadata.is_file() {
+                let mtime = metadata.modified()?;
+
+                // Check cache: if mtime unchanged, use cached value
+                if let Some((cached_bytes, cached_mtime)) = self.cache.get(&path)
+                    && *cached_mtime == mtime
+                {
+                    total += cached_bytes;
+                    continue;
+                }
+
+                // File is new or modified - get actual disk usage
+                let bytes = metadata.blocks() * 512;
+                self.cache.insert(path, (bytes, mtime));
+                total += bytes;
+            } else if metadata.is_dir() {
+                total += self.scan_recursive(&path)?;
+            }
+        }
+
+        Ok(total)
+    }
+}

crates/brk_bencher/src/io.rs

@@ -0,0 +1,81 @@
+use std::fs::File;
+use std::io::{self, Write};
+use std::path::Path;
+
+#[cfg(target_os = "linux")]
+use std::fs;
+
+#[cfg(target_os = "macos")]
+use libproc::pid_rusage::{pidrusage, RUsageInfoV2};
+
+pub struct IoMonitor {
+    pid: u32,
+    writer: File,
+}
+
+impl IoMonitor {
+    pub fn new(pid: u32, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,bytes_read,bytes_written")?;
+
+        Ok(Self { pid, writer })
+    }
+
+    /// Record I/O usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok((read, written)) = self.get_io_usage() {
+            writeln!(self.writer, "{},{},{}", elapsed_ms, read, written)?;
+        }
+        Ok(())
+    }
+
+    /// Get I/O usage in bytes
+    /// Returns (bytes_read, bytes_written)
+    fn get_io_usage(&self) -> io::Result<(u64, u64)> {
+        #[cfg(target_os = "linux")]
+        {
+            self.get_io_usage_linux()
+        }
+
+        #[cfg(target_os = "macos")]
+        {
+            self.get_io_usage_macos()
+        }
+    }
+
+    #[cfg(target_os = "linux")]
+    fn get_io_usage_linux(&self) -> io::Result<(u64, u64)> {
+        let io_content = fs::read_to_string(format!("/proc/{}/io", self.pid))?;
+
+        let mut read_bytes = None;
+        let mut write_bytes = None;
+
+        for line in io_content.lines() {
+            if line.starts_with("read_bytes:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    read_bytes = value_str.parse::<u64>().ok();
+                }
+            } else if line.starts_with("write_bytes:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    write_bytes = value_str.parse::<u64>().ok();
+                }
+            }
+        }
+
+        match (read_bytes, write_bytes) {
+            (Some(r), Some(w)) => Ok((r, w)),
+            _ => Err(io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse I/O stats from /proc/[pid]/io",
+            )),
+        }
+    }
+
+    #[cfg(target_os = "macos")]
+    fn get_io_usage_macos(&self) -> io::Result<(u64, u64)> {
+        match pidrusage::<RUsageInfoV2>(self.pid as i32) {
+            Ok(info) => Ok((info.ri_diskio_bytesread, info.ri_diskio_byteswritten)),
+            Err(_) => Err(io::Error::other("Failed to get process I/O stats")),
+        }
+    }
+}

crates/brk_bencher/src/lib.rs

@@ -0,0 +1,160 @@
+use std::{
+    fs,
+    path::{Path, PathBuf},
+    sync::{
+        Arc,
+        atomic::{AtomicBool, Ordering},
+    },
+    thread::{self, JoinHandle},
+    time::{Duration, Instant, SystemTime, UNIX_EPOCH},
+};
+
+use brk_error::{Error, Result};
+
+mod disk;
+mod io;
+mod memory;
+mod progression;
+
+use disk::*;
+use io::*;
+use memory::*;
+use parking_lot::Mutex;
+use progression::*;
+
+#[derive(Clone)]
+pub struct Bencher(Arc<BencherInner>);
+
+struct BencherInner {
+    bench_dir: PathBuf,
+    monitored_path: PathBuf,
+    stop_flag: Arc<AtomicBool>,
+    monitor_thread: Mutex<Option<JoinHandle<Result<()>>>>,
+    progression: Arc<ProgressionMonitor>,
+}
+
+impl Bencher {
+    /// Create a new bencher for the given crate name
+    /// Creates directory structure: workspace_root/benches/{crate_name}/{timestamp}/
+    pub fn new(crate_name: &str, workspace_root: &Path, monitored_path: &Path) -> Result<Self> {
+        let timestamp = SystemTime::now().duration_since(UNIX_EPOCH)?.as_secs();
+
+        let bench_dir = workspace_root
+            .join("benches")
+            .join(crate_name)
+            .join(timestamp.to_string());
+
+        fs::create_dir_all(&bench_dir)?;
+
+        let progress_csv = bench_dir.join("progress.csv");
+        let progression = Arc::new(ProgressionMonitor::new(&progress_csv)?);
+        let progression_clone = progression.clone();
+
+        // Register hook with logger
+        brk_logger::register_hook(move |message| {
+            progression_clone.check_and_record(message);
+        })
+        .map_err(|e| std::io::Error::new(std::io::ErrorKind::AlreadyExists, e))?;
+
+        Ok(Self(Arc::new(BencherInner {
+            bench_dir,
+            monitored_path: monitored_path.to_path_buf(),
+            stop_flag: Arc::new(AtomicBool::new(false)),
+            progression,
+            monitor_thread: Mutex::new(None),
+        })))
+    }
+
+    /// Create a bencher using CARGO_MANIFEST_DIR to find workspace root
+    pub fn from_cargo_env(crate_name: &str, monitored_path: &Path) -> Result<Self> {
+        let mut current = std::env::current_dir()
+            .map_err(|e| format!("Failed to get current directory: {}", e))
+            .unwrap();
+
+        let workspace_root = loop {
+            let cargo_toml = current.join("Cargo.toml");
+            if cargo_toml.exists() {
+                let contents = std::fs::read_to_string(&cargo_toml)
+                    .map_err(|e| format!("Failed to read Cargo.toml: {}", e))
+                    .unwrap();
+                if contents.contains("[workspace]") {
+                    break current;
+                }
+            }
+
+            current = current
+                .parent()
+                .ok_or(Error::NotFound("Workspace root not found".into()))?
+                .to_path_buf();
+        };
+
+        Self::new(crate_name, &workspace_root, monitored_path)
+    }
+
+    /// Start monitoring disk usage and memory footprint
+    pub fn start(&mut self) -> Result<()> {
+        if self.0.monitor_thread.lock().is_some() {
+            return Err(Error::Internal("Bencher already started"));
+        }
+
+        let stop_flag = self.0.stop_flag.clone();
+        let bench_dir = self.0.bench_dir.clone();
+        let monitored_path = self.0.monitored_path.clone();
+
+        let handle =
+            thread::spawn(move || monitor_resources(&monitored_path, &bench_dir, stop_flag));
+
+        *self.0.monitor_thread.lock() = Some(handle);
+        Ok(())
+    }
+
+    /// Stop monitoring and wait for the thread to finish
+    pub fn stop(&self) -> Result<()> {
+        self.0.stop_flag.store(true, Ordering::Relaxed);
+
+        if let Some(handle) = self.0.monitor_thread.lock().take() {
+            handle.join().map_err(|_| Error::Internal("Monitor thread panicked"))??;
+        }
+
+        self.0.progression.flush()?;
+
+        Ok(())
+    }
+}
+
+impl Drop for Bencher {
+    fn drop(&mut self) {
+        let _ = self.stop();
+    }
+}
+
+fn monitor_resources(
+    monitored_path: &Path,
+    bench_dir: &Path,
+    stop_flag: Arc<AtomicBool>,
+) -> Result<()> {
+    let pid = std::process::id();
+    let start = Instant::now();
+
+    let mut disk_monitor = DiskMonitor::new(monitored_path, &bench_dir.join("disk.csv"))?;
+    let mut memory_monitor = MemoryMonitor::new(pid, &bench_dir.join("memory.csv"))?;
+    let mut io_monitor = IoMonitor::new(pid, &bench_dir.join("io.csv"))?;
+
+    'l: loop {
+        let elapsed_ms = start.elapsed().as_millis();
+
+        disk_monitor.record(elapsed_ms)?;
+        memory_monitor.record(elapsed_ms)?;
+        io_monitor.record(elapsed_ms)?;
+
+        for _ in 0..50 {
+            // 50 * 100ms = 5 seconds
+            if stop_flag.load(Ordering::Relaxed) {
+                break 'l;
+            }
+            thread::sleep(Duration::from_millis(100));
+        }
+    }
+
+    Ok(())
+}

crates/brk_bencher/src/memory.rs

@@ -0,0 +1,141 @@
+use std::fs::File;
+use std::io::{self, Write};
+use std::path::Path;
+
+#[cfg(target_os = "linux")]
+use std::fs;
+
+#[cfg(target_os = "macos")]
+use std::process::Command;
+
+pub struct MemoryMonitor {
+    pid: u32,
+    writer: File,
+}
+
+impl MemoryMonitor {
+    pub fn new(pid: u32, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,phys_footprint,phys_footprint_peak")?;
+
+        Ok(Self { pid, writer })
+    }
+
+    /// Record memory usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok((footprint, peak)) = self.get_memory_usage() {
+            writeln!(self.writer, "{},{},{}", elapsed_ms, footprint, peak)?;
+        }
+        Ok(())
+    }
+
+    /// Get memory usage in bytes
+    /// Returns (current_bytes, peak_bytes)
+    fn get_memory_usage(&self) -> io::Result<(u64, u64)> {
+        #[cfg(target_os = "linux")]
+        {
+            self.get_memory_usage_linux()
+        }
+
+        #[cfg(target_os = "macos")]
+        {
+            self.get_memory_usage_macos()
+        }
+    }
+
+    #[cfg(target_os = "linux")]
+    fn get_memory_usage_linux(&self) -> io::Result<(u64, u64)> {
+        let status_content = fs::read_to_string(format!("/proc/{}/status", self.pid))?;
+
+        let mut vm_rss = None;
+        let mut vm_hwm = None;
+
+        for line in status_content.lines() {
+            if line.starts_with("VmRSS:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    if let Ok(kb) = value_str.parse::<u64>() {
+                        vm_rss = Some(kb * 1024); // KiB to bytes
+                    }
+                }
+            } else if line.starts_with("VmHWM:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    if let Ok(kb) = value_str.parse::<u64>() {
+                        vm_hwm = Some(kb * 1024); // KiB to bytes
+                    }
+                }
+            }
+        }
+
+        match (vm_rss, vm_hwm) {
+            (Some(rss), Some(hwm)) => Ok((rss, hwm)),
+            _ => Err(io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse memory info from /proc/[pid]/status",
+            )),
+        }
+    }
+
+    #[cfg(target_os = "macos")]
+    fn get_memory_usage_macos(&self) -> io::Result<(u64, u64)> {
+        let output = Command::new("footprint")
+            .args(["-p", &self.pid.to_string()])
+            .output()?;
+
+        let stdout = String::from_utf8(output.stdout).map_err(|_| {
+            io::Error::new(io::ErrorKind::InvalidData, "Invalid UTF-8 from footprint")
+        })?;
+
+        parse_footprint_output(&stdout).ok_or_else(|| {
+            io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse footprint output",
+            )
+        })
+    }
+}
+
+#[cfg(target_os = "macos")]
+fn parse_footprint_output(output: &str) -> Option<(u64, u64)> {
+    let mut phys_footprint = None;
+    let mut phys_footprint_peak = None;
+
+    for line in output.lines() {
+        let line = line.trim();
+
+        if line.starts_with("phys_footprint:") {
+            // Format: "phys_footprint: 7072 KB"
+            let parts: Vec<&str> = line.split_whitespace().collect();
+            if parts.len() >= 3 {
+                // parts[0] = "phys_footprint:"
+                // parts[1] = "7072"
+                // parts[2] = "KB"
+                phys_footprint = parse_size_to_bytes(parts[1], parts[2]);
+            }
+        } else if line.starts_with("phys_footprint_peak:") {
+            // Format: "phys_footprint_peak: 15 MB"
+            let parts: Vec<&str> = line.split_whitespace().collect();
+            if parts.len() >= 3 {
+                phys_footprint_peak = parse_size_to_bytes(parts[1], parts[2]);
+            }
+        }
+    }
+
+    match (phys_footprint, phys_footprint_peak) {
+        (Some(f), Some(p)) => Some((f, p)),
+        _ => None,
+    }
+}
+
+#[cfg(target_os = "macos")]
+fn parse_size_to_bytes(value: &str, unit: &str) -> Option<u64> {
+    let value: f64 = value.parse().ok()?;
+
+    let multiplier = match unit.to_uppercase().as_str() {
+        "KB" => 1024.0,                   // KiB to bytes
+        "MB" => 1024.0 * 1024.0,          // MiB to bytes
+        "GB" => 1024.0 * 1024.0 * 1024.0, // GiB to bytes
+        _ => return None,
+    };
+
+    Some((value * multiplier) as u64)
+}

crates/brk_bencher/src/progression.rs

@@ -0,0 +1,74 @@
+use parking_lot::Mutex;
+use std::{
+    fs,
+    io::{self, BufWriter, Write},
+    path::Path,
+    time::Instant,
+};
+
+/// Patterns to match for progress tracking.
+const PROGRESS_PATTERNS: &[&str] = &[
+    "block ",    // "Indexing block 123..."
+    "chain at ", // "Processing chain at 456..."
+];
+
+pub struct ProgressionMonitor {
+    csv_file: Mutex<BufWriter<fs::File>>,
+    start_time: Instant,
+}
+
+impl ProgressionMonitor {
+    pub fn new(csv_path: &Path) -> io::Result<Self> {
+        let mut csv_file = BufWriter::new(fs::File::create(csv_path)?);
+        writeln!(csv_file, "timestamp_ms,value")?;
+
+        Ok(Self {
+            csv_file: Mutex::new(csv_file),
+            start_time: Instant::now(),
+        })
+    }
+
+    /// Check message for progress patterns and record if found
+    #[inline]
+    pub fn check_and_record(&self, message: &str) {
+        let Some(value) = parse_progress(message) else {
+            return;
+        };
+
+        if value % 10 != 0 {
+            return;
+        }
+
+        let elapsed_ms = self.start_time.elapsed().as_millis();
+        let _ = writeln!(self.csv_file.lock(), "{},{}", elapsed_ms, value);
+    }
+
+    pub fn flush(&self) -> io::Result<()> {
+        self.csv_file.lock().flush()
+    }
+}
+
+/// Parse progress value from message
+#[inline]
+fn parse_progress(message: &str) -> Option<u64> {
+    PROGRESS_PATTERNS
+        .iter()
+        .find_map(|pattern| parse_number_after(message, pattern))
+}
+
+/// Extract number immediately following the pattern
+#[inline]
+fn parse_number_after(message: &str, pattern: &str) -> Option<u64> {
+    let start = message.find(pattern)?;
+    let after = &message[start + pattern.len()..];
+
+    let end = after
+        .find(|c: char| !c.is_ascii_digit())
+        .unwrap_or(after.len());
+
+    if end == 0 {
+        return None;
+    }
+
+    after[..end].parse().ok()
+}

crates/brk_bencher_visualizer/Cargo.toml

@@ -0,0 +1,12 @@
+[package]
+name = "brk_bencher_visualizer"
+description = "A generator of charts for brk_bencher"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+plotters = "0.3.7"

crates/brk_bencher_visualizer/README.md

@@ -0,0 +1,34 @@
+# brk_bencher_visualizer
+
+SVG chart generation for benchmark visualization.
+
+## What It Enables
+
+Turn benchmark CSV data into publication-ready SVG charts showing disk usage, memory (current/peak), progress, and I/O over time. Compare multiple runs side-by-side with automatic color coding.
+
+## Key Features
+
+- **Multi-run comparison**: Overlay multiple benchmark runs with distinct colors
+- **Dual-axis charts**: Memory charts show both current and peak usage (solid vs dashed lines)
+- **Smart scaling**: Automatic unit conversion for bytes (KB/MB/GB) and time (seconds/minutes/hours)
+- **Per-run trimming**: Aligns data by progress cutoffs for fair comparison
+- **Dark theme**: Clean, readable charts with monospace fonts
+
+## Core API
+
+```rust,ignore
+let viz = Visualizer::from_cargo_env()?;
+viz.generate_all_charts()?;  // Process all crates in benches/
+```
+
+## Chart Types
+
+- `disk.svg` - Storage consumption over time
+- `memory.svg` - Current + peak memory usage
+- `progress.svg` - Processing progress (e.g., blocks indexed)
+- `io_read.svg` / `io_write.svg` - I/O throughput
+
+## Input Format
+
+Reads CSV files from `benches/<crate>/<run_id>/`:
+- `disk.csv`, `memory.csv`, `progress.csv`, `io.csv`

crates/brk_bencher_visualizer/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bencher_visualizer/src/chart.rs

@@ -0,0 +1,251 @@
+use crate::data::{DataPoint, DualRun, Result, Run};
+use crate::format;
+use plotters::prelude::*;
+use std::path::Path;
+
+const FONT: &str = "monospace";
+const FONT_SIZE: i32 = 20;
+const FONT_SIZE_BIG: i32 = 30;
+const SIZE: (u32, u32) = (2000, 1000);
+const TIME_BUFFER_MS: u64 = 10_000;
+
+const BG_COLOR: RGBColor = RGBColor(18, 18, 24);
+const TEXT_COLOR: RGBColor = RGBColor(230, 230, 240);
+const COLORS: [RGBColor; 6] = [
+    RGBColor(255, 99, 132),  // Pink/Red
+    RGBColor(54, 162, 235),  // Blue
+    RGBColor(75, 192, 192),  // Teal
+    RGBColor(255, 206, 86),  // Yellow
+    RGBColor(153, 102, 255), // Purple
+    RGBColor(255, 159, 64),  // Orange
+];
+
+pub enum YAxisFormat {
+    Bytes,
+    Number,
+}
+
+pub struct ChartConfig<'a> {
+    pub output_path: &'a Path,
+    pub title: String,
+    pub y_label: String,
+    pub y_format: YAxisFormat,
+}
+
+/// Generate a simple line chart from runs
+pub fn generate(config: ChartConfig, runs: &[Run]) -> Result<()> {
+    if runs.is_empty() {
+        return Ok(());
+    }
+
+    let max_time_ms = runs.iter().map(|r| r.max_timestamp()).max().unwrap_or(1000) + TIME_BUFFER_MS;
+    let max_time_s = max_time_ms as f64 / 1000.0;
+    let max_value = runs.iter().map(|r| r.max_value()).fold(0.0, f64::max);
+
+    let (time_scaled, time_divisor, time_label) = format::time(max_time_s);
+    let (value_scaled, scale_factor, y_label) = scale_y_axis(max_value, &config.y_label, &config.y_format);
+    let x_labels = label_count(time_scaled);
+
+    let root = SVGBackend::new(config.output_path, SIZE).into_drawing_area();
+    root.fill(&BG_COLOR)?;
+
+    let mut chart = ChartBuilder::on(&root)
+        .caption(&config.title, (FONT, FONT_SIZE_BIG).into_font().color(&TEXT_COLOR))
+        .margin(20)
+        .margin_right(40)
+        .x_label_area_size(50)
+        .margin_left(50)
+        .right_y_label_area_size(75)
+        .build_cartesian_2d(0.0..time_scaled * 1.025, 0.0..value_scaled * 1.1)?;
+
+    configure_mesh(&mut chart, time_label, &y_label, &config.y_format, x_labels)?;
+
+    for (idx, run) in runs.iter().enumerate() {
+        let color = COLORS[idx % COLORS.len()];
+        draw_series(&mut chart, &run.data, &run.id, color, time_divisor, scale_factor)?;
+    }
+
+    configure_legend(&mut chart)?;
+    root.present()?;
+    println!("Generated: {}", config.output_path.display());
+    Ok(())
+}
+
+/// Generate a chart with dual series per run (e.g., current + peak memory)
+pub fn generate_dual(
+    config: ChartConfig,
+    runs: &[DualRun],
+    primary_suffix: &str,
+    secondary_suffix: &str,
+) -> Result<()> {
+    if runs.is_empty() {
+        return Ok(());
+    }
+
+    let max_time_ms = runs
+        .iter()
+        .flat_map(|r| r.primary.iter().chain(r.secondary.iter()))
+        .map(|d| d.timestamp_ms)
+        .max()
+        .unwrap_or(1000)
+        + TIME_BUFFER_MS;
+    let max_time_s = max_time_ms as f64 / 1000.0;
+    let max_value = runs.iter().map(|r| r.max_value()).fold(0.0, f64::max);
+
+    let (time_scaled, time_divisor, time_label) = format::time(max_time_s);
+    let (value_scaled, scale_factor, y_label) = scale_y_axis(max_value, &config.y_label, &config.y_format);
+    let x_labels = label_count(time_scaled);
+
+    let root = SVGBackend::new(config.output_path, SIZE).into_drawing_area();
+    root.fill(&BG_COLOR)?;
+
+    let mut chart = ChartBuilder::on(&root)
+        .caption(&config.title, (FONT, FONT_SIZE_BIG).into_font().color(&TEXT_COLOR))
+        .margin(20)
+        .margin_right(40)
+        .x_label_area_size(50)
+        .margin_left(50)
+        .right_y_label_area_size(75)
+        .build_cartesian_2d(0.0..time_scaled * 1.025, 0.0..value_scaled * 1.1)?;
+
+    configure_mesh(&mut chart, time_label, &y_label, &config.y_format, x_labels)?;
+
+    for (idx, run) in runs.iter().enumerate() {
+        let color = COLORS[idx % COLORS.len()];
+
+        // Primary series (solid)
+        draw_series(
+            &mut chart,
+            &run.primary,
+            &format!("{} {}", run.id, primary_suffix),
+            color,
+            time_divisor,
+            scale_factor,
+        )?;
+
+        // Secondary series (dashed)
+        draw_dashed_series(
+            &mut chart,
+            &run.secondary,
+            &format!("{} {}", run.id, secondary_suffix),
+            color.mix(0.5),
+            time_divisor,
+            scale_factor,
+        )?;
+    }
+
+    configure_legend(&mut chart)?;
+    root.present()?;
+    println!("Generated: {}", config.output_path.display());
+    Ok(())
+}
+
+fn scale_y_axis(max_value: f64, base_label: &str, y_format: &YAxisFormat) -> (f64, f64, String) {
+    match y_format {
+        YAxisFormat::Bytes => {
+            let (scaled, unit) = format::bytes(max_value);
+            let factor = max_value / scaled;
+            (scaled, factor, format!("{} ({})", base_label, unit))
+        }
+        YAxisFormat::Number => (max_value, 1.0, base_label.to_string()),
+    }
+}
+
+/// Calculate appropriate label count to avoid duplicates when rounding to integers
+fn label_count(max_value: f64) -> usize {
+    let max_int = max_value.ceil() as usize;
+    // Don't exceed the range, cap at 12 for readability
+    max_int.clamp(2, 12)
+}
+
+type Chart<'a, 'b> = ChartContext<
+    'a,
+    SVGBackend<'b>,
+    Cartesian2d<plotters::coord::types::RangedCoordf64, plotters::coord::types::RangedCoordf64>,
+>;
+
+fn configure_mesh(chart: &mut Chart, x_label: &str, y_label: &str, y_format: &YAxisFormat, x_labels: usize) -> Result<()> {
+    let y_formatter: Box<dyn Fn(&f64) -> String> = match y_format {
+        YAxisFormat::Bytes => Box::new(|y: &f64| {
+            if y.fract() == 0.0 {
+                format!("{:.0}", y)
+            } else {
+                format!("{:.1}", y)
+            }
+        }),
+        YAxisFormat::Number => Box::new(|y: &f64| format::axis_number(*y)),
+    };
+
+    chart
+        .configure_mesh()
+        .disable_mesh()
+        .x_desc(x_label)
+        .y_desc(y_label)
+        .x_label_formatter(&|x| format!("{:.0}", x))
+        .y_label_formatter(&y_formatter)
+        .x_labels(x_labels)
+        .y_labels(10)
+        .x_label_style((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.7)))
+        .y_label_style((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.7)))
+        .axis_style(TEXT_COLOR.mix(0.3))
+        .draw()?;
+    Ok(())
+}
+
+fn draw_series(
+    chart: &mut Chart,
+    data: &[DataPoint],
+    label: &str,
+    color: RGBColor,
+    time_divisor: f64,
+    scale_factor: f64,
+) -> Result<()> {
+    let points = data
+        .iter()
+        .map(|d| (d.timestamp_ms as f64 / 1000.0 / time_divisor, d.value / scale_factor));
+
+    chart
+        .draw_series(LineSeries::new(points, color.stroke_width(1)))?
+        .label(label)
+        .legend(move |(x, y)| PathElement::new(vec![(x, y), (x + 20, y)], color.stroke_width(1)));
+    Ok(())
+}
+
+fn draw_dashed_series(
+    chart: &mut Chart,
+    data: &[DataPoint],
+    label: &str,
+    color: RGBAColor,
+    time_divisor: f64,
+    scale_factor: f64,
+) -> Result<()> {
+    let points: Vec<_> = data
+        .iter()
+        .map(|d| (d.timestamp_ms as f64 / 1000.0 / time_divisor, d.value / scale_factor))
+        .collect();
+
+    // Draw dashed line by skipping every other segment
+    chart
+        .draw_series(
+            points
+                .windows(2)
+                .enumerate()
+                .filter(|(i, _)| i % 2 == 0)
+                .map(|(_, w)| PathElement::new(vec![w[0], w[1]], color.stroke_width(2))),
+        )?
+        .label(label)
+        .legend(move |(x, y)| PathElement::new(vec![(x, y), (x + 10, y), (x + 20, y)], color.stroke_width(2)));
+    Ok(())
+}
+
+fn configure_legend<'a>(chart: &mut Chart<'a, 'a>) -> Result<()> {
+    chart
+        .configure_series_labels()
+        .position(SeriesLabelPosition::UpperLeft)
+        .label_font((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.9)))
+        .background_style(BG_COLOR.mix(0.98))
+        .border_style(BG_COLOR)
+        .margin(10)
+        .draw()?;
+    Ok(())
+}

crates/brk_bencher_visualizer/src/data.rs

@@ -0,0 +1,238 @@
+use std::{collections::HashMap, fs, path::Path};
+
+pub type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
+
+#[derive(Debug, Clone)]
+pub struct DataPoint {
+    pub timestamp_ms: u64,
+    pub value: f64,
+}
+
+/// Per-run cutoff timestamps for fair comparison
+pub struct Cutoffs {
+    by_id: HashMap<String, u64>,
+    default: u64,
+}
+
+impl Cutoffs {
+    /// Calculate cutoffs from progress runs.
+    /// Finds the common max progress, then returns when each run reached it.
+    pub fn from_progress(progress_runs: &[Run]) -> Self {
+        const TIME_BUFFER_MS: u64 = 10_000;
+
+        if progress_runs.is_empty() {
+            return Self {
+                by_id: HashMap::new(),
+                default: u64::MAX,
+            };
+        }
+
+        // Find the minimum of max progress values (the common point all runs reached)
+        let common_progress = progress_runs
+            .iter()
+            .map(|r| r.max_value())
+            .fold(f64::MAX, f64::min);
+
+        let by_id: HashMap<_, _> = progress_runs
+            .iter()
+            .map(|run| {
+                let cutoff = run
+                    .data
+                    .iter()
+                    .find(|d| d.value >= common_progress)
+                    .map(|d| d.timestamp_ms)
+                    .unwrap_or_else(|| run.max_timestamp())
+                    .saturating_add(TIME_BUFFER_MS);
+                (run.id.clone(), cutoff)
+            })
+            .collect();
+
+        let default = by_id.values().copied().max().unwrap_or(u64::MAX);
+
+        Self { by_id, default }
+    }
+
+    pub fn get(&self, id: &str) -> u64 {
+        self.by_id.get(id).copied().unwrap_or(self.default)
+    }
+
+    pub fn trim_runs(&self, runs: &[Run]) -> Vec<Run> {
+        runs.iter().map(|r| r.trimmed(self.get(&r.id))).collect()
+    }
+
+    pub fn trim_dual_runs(&self, runs: &[DualRun]) -> Vec<DualRun> {
+        runs.iter().map(|r| r.trimmed(self.get(&r.id))).collect()
+    }
+}
+
+#[derive(Debug, Clone)]
+pub struct Run {
+    pub id: String,
+    pub data: Vec<DataPoint>,
+}
+
+impl Run {
+    pub fn max_timestamp(&self) -> u64 {
+        self.data.iter().map(|d| d.timestamp_ms).max().unwrap_or(0)
+    }
+
+    pub fn max_value(&self) -> f64 {
+        self.data.iter().map(|d| d.value).fold(0.0, f64::max)
+    }
+
+    pub fn trimmed(&self, max_timestamp_ms: u64) -> Self {
+        Self {
+            id: self.id.clone(),
+            data: self
+                .data
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+        }
+    }
+}
+
+/// Two data series from a single run (e.g., memory footprint + peak, or io read + write)
+#[derive(Debug, Clone)]
+pub struct DualRun {
+    pub id: String,
+    pub primary: Vec<DataPoint>,
+    pub secondary: Vec<DataPoint>,
+}
+
+impl DualRun {
+    pub fn trimmed(&self, max_timestamp_ms: u64) -> Self {
+        Self {
+            id: self.id.clone(),
+            primary: self
+                .primary
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+            secondary: self
+                .secondary
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+        }
+    }
+
+    pub fn max_value(&self) -> f64 {
+        self.primary
+            .iter()
+            .chain(self.secondary.iter())
+            .map(|d| d.value)
+            .fold(0.0, f64::max)
+    }
+}
+
+pub fn read_runs(crate_path: &Path, filename: &str) -> Result<Vec<Run>> {
+    let mut runs = Vec::new();
+
+    for entry in fs::read_dir(crate_path)? {
+        let run_path = entry?.path();
+        if !run_path.is_dir() {
+            continue;
+        }
+
+        let run_id = run_path
+            .file_name()
+            .and_then(|n| n.to_str())
+            .ok_or("Invalid run ID")?
+            .to_string();
+
+        // Skip underscore-prefixed or numeric-only directories
+        if run_id.starts_with('_') || run_id.chars().all(|c| c.is_ascii_digit()) {
+            continue;
+        }
+
+        let csv_path = run_path.join(filename);
+        if csv_path.exists()
+            && let Ok(data) = read_csv(&csv_path)
+        {
+            runs.push(Run { id: run_id, data });
+        }
+    }
+
+    Ok(runs)
+}
+
+pub fn read_dual_runs(crate_path: &Path, filename: &str) -> Result<Vec<DualRun>> {
+    let mut runs = Vec::new();
+
+    for entry in fs::read_dir(crate_path)? {
+        let run_path = entry?.path();
+        if !run_path.is_dir() {
+            continue;
+        }
+
+        let run_id = run_path
+            .file_name()
+            .and_then(|n| n.to_str())
+            .ok_or("Invalid run ID")?
+            .to_string();
+
+        if run_id.starts_with('_') || run_id.chars().all(|c| c.is_ascii_digit()) {
+            continue;
+        }
+
+        let csv_path = run_path.join(filename);
+        if csv_path.exists()
+            && let Ok((primary, secondary)) = read_dual_csv(&csv_path)
+        {
+            runs.push(DualRun {
+                id: run_id,
+                primary,
+                secondary,
+            });
+        }
+    }
+
+    Ok(runs)
+}
+
+fn read_csv(path: &Path) -> Result<Vec<DataPoint>> {
+    let content = fs::read_to_string(path)?;
+    let data = content
+        .lines()
+        .skip(1) // header
+        .filter_map(|line| {
+            let mut parts = line.split(',');
+            let timestamp_ms = parts.next()?.parse().ok()?;
+            let value = parts.next()?.parse().ok()?;
+            Some(DataPoint {
+                timestamp_ms,
+                value,
+            })
+        })
+        .collect();
+    Ok(data)
+}
+
+fn read_dual_csv(path: &Path) -> Result<(Vec<DataPoint>, Vec<DataPoint>)> {
+    let content = fs::read_to_string(path)?;
+    let mut primary = Vec::new();
+    let mut secondary = Vec::new();
+
+    for line in content.lines().skip(1) {
+        let mut parts = line.split(',');
+        if let (Some(ts), Some(v1), Some(v2)) = (parts.next(), parts.next(), parts.next())
+            && let (Ok(timestamp_ms), Ok(val1), Ok(val2)) =
+                (ts.parse(), v1.parse::<f64>(), v2.parse::<f64>())
+        {
+            primary.push(DataPoint {
+                timestamp_ms,
+                value: val1,
+            });
+            secondary.push(DataPoint {
+                timestamp_ms,
+                value: val2,
+            });
+        }
+    }
+
+    Ok((primary, secondary))
+}

crates/brk_bencher_visualizer/src/format.rs

@@ -0,0 +1,45 @@
+const KIB: f64 = 1024.0;
+const MIB: f64 = KIB * 1024.0;
+const GIB: f64 = MIB * 1024.0;
+
+const MINUTE: f64 = 60.0;
+const HOUR: f64 = 3600.0;
+
+/// Returns (scaled_value, unit_suffix)
+pub fn bytes(bytes: f64) -> (f64, &'static str) {
+    if bytes >= GIB {
+        (bytes / GIB, "GiB")
+    } else if bytes >= MIB {
+        (bytes / MIB, "MiB")
+    } else if bytes >= KIB {
+        (bytes / KIB, "KiB")
+    } else {
+        (bytes, "bytes")
+    }
+}
+
+/// Returns (scaled_value, divisor, axis_label)
+pub fn time(seconds: f64) -> (f64, f64, &'static str) {
+    if seconds >= HOUR * 2.0 {
+        (seconds / HOUR, HOUR, "Time (h)")
+    } else if seconds >= MINUTE * 2.0 {
+        (seconds / MINUTE, MINUTE, "Time (min)")
+    } else {
+        (seconds, 1.0, "Time (s)")
+    }
+}
+
+pub fn axis_number(value: f64) -> String {
+    if value >= 1000.0 {
+        let k = value / 1000.0;
+        if k.fract() == 0.0 || k >= 100.0 {
+            format!("{:.0}k", k)
+        } else if k >= 10.0 {
+            format!("{:.1}k", k)
+        } else {
+            format!("{:.2}k", k)
+        }
+    } else {
+        format!("{:.0}", value)
+    }
+}

crates/brk_bencher_visualizer/src/lib.rs

@@ -0,0 +1,249 @@
+mod chart;
+mod data;
+mod format;
+
+use data::{read_dual_runs, read_runs, Cutoffs, DualRun, Result, Run};
+use std::{
+    fs,
+    path::{Path, PathBuf},
+};
+
+pub struct Visualizer {
+    workspace_root: PathBuf,
+}
+
+impl Visualizer {
+    pub fn new(workspace_root: impl AsRef<Path>) -> Self {
+        Self {
+            workspace_root: workspace_root.as_ref().to_path_buf(),
+        }
+    }
+
+    pub fn from_cargo_env() -> Result<Self> {
+        let workspace_root = Path::new(env!("CARGO_MANIFEST_DIR"))
+            .parent()
+            .and_then(|p| p.parent())
+            .ok_or("Failed to find workspace root")?
+            .to_path_buf();
+        Ok(Self { workspace_root })
+    }
+
+    pub fn generate_all_charts(&self) -> Result<()> {
+        let benches_dir = self.workspace_root.join("benches");
+        if !benches_dir.exists() {
+            return Err("Benches directory does not exist".into());
+        }
+
+        for entry in fs::read_dir(&benches_dir)? {
+            let path = entry?.path();
+            if path.is_dir() {
+                let crate_name = path
+                    .file_name()
+                    .and_then(|n| n.to_str())
+                    .ok_or("Invalid crate name")?;
+
+                println!("Generating charts for crate: {}", crate_name);
+                self.generate_crate_charts(&path, crate_name)?;
+            }
+        }
+
+        Ok(())
+    }
+
+    fn generate_crate_charts(&self, crate_path: &Path, crate_name: &str) -> Result<()> {
+        let disk_runs = read_runs(crate_path, "disk.csv")?;
+        let memory_runs = read_dual_runs(crate_path, "memory.csv")?;
+        let progress_runs = read_runs(crate_path, "progress.csv")?;
+        let io_runs = read_dual_runs(crate_path, "io.csv")?;
+
+        // Combined charts (all runs)
+        self.generate_combined_charts(crate_path, crate_name, &disk_runs, &memory_runs, &progress_runs, &io_runs)?;
+
+        // Individual charts (one per run)
+        self.generate_individual_charts(crate_path, crate_name, &disk_runs, &memory_runs, &progress_runs, &io_runs)?;
+
+        Ok(())
+    }
+
+    fn generate_combined_charts(
+        &self,
+        crate_path: &Path,
+        crate_name: &str,
+        disk_runs: &[Run],
+        memory_runs: &[DualRun],
+        progress_runs: &[Run],
+        io_runs: &[DualRun],
+    ) -> Result<()> {
+        let cutoffs = Cutoffs::from_progress(progress_runs);
+
+        // Trim data to per-run cutoffs for fair comparison
+        let disk_trimmed = cutoffs.trim_runs(disk_runs);
+        let memory_trimmed = cutoffs.trim_dual_runs(memory_runs);
+        let io_trimmed = cutoffs.trim_dual_runs(io_runs);
+
+        if !disk_trimmed.is_empty() {
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("disk.svg"),
+                    title: format!("{} — Disk Usage", crate_name),
+                    y_label: "Disk Usage".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &disk_trimmed,
+            )?;
+        }
+
+        if !memory_trimmed.is_empty() {
+            chart::generate_dual(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("memory.svg"),
+                    title: format!("{} — Memory", crate_name),
+                    y_label: "Memory".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &memory_trimmed,
+                "(current)",
+                "(peak)",
+            )?;
+        }
+
+        if !progress_runs.is_empty() {
+            let progress_trimmed = cutoffs.trim_runs(progress_runs);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("progress.svg"),
+                    title: format!("{} — Progress", crate_name),
+                    y_label: "Progress".to_string(),
+                    y_format: chart::YAxisFormat::Number,
+                },
+                &progress_trimmed,
+            )?;
+        }
+
+        if !io_trimmed.is_empty() {
+            // I/O Read (primary column)
+            let io_read: Vec<_> = io_trimmed
+                .iter()
+                .map(|r| Run {
+                    id: r.id.clone(),
+                    data: r.primary.clone(),
+                })
+                .collect();
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("io_read.svg"),
+                    title: format!("{} — I/O Read", crate_name),
+                    y_label: "Bytes Read".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &io_read,
+            )?;
+
+            // I/O Write (secondary column)
+            let io_write: Vec<_> = io_trimmed
+                .iter()
+                .map(|r| Run {
+                    id: r.id.clone(),
+                    data: r.secondary.clone(),
+                })
+                .collect();
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("io_write.svg"),
+                    title: format!("{} — I/O Write", crate_name),
+                    y_label: "Bytes Written".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &io_write,
+            )?;
+        }
+
+        Ok(())
+    }
+
+    fn generate_individual_charts(
+        &self,
+        crate_path: &Path,
+        crate_name: &str,
+        disk_runs: &[Run],
+        memory_runs: &[DualRun],
+        progress_runs: &[Run],
+        io_runs: &[DualRun],
+    ) -> Result<()> {
+        for run in disk_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("disk.svg"),
+                    title: format!("{} — Disk Usage", crate_name),
+                    y_label: "Disk Usage".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(run),
+            )?;
+        }
+
+        for run in memory_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate_dual(
+                chart::ChartConfig {
+                    output_path: &run_path.join("memory.svg"),
+                    title: format!("{} — Memory", crate_name),
+                    y_label: "Memory".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(run),
+                "(current)",
+                "(peak)",
+            )?;
+        }
+
+        for run in progress_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("progress.svg"),
+                    title: format!("{} — Progress", crate_name),
+                    y_label: "Progress".to_string(),
+                    y_format: chart::YAxisFormat::Number,
+                },
+                std::slice::from_ref(run),
+            )?;
+        }
+
+        for run in io_runs {
+            let run_path = crate_path.join(&run.id);
+
+            let read_run = Run {
+                id: run.id.clone(),
+                data: run.primary.clone(),
+            };
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("io_read.svg"),
+                    title: format!("{} — I/O Read", crate_name),
+                    y_label: "Bytes Read".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(&read_run),
+            )?;
+
+            let write_run = Run {
+                id: run.id.clone(),
+                data: run.secondary.clone(),
+            };
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("io_write.svg"),
+                    title: format!("{} — I/O Write", crate_name),
+                    y_label: "Bytes Written".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(&write_run),
+            )?;
+        }
+
+        Ok(())
+    }
+}
+

crates/brk_bencher_visualizer/src/main.rs

@@ -0,0 +1,6 @@
+use brk_bencher_visualizer::Visualizer;
+
+fn main() {
+    let v = Visualizer::from_cargo_env().unwrap();
+    v.generate_all_charts().unwrap();
+}

crates/brk_binder/Cargo.toml

@@ -0,0 +1,13 @@
+[package]
+name = "brk_binder"
+description = "A generator of binding files for other languages"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+brk_query = { workspace = true }
+brk_types = { workspace = true }

crates/brk_binder/README.md

@@ -0,0 +1,42 @@
+# brk_binder
+
+Code generation for BRK client libraries.
+
+## What It Enables
+
+Generate typed metric catalogs and constants for JavaScript/TypeScript clients. Keeps frontend code in sync with available metrics without manual maintenance.
+
+## Key Features
+
+- **Metric catalog**: Generates `metrics.js` with all metric IDs and their supported indexes
+- **Compression**: Metric names compressed via word-to-base62 mapping for smaller bundles
+- **Mining pools**: Generates `pools.js` with pool ID to name mapping
+- **Version sync**: Generates `version.js` matching server version
+
+## Core API
+
+```rust,ignore
+generate_js_files(&query, &modules_path)?;
+```
+
+## Generated Files
+
+```
+modules/brk-client/generated/
+├── version.js    # export const VERSION = "vX.Y.Z"
+├── metrics.js    # INDEXES, COMPRESSED_METRIC_TO_INDEXES
+└── pools.js      # POOL_ID_TO_POOL_NAME
+```
+
+## Metric Compression
+
+To minimize bundle size, metric names are compressed:
+1. Extract all words from metric names
+2. Sort by frequency
+3. Map to base52 codes (A-Z, a-z)
+4. Store compressed metric → index group mapping
+
+## Built On
+
+- `brk_query` for metric enumeration
+- `brk_types` for mining pool data

crates/brk_binder/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_binder/src/js.rs

@@ -0,0 +1,240 @@
+use std::{
+    collections::{BTreeMap, HashMap},
+    fs, io,
+    path::Path,
+};
+
+use brk_query::Query;
+use brk_types::{Index, pools};
+
+use super::VERSION;
+
+const AUTO_GENERATED_DISCLAIMER: &str = "//
+// File auto-generated, any modifications will be overwritten
+//";
+
+pub fn generate_js_files(query: &Query, modules_path: &Path) -> io::Result<()> {
+    let path = modules_path.join("brk-client");
+
+    if !fs::exists(&path)? {
+        return Ok(());
+    }
+
+    let path = path.join("generated");
+    fs::create_dir_all(&path)?;
+
+    generate_version_file(&path)?;
+    generate_metrics_file(query, &path)?;
+    generate_pools_file(&path)
+}
+
+fn generate_version_file(parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("version.js"));
+
+    let contents = format!(
+        "{AUTO_GENERATED_DISCLAIMER}
+
+export const VERSION = \"v{VERSION}\";
+"
+    );
+
+    fs::write(path, contents)
+}
+
+fn generate_pools_file(parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("pools.js"));
+
+    let pools = pools();
+
+    let mut contents = format!("{AUTO_GENERATED_DISCLAIMER}\n");
+
+    contents += "
+/**
+ * @typedef {typeof POOL_ID_TO_POOL_NAME} PoolIdToPoolName
+ * @typedef {keyof PoolIdToPoolName} PoolId
+ */
+
+export const POOL_ID_TO_POOL_NAME = /** @type {const} */ ({
+";
+
+    let mut sorted_pools: Vec<_> = pools.iter().collect();
+    sorted_pools.sort_by(|a, b| a.name.to_lowercase().cmp(&b.name.to_lowercase()));
+
+    contents += &sorted_pools
+        .iter()
+        .map(|pool| {
+            let slug = pool.slug();
+            format!("  {slug}: \"{}\",", pool.name)
+        })
+        .collect::<Vec<_>>()
+        .join("\n");
+
+    contents += "\n});\n";
+
+    fs::write(path, contents)
+}
+
+fn generate_metrics_file(query: &Query, parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("metrics.js"));
+
+    let indexes = Index::all();
+
+    let mut contents = format!(
+        "{AUTO_GENERATED_DISCLAIMER}
+
+export const INDEXES = /** @type {{const}} */ ([
+{}
+]);
+
+/**
+ * @typedef {{typeof INDEXES[number]}} IndexName
+ */
+",
+        indexes
+            .iter()
+            .map(|i| format!("  \"{}\"", i.serialize_long()))
+            .collect::<Vec<_>>()
+            .join(",\n")
+    );
+
+    // contents += &indexes
+    //     .iter()
+    //     .map(|i| format!(" * @typedef {{\"{}\"}} {i}", i.serialize_long()))
+    //     .collect::<Vec<_>>()
+    //     .join("\n");
+
+    //     contents += &format!(
+    //         "
+    //  * @typedef {{{}}} Index
+    //  */
+    // ",
+    //         indexes
+    //             .iter()
+    //             .map(|i| i.to_string())
+    //             .collect::<Vec<_>>()
+    //             .join(" | ")
+    //     );
+
+    let mut unique_index_groups = BTreeMap::new();
+
+    let mut word_to_freq: BTreeMap<_, usize> = BTreeMap::new();
+    query.metric_to_index_to_vec().keys().for_each(|metric| {
+        metric.split("_").for_each(|word| {
+            *word_to_freq.entry(word).or_default() += 1;
+        });
+    });
+    let mut word_to_freq = word_to_freq.into_iter().collect::<Vec<_>>();
+    word_to_freq.sort_unstable_by(|a, b| b.1.cmp(&a.1).then(a.0.cmp(b.0)));
+    let words = word_to_freq
+        .into_iter()
+        .map(|(str, _)| str)
+        .collect::<Vec<_>>();
+
+    contents += &format!(
+        "
+export const INDEX_TO_WORD = [
+  {}
+];
+
+",
+        words
+            .iter()
+            .enumerate()
+            .map(|(index, word)| format!("\"{word}\", // {}", index_to_letters(index)))
+            .collect::<Vec<_>>()
+            .join("\n  ")
+    );
+
+    let word_to_base62 = words
+        .into_iter()
+        .enumerate()
+        .map(|(i, w)| (w, index_to_letters(i)))
+        .collect::<HashMap<_, _>>();
+
+    let mut ser_metric_to_indexes = "
+/** @type {Record<string, IndexName[]>} */
+export const COMPRESSED_METRIC_TO_INDEXES = {
+"
+    .to_string();
+
+    query
+        .metric_to_index_to_vec()
+        .iter()
+        .for_each(|(metric, index_to_vec)| {
+            let indexes = index_to_vec
+                .keys()
+                .map(|i| format!("\"{}\"", i.serialize_long()))
+                .collect::<Vec<_>>()
+                .join(", ");
+            let indexes = format!("[{indexes}]");
+            let unique = unique_index_groups.len();
+            let index = index_to_letters(*unique_index_groups.entry(indexes).or_insert(unique));
+
+            let compressed_metric = metric.split('_').fold(String::new(), |mut acc, w| {
+                if !acc.is_empty() {
+                    acc.push('_');
+                }
+                acc.push_str(&word_to_base62[w]);
+                acc
+            });
+
+            ser_metric_to_indexes += &format!("  {compressed_metric}: {index},\n");
+        });
+
+    ser_metric_to_indexes += "};
+";
+
+    let mut sorted_groups: Vec<_> = unique_index_groups.into_iter().collect();
+    sorted_groups.sort_by_key(|(_, index)| *index);
+    sorted_groups.into_iter().for_each(|(group, index)| {
+        let index = index_to_letters(index);
+        contents += &format!("/** @type {{IndexName[]}} */\nconst {index} = {group};\n");
+    });
+
+    contents += &ser_metric_to_indexes;
+
+    fs::write(path, contents)
+}
+
+fn index_to_letters(mut index: usize) -> String {
+    if index < 52 {
+        return (index_to_char(index) as char).to_string();
+    }
+    let mut result = [0u8; 8];
+    let mut pos = 8;
+    loop {
+        pos -= 1;
+        result[pos] = index_to_char(index % 52);
+        index /= 52;
+        if index == 0 {
+            break;
+        }
+        index -= 1;
+    }
+    unsafe { String::from_utf8_unchecked(result[pos..].to_vec()) }
+}
+
+fn index_to_char(index: usize) -> u8 {
+    match index {
+        0..=25 => b'A' + index as u8,
+        26..=51 => b'a' + (index - 26) as u8,
+        _ => unreachable!(),
+    }
+}
+
+// fn letters_to_index(s: &str) -> usize {
+//     let mut result = 0;
+//     for byte in s.bytes() {
+//         let value = char_to_index(byte) as usize;
+//         result = result * 52 + value + 1;
+//     }
+//     result - 1
+// }
+
+// fn char_to_index(byte: u8) -> u8 {
+//     match byte {
+//         b'A'..=b'Z' => byte - b'A',
+//         b'a'..=b'z' => byte - b'a' + 26,
+//         _ => 255, // Invalid
+//     }
+// }

crates/brk_binder/src/lib.rs

@@ -0,0 +1,5 @@
+mod js;
+
+pub use js::*;
+
+pub const VERSION: &str = env!("CARGO_PKG_VERSION");

crates/brk_binder/src/python.rs

@@ -0,0 +1 @@
+// TODO ?

crates/brk_bundler/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "brk_bundler"
+description = "A thin wrapper around rolldown"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+log = { workspace = true }
+notify = "8.2.0"
+# rolldown = { path = "../../../rolldown/crates/rolldown", package = "brk_rolldown" }
+rolldown = { version = "0.5.1", package = "brk_rolldown" }
+sugar_path = "1.2.1"
+tokio = { workspace = true }

crates/brk_bundler/README.md

@@ -0,0 +1,32 @@
+# brk_bundler
+
+JavaScript bundling with watch mode for BRK web interfaces.
+
+## What It Enables
+
+Bundle and minify JavaScript modules using Rolldown, with file watching for development. Handles module copying, source map generation, and cache-busting via hashed filenames.
+
+## Key Features
+
+- **Rolldown integration**: Fast Rust-based bundler with tree-shaking and minification
+- **Watch mode**: Rebuilds on file changes with live module syncing
+- **Source maps**: Full debugging support in production builds
+- **Cache busting**: Hashes main bundle filename, updates HTML references automatically
+- **Service worker versioning**: Injects package version into service worker files
+
+## Core API
+
+```rust,ignore
+// One-shot build
+let dist = bundle(modules_path, websites_path, "src", false).await?;
+
+// Watch mode for development
+bundle(modules_path, websites_path, "src", true).await?;
+```
+
+## Build Pipeline
+
+1. Copy shared modules to source scripts directory
+2. Bundle with Rolldown (minified, with source maps)
+3. Update `index.html` with hashed script references
+4. Inject version into service worker

crates/brk_bundler/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bundler/src/lib.rs

@@ -0,0 +1,193 @@
+#![doc = include_str!("../README.md")]
+
+use std::{
+    fs, io,
+    path::{Path, PathBuf},
+    sync::Arc,
+};
+
+use log::error;
+use notify::{EventKind, RecursiveMode, Watcher};
+use rolldown::{
+    Bundler, BundlerOptions, InlineConstConfig, InlineConstMode, InlineConstOption,
+    OptimizationOption, RawMinifyOptions, SourceMapType,
+};
+use sugar_path::SugarPath;
+use tokio::sync::Mutex;
+
+const VERSION: &str = env!("CARGO_PKG_VERSION");
+
+pub async fn bundle(
+    modules_path: &Path,
+    websites_path: &Path,
+    source_folder: &str,
+    watch: bool,
+) -> io::Result<PathBuf> {
+    let relative_modules_path = modules_path;
+    let relative_source_path = websites_path.join(source_folder);
+    let relative_dist_path = websites_path.join("dist");
+
+    let absolute_modules_path = relative_modules_path.absolutize();
+    let absolute_modules_path_clone = absolute_modules_path.clone();
+    let absolute_websites_path = websites_path.absolutize();
+    let absolute_websites_path_clone = absolute_websites_path.clone();
+
+    let absolute_source_path = relative_source_path.absolutize();
+    let absolute_source_index_path = absolute_source_path.join("index.html");
+    let absolute_source_index_path_clone = absolute_source_index_path.clone();
+    let absolute_source_scripts_path = absolute_source_path.join("scripts");
+    let absolute_source_scripts_modules_path = absolute_source_scripts_path.join("modules");
+    let absolute_source_sw_path = absolute_source_path.join("service-worker.js");
+    let absolute_source_sw_path_clone = absolute_source_sw_path.clone();
+
+    let absolute_dist_path = relative_dist_path.absolutize();
+    let absolute_dist_scripts_path = absolute_dist_path.join("scripts");
+    let absolute_dist_scripts_entry_path = absolute_dist_scripts_path.join("entry.js");
+    let absolute_dist_scripts_entry_path_clone = absolute_dist_scripts_entry_path.clone();
+    let absolute_dist_index_path = absolute_dist_path.join("index.html");
+    let absolute_dist_sw_path = absolute_dist_path.join("service-worker.js");
+
+    let _ = fs::remove_dir_all(&absolute_dist_path);
+    let _ = fs::remove_dir_all(&absolute_source_scripts_modules_path);
+    copy_dir_all(
+        &absolute_modules_path,
+        &absolute_source_scripts_modules_path,
+    )?;
+    copy_dir_all(&absolute_source_path, &absolute_dist_path)?;
+    fs::remove_dir_all(&absolute_dist_scripts_path)?;
+    fs::create_dir(&absolute_dist_scripts_path)?;
+
+    // dbg!(BundlerOptions::default());
+
+    let mut bundler = Bundler::new(BundlerOptions {
+        input: Some(vec![format!("./{source_folder}/scripts/entry.js").into()]),
+        dir: Some("./dist/scripts".to_string()),
+        cwd: Some(absolute_websites_path),
+        minify: Some(RawMinifyOptions::Bool(true)),
+        sourcemap: Some(SourceMapType::File),
+        // advanced_chunks: Some(AdvancedChunksOptions {
+        //     // min_size: Some(1000.0),
+        //     min_share_count: Some(20),
+        //     // min_module_size: S
+        //     // include_dependencies_recursively: Some(true),
+        //     ..Default::default()
+        // }),
+        //
+        // inline_dynamic_imports
+        // experimental: Some(ExperimentalOptions {
+        //     strict_execution_order: Some(true),
+        //     ..Default::default()
+        // }),
+        optimization: Some(OptimizationOption {
+            inline_const: Some(InlineConstOption::Config(InlineConstConfig {
+                mode: Some(InlineConstMode::All),
+                ..Default::default()
+            })),
+            // Needs benchmarks
+            // pife_for_module_wrappers: Some(true),
+            ..Default::default()
+        }),
+        ..Default::default()
+    })
+    .unwrap();
+
+    if let Err(error) = bundler.write().await {
+        error!("{error:?}");
+    }
+
+    let update_dist_index = move || {
+        let mut contents = fs::read_to_string(&absolute_source_index_path).unwrap();
+
+        if let Ok(entry) = fs::read_to_string(&absolute_dist_scripts_entry_path_clone)
+            && let Some(start) = entry.find("main")
+            && let Some(end) = entry.find(".js")
+        {
+            let main_hashed = &entry[start..end];
+            contents = contents.replace("/scripts/main.js", &format!("/scripts/{main_hashed}.js"));
+        }
+
+        let _ = fs::write(&absolute_dist_index_path, contents);
+    };
+
+    let update_source_sw = move || {
+        let contents = fs::read_to_string(&absolute_source_sw_path)
+            .unwrap()
+            .replace("__VERSION__", &format!("v{VERSION}"));
+        let _ = fs::write(&absolute_dist_sw_path, contents);
+    };
+
+    update_dist_index();
+    update_source_sw();
+
+    if !watch {
+        return Ok(relative_dist_path);
+    }
+
+    tokio::spawn(async move {
+        let mut event_watcher = notify::recommended_watcher(
+            move |res: Result<notify::Event, notify::Error>| match res {
+                Ok(event) => match event.kind {
+                    EventKind::Create(_) => event.paths,
+                    EventKind::Modify(_) => event.paths,
+                    _ => vec![],
+                }
+                .into_iter()
+                .for_each(|path| {
+                    let path = path.absolutize();
+
+                    if path == absolute_dist_scripts_entry_path
+                        || path == absolute_source_index_path_clone
+                    {
+                        update_dist_index();
+                    } else if path == absolute_source_sw_path_clone {
+                        update_source_sw();
+                    } else if let Ok(suffix) = path.strip_prefix(&absolute_modules_path) {
+                        let source_modules_path = absolute_source_scripts_modules_path.join(suffix);
+                        if path.is_file() {
+                            let _ = fs::create_dir_all(path.parent().unwrap());
+                            let _ = fs::copy(&path, &source_modules_path);
+                        }
+                    } else if let Ok(suffix) = path.strip_prefix(&absolute_source_path)
+                        // scripts are handled by rolldown
+                        && !path.starts_with(&absolute_source_scripts_path)
+                    {
+                        let dist_path = absolute_dist_path.join(suffix);
+                        if path.is_file() {
+                            let _ = fs::create_dir_all(path.parent().unwrap());
+                            let _ = fs::copy(&path, &dist_path);
+                        }
+                    }
+                }),
+                Err(e) => error!("watch error: {e:?}"),
+            },
+        )
+        .unwrap();
+
+        event_watcher
+            .watch(&absolute_websites_path_clone, RecursiveMode::Recursive)
+            .unwrap();
+        event_watcher
+            .watch(&absolute_modules_path_clone, RecursiveMode::Recursive)
+            .unwrap();
+
+        let watcher = rolldown::Watcher::new(vec![Arc::new(Mutex::new(bundler))], None).unwrap();
+
+        watcher.start().await;
+    });
+
+    Ok(relative_dist_path)
+}
+
+fn copy_dir_all(src: impl AsRef<Path>, dst: impl AsRef<Path>) -> io::Result<()> {
+    fs::create_dir_all(&dst)?;
+    for entry in fs::read_dir(src)? {
+        let entry = entry?;
+        let ty = entry.file_type()?;
+        if ty.is_dir() {
+            copy_dir_all(entry.path(), dst.as_ref().join(entry.file_name()))?;
+        } else {
+            fs::copy(entry.path(), dst.as_ref().join(entry.file_name()))?;
+        }
+    }
+    Ok(())
+}

crates/brk_cli/Cargo.toml

@@ -0,0 +1,41 @@
+[package]
+name = "brk_cli"
+description = "A command line interface to run a BRK instance"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+brk_binder = { workspace = true }
+brk_bundler = { workspace = true }
+brk_computer = { workspace = true }
+brk_error = { workspace = true }
+brk_fetcher = { workspace = true }
+brk_indexer = { workspace = true }
+brk_iterator = { workspace = true }
+brk_logger = { workspace = true }
+brk_mempool = { workspace = true }
+brk_query = { workspace = true }
+brk_reader = { workspace = true }
+brk_rpc = { workspace = true }
+brk_server = { workspace = true }
+clap = { version = "4.5.53", features = ["derive", "string"] }
+color-eyre = { workspace = true }
+log = { workspace = true }
+mimalloc = { workspace = true }
+minreq = { workspace = true }
+serde = { workspace = true }
+tokio = { workspace = true }
+toml = "0.9.10"
+vecdb = { workspace = true }
+zip = { version = "6.0.0", default-features = false, features = ["deflate"] }
+
+[[bin]]
+name = "brk"
+path = "src/main.rs"
+
+[package.metadata.dist]
+dist = false

crates/brk_cli/README.md

@@ -0,0 +1,46 @@
+# brk_cli
+
+Command-line interface for running the Bitcoin Research Kit.
+
+## What It Enables
+
+Run a full BRK instance: index the blockchain, compute metrics, serve the API, and optionally host a web interface. Continuously syncs with new blocks.
+
+## Key Features
+
+- **All-in-one**: Single binary runs indexer, computer, mempool monitor, and server
+- **Auto-sync**: Waits for new blocks and processes them automatically
+- **Web interface**: Downloads and bundles frontend from GitHub releases
+- **Configurable**: TOML config for RPC, paths, and features
+- **Collision checking**: Optional TXID collision validation mode
+- **Memory optimized**: Uses mimalloc allocator, 512MB stack for deep recursion
+
+## Usage
+
+```bash
+# See all options
+brk --help
+
+# The CLI will:
+# 1. Index new blocks
+# 2. Compute derived metrics
+# 3. Start mempool monitor
+# 4. Launch API server (port 3110)
+# 5. Wait for new blocks and repeat
+```
+
+## Components
+
+1. **Indexer**: Processes blocks into queryable indexes
+2. **Computer**: Derives 1000+ on-chain metrics
+3. **Mempool**: Real-time fee estimation
+4. **Server**: REST API + MCP endpoint
+5. **Bundler**: JS bundling for web interface (if enabled)
+
+## Built On
+
+- `brk_indexer` for blockchain indexing
+- `brk_computer` for metric computation
+- `brk_mempool` for mempool monitoring
+- `brk_server` for HTTP API
+- `brk_bundler` for web interface bundling

crates/brk_cli/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_cli/src/config.rs

@@ -0,0 +1,318 @@
+use std::{
+    fs,
+    path::{Path, PathBuf},
+};
+
+use brk_error::{Error, Result};
+use brk_fetcher::Fetcher;
+use brk_rpc::{Auth, Client};
+use clap::Parser;
+use serde::{Deserialize, Deserializer, Serialize};
+
+use crate::{default_brk_path, dot_brk_path, website::Website};
+
+const DOWNLOADS: &str = "downloads";
+
+#[derive(Parser, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)]
+#[command(version, about)]
+pub struct Config {
+    /// Bitcoin main directory path, defaults: ~/.bitcoin, ~/Library/Application\ Support/Bitcoin, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PATH")]
+    bitcoindir: Option<String>,
+
+    /// Bitcoin blocks directory path, default: --bitcoindir/blocks, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PATH")]
+    blocksdir: Option<String>,
+
+    /// Bitcoin Research Kit outputs directory path, default: ~/.brk, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PATH")]
+    brkdir: Option<String>,
+
+    /// Activate fetching prices from BRK's API and the computation of all price related datasets, default: true, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(short = 'F', long, value_name = "BOOL")]
+    fetch: Option<bool>,
+
+    /// Activate fetching prices from exchanges APIs if `fetch` is also set to `true`, default: true, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "BOOL")]
+    exchanges: Option<bool>,
+
+    /// Website served by the server, default: default, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(short, long)]
+    website: Option<Website>,
+
+    /// Bitcoin RPC ip, default: localhost, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "IP")]
+    rpcconnect: Option<String>,
+
+    /// Bitcoin RPC port, default: 8332, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PORT")]
+    rpcport: Option<u16>,
+
+    /// Bitcoin RPC cookie file, default: --bitcoindir/.cookie, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PATH")]
+    rpccookiefile: Option<String>,
+
+    /// Bitcoin RPC username, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "USERNAME")]
+    rpcuser: Option<String>,
+
+    /// Bitcoin RPC password, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "PASSWORD")]
+    rpcpassword: Option<String>,
+
+    /// DEV: Activate checking address hashes for collisions when indexing, default: false, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(skip)]
+    check_collisions: Option<bool>,
+}
+
+impl Config {
+    pub fn import() -> Result<Self> {
+        let config_args = Some(Config::parse());
+
+        let path = dot_brk_path();
+
+        let _ = fs::create_dir_all(&path);
+
+        let path = path.join("config.toml");
+
+        let mut config_saved = Self::read(&path);
+
+        if let Some(mut config_args) = config_args {
+            if let Some(bitcoindir) = config_args.bitcoindir.take() {
+                config_saved.bitcoindir = Some(bitcoindir);
+            }
+
+            if let Some(blocksdir) = config_args.blocksdir.take() {
+                config_saved.blocksdir = Some(blocksdir);
+            }
+
+            if let Some(brkdir) = config_args.brkdir.take() {
+                config_saved.brkdir = Some(brkdir);
+            }
+
+            if let Some(fetch) = config_args.fetch.take() {
+                config_saved.fetch = Some(fetch);
+            }
+
+            if let Some(exchanges) = config_args.exchanges.take() {
+                config_saved.exchanges = Some(exchanges);
+            }
+
+            if let Some(website) = config_args.website.take() {
+                config_saved.website = Some(website);
+            }
+
+            if let Some(rpcconnect) = config_args.rpcconnect.take() {
+                config_saved.rpcconnect = Some(rpcconnect);
+            }
+
+            if let Some(rpcport) = config_args.rpcport.take() {
+                config_saved.rpcport = Some(rpcport);
+            }
+
+            if let Some(rpccookiefile) = config_args.rpccookiefile.take() {
+                config_saved.rpccookiefile = Some(rpccookiefile);
+            }
+
+            if let Some(rpcuser) = config_args.rpcuser.take() {
+                config_saved.rpcuser = Some(rpcuser);
+            }
+
+            if let Some(rpcpassword) = config_args.rpcpassword.take() {
+                config_saved.rpcpassword = Some(rpcpassword);
+            }
+
+            if let Some(check_collisions) = config_args.check_collisions.take() {
+                config_saved.check_collisions = Some(check_collisions);
+            }
+
+            if config_args != Config::default() {
+                dbg!(config_args);
+                panic!("Didn't consume the full config")
+            }
+        }
+
+        let config = config_saved;
+
+        config.check();
+
+        config.write(&path)?;
+
+        Ok(config)
+    }
+
+    fn check(&self) {
+        if !self.bitcoindir().is_dir() {
+            println!("{:?} isn't a valid directory", self.bitcoindir());
+            println!("Please use the --bitcoindir parameter to set a valid path.");
+            println!("Run the program with '-h' for help.");
+            std::process::exit(1);
+        }
+
+        if !self.blocksdir().is_dir() {
+            println!("{:?} isn't a valid directory", self.blocksdir());
+            println!("Please use the --blocksdir parameter to set a valid path.");
+            println!("Run the program with '-h' for help.");
+            std::process::exit(1);
+        }
+
+        if !self.brkdir().is_dir() {
+            println!("{:?} isn't a valid directory", self.brkdir());
+            println!("Please use the --brkdir parameter to set a valid path.");
+            println!("Run the program with '-h' for help.");
+            std::process::exit(1);
+        }
+
+        if self.rpc_auth().is_err() {
+            println!(
+                "Unsuccessful authentication with the RPC client.
+First make sure that `bitcoind` is running. If it is then please either set --rpccookiefile or --rpcuser and --rpcpassword as the default values seemed to have failed.
+Finally, you can run the program with '-h' for help."
+            );
+            std::process::exit(1);
+        }
+    }
+
+    fn read(path: &Path) -> Self {
+        fs::read_to_string(path).map_or_else(
+            |_| Config::default(),
+            |contents| toml::from_str(&contents).unwrap_or_default(),
+        )
+    }
+
+    fn write(&self, path: &Path) -> std::io::Result<()> {
+        fs::write(path, toml::to_string(self).unwrap())
+    }
+
+    pub fn rpc(&self) -> Result<Client> {
+        Client::new(
+            &format!(
+                "http://{}:{}",
+                self.rpcconnect().unwrap_or(&"localhost".to_string()),
+                self.rpcport().unwrap_or(8332)
+            ),
+            self.rpc_auth()?,
+        )
+    }
+
+    fn rpc_auth(&self) -> Result<Auth> {
+        let cookie = self.path_cookiefile();
+
+        if cookie.is_file() {
+            Ok(Auth::CookieFile(cookie))
+        } else if self.rpcuser.is_some() && self.rpcpassword.is_some() {
+            Ok(Auth::UserPass(
+                self.rpcuser.clone().unwrap(),
+                self.rpcpassword.clone().unwrap(),
+            ))
+        } else {
+            Err(Error::AuthFailed)
+        }
+    }
+
+    fn rpcconnect(&self) -> Option<&String> {
+        self.rpcconnect.as_ref()
+    }
+
+    fn rpcport(&self) -> Option<u16> {
+        self.rpcport
+    }
+
+    pub fn bitcoindir(&self) -> PathBuf {
+        self.bitcoindir
+            .as_ref()
+            .map_or_else(Client::default_bitcoin_path, |s| {
+                Self::fix_user_path(s.as_ref())
+            })
+    }
+
+    pub fn blocksdir(&self) -> PathBuf {
+        self.blocksdir.as_ref().map_or_else(
+            || self.bitcoindir().join("blocks"),
+            |blocksdir| Self::fix_user_path(blocksdir.as_str()),
+        )
+    }
+
+    pub fn brkdir(&self) -> PathBuf {
+        self.brkdir
+            .as_ref()
+            .map_or_else(default_brk_path, |s| Self::fix_user_path(s.as_ref()))
+    }
+
+    pub fn harsdir(&self) -> PathBuf {
+        self.brkdir().join("hars")
+    }
+
+    pub fn downloads_dir(&self) -> PathBuf {
+        dot_brk_path().join(DOWNLOADS)
+    }
+
+    fn path_cookiefile(&self) -> PathBuf {
+        self.rpccookiefile.as_ref().map_or_else(
+            || self.bitcoindir().join(".cookie"),
+            |p| Self::fix_user_path(p.as_str()),
+        )
+    }
+
+    fn fix_user_path(path: &str) -> PathBuf {
+        let fix = move |pattern: &str| {
+            if path.starts_with(pattern) {
+                let path = &path
+                    .replace(&format!("{pattern}/"), "")
+                    .replace(pattern, "");
+
+                let home = std::env::var("HOME").unwrap();
+
+                Some(Path::new(&home).join(path))
+            } else {
+                None
+            }
+        };
+
+        fix("~").unwrap_or_else(|| fix("$HOME").unwrap_or_else(|| PathBuf::from(&path)))
+    }
+
+    pub fn website(&self) -> Website {
+        self.website.unwrap_or(Website::Bitview)
+    }
+
+    pub fn fetch(&self) -> bool {
+        self.fetch.is_none_or(|b| b)
+    }
+
+    pub fn exchanges(&self) -> bool {
+        self.exchanges.is_none_or(|b| b)
+    }
+
+    pub fn fetcher(&self) -> Option<Fetcher> {
+        self.fetch()
+            .then(|| Fetcher::import(self.exchanges(), Some(self.harsdir().as_path())).unwrap())
+    }
+
+    pub fn check_collisions(&self) -> bool {
+        self.check_collisions.is_some_and(|b| b)
+    }
+}
+
+fn default_on_error<'de, D, T>(deserializer: D) -> Result<T, D::Error>
+where
+    D: Deserializer<'de>,
+    T: Deserialize<'de> + Default,
+{
+    match T::deserialize(deserializer) {
+        Ok(v) => Ok(v),
+        Err(_) => Ok(T::default()),
+    }
+}

crates/brk_cli/src/main.rs

@@ -0,0 +1,166 @@
+#![doc = include_str!("../README.md")]
+
+use std::{
+    fs,
+    io::Cursor,
+    path::Path,
+    thread::{self, sleep},
+    time::Duration,
+};
+
+use brk_binder::generate_js_files;
+use brk_bundler::bundle;
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_mempool::Mempool;
+use brk_query::AsyncQuery;
+use brk_reader::Reader;
+use brk_server::{Server, VERSION};
+use log::info;
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+mod config;
+mod paths;
+mod website;
+
+use crate::{config::Config, paths::*};
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> color_eyre::Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+pub fn run() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    fs::create_dir_all(dot_brk_path())?;
+
+    brk_logger::init(Some(&dot_brk_log_path()))?;
+
+    let config = Config::import()?;
+
+    let client = config.rpc()?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let reader = Reader::new(config.blocksdir(), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&config.brkdir())?;
+
+    let mut computer = Computer::forced_import(&config.brkdir(), &indexer, config.fetcher())?;
+
+    let mempool = Mempool::new(&client);
+
+    let mempool_clone = mempool.clone();
+    thread::spawn(move || {
+        mempool_clone.start();
+    });
+
+    let query = AsyncQuery::build(&reader, &indexer, &computer, Some(mempool));
+
+    let website = config.website();
+
+    let downloads_path = config.downloads_dir();
+
+    let future = async move {
+        let bundle_path = if website.is_some() {
+            let websites_dev_path = Path::new("../../websites");
+            let modules_dev_path = Path::new("../../modules");
+
+            let websites_path;
+            let modules_path;
+
+            if fs::exists(websites_dev_path)? && fs::exists(modules_dev_path)? {
+                websites_path = websites_dev_path.to_path_buf();
+                modules_path = modules_dev_path.to_path_buf();
+            } else {
+                let downloaded_brk_path = downloads_path.join(format!("brk-{VERSION}"));
+
+                let downloaded_websites_path = downloaded_brk_path.join("websites");
+                let downloaded_modules_path = downloaded_brk_path.join("modules");
+
+                if !fs::exists(&downloaded_websites_path)? {
+                    info!("Downloading source from Github...");
+
+                    let url = format!(
+                        "https://github.com/bitcoinresearchkit/brk/archive/refs/tags/v{VERSION}.zip",
+                    );
+
+                    let response = minreq::get(url).send()?;
+                    let bytes = response.as_bytes();
+                    let cursor = Cursor::new(bytes);
+
+                    let mut zip = zip::ZipArchive::new(cursor).unwrap();
+
+                    zip.extract(downloads_path).unwrap();
+                }
+
+                websites_path = downloaded_websites_path;
+                modules_path = downloaded_modules_path;
+            }
+
+            generate_js_files(query.inner(), &modules_path)?;
+
+            Some(
+                bundle(
+                    &modules_path,
+                    &websites_path,
+                    website.to_folder_name(),
+                    true,
+                )
+                .await?,
+            )
+        } else {
+            None
+        };
+
+        let server = Server::new(&query, bundle_path);
+
+        tokio::spawn(async move {
+            server.serve(true).await.unwrap();
+        });
+
+        Ok(()) as Result<()>
+    };
+
+    let runtime = tokio::runtime::Builder::new_multi_thread()
+        .enable_all()
+        .build()?;
+
+    let _handle = runtime.spawn(future);
+
+    loop {
+        client.wait_for_synced_node()?;
+
+        let last_height = client.get_last_height()?;
+
+        info!("{} blocks found.", u32::from(last_height) + 1);
+
+        let starting_indexes = if config.check_collisions() {
+            indexer.checked_index(&blocks, &client, &exit)?
+        } else {
+            indexer.index(&blocks, &client, &exit)?
+        };
+
+        computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+
+        info!("Waiting for new blocks...");
+
+        while last_height == client.get_last_height()? {
+            sleep(Duration::from_secs(1))
+        }
+    }
+}

crates/brk_cli/src/paths.rs

@@ -0,0 +1,14 @@
+use std::path::{Path, PathBuf};
+
+pub fn dot_brk_path() -> PathBuf {
+    let home = std::env::var("HOME").unwrap();
+    Path::new(&home).join(".brk")
+}
+
+pub fn dot_brk_log_path() -> PathBuf {
+    dot_brk_path().join("log")
+}
+
+pub fn default_brk_path() -> PathBuf {
+    dot_brk_path()
+}

crates/brk_cli/src/website.rs

@@ -0,0 +1,28 @@
+use clap::ValueEnum;
+use serde::{Deserialize, Serialize};
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize, ValueEnum)]
+#[serde(rename_all = "lowercase")]
+pub enum Website {
+    None,
+    Bitview,
+    Custom,
+}
+
+impl Website {
+    pub fn is_none(&self) -> bool {
+        self == &Self::None
+    }
+
+    pub fn is_some(&self) -> bool {
+        !self.is_none()
+    }
+
+    pub fn to_folder_name(self) -> &'static str {
+        match self {
+            Self::Custom => "custom",
+            Self::Bitview => "bitview",
+            Self::None => unreachable!(),
+        }
+    }
+}

crates/brk_computer/.gitignore

@@ -0,0 +1 @@
+bottlenecks.md

crates/brk_computer/Cargo.toml

@@ -0,0 +1,36 @@
+[package]
+name = "brk_computer"
+description = "A Bitcoin dataset computer built on top of brk_indexer"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+bitcoin = { workspace = true }
+brk_bencher = { workspace = true }
+brk_error = { workspace = true }
+brk_fetcher = { workspace = true }
+brk_grouper = { workspace = true }
+brk_indexer = { workspace = true }
+brk_iterator = { workspace = true }
+brk_logger = { workspace = true }
+brk_reader = { workspace = true }
+brk_rpc = { workspace = true }
+brk_store = { workspace = true }
+brk_traversable = { workspace = true }
+brk_types = { workspace = true }
+derive_deref = { workspace = true }
+log = { workspace = true }
+pco = "0.4.7"
+rayon = { workspace = true }
+rustc-hash = { workspace = true }
+serde = { workspace = true }
+smallvec = { workspace = true }
+vecdb = { workspace = true }
+
+[dev-dependencies]
+color-eyre = { workspace = true }
+mimalloc = { workspace = true }

crates/brk_computer/README.md

@@ -0,0 +1,61 @@
+# brk_computer
+
+Derived metrics computation engine for Bitcoin on-chain analytics.
+
+## What It Enables
+
+Compute 1000+ on-chain metrics from indexed blockchain data: supply breakdowns, realized/unrealized P&L, SOPR, MVRV, cohort analysis (by age, amount, address type), cointime economics, mining pool attribution, and price-weighted valuations.
+
+## Key Features
+
+- **Cohort metrics**: Filter by UTXO age (STH/LTH, age bands), amount ranges, address types
+- **Stateful computation**: Track per-UTXO cost basis, realized/unrealized states
+- **Multi-index support**: Metrics available by height, date, week, month, year, decade
+- **Price integration**: USD-denominated metrics when price data available
+- **Mining pool attribution**: Tag blocks/rewards to known pools
+- **Cointime economics**: Liveliness, vaultedness, activity-weighted metrics
+- **Incremental updates**: Resume from checkpoints, compute only new blocks
+
+## Core API
+
+```rust,ignore
+let mut computer = Computer::forced_import(&outputs_path, &indexer, fetcher)?;
+
+// Compute all metrics for new blocks
+computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+
+// Access computed data
+let supply = computer.chain.height_to_supply.get(height)?;
+let realized_cap = computer.stateful.utxo.all.height_to_realized_cap.get(height)?;
+```
+
+## Metric Categories
+
+| Module | Examples |
+|--------|----------|
+| `chain` | Supply, subsidy, fees, transaction counts |
+| `stateful` | Realized cap, MVRV, SOPR, unrealized P&L |
+| `cointime` | Liveliness, vaultedness, true market mean |
+| `pools` | Per-pool block counts, rewards, fees |
+| `market` | Market cap, NVT, Puell multiple |
+| `price` | Height-to-price mapping from fetched data |
+
+## Cohort System
+
+UTXO and address cohorts support filtering by:
+- **Age**: STH (<150d), LTH (≥150d), age bands (1d, 1w, 1m, 3m, 6m, 1y, 2y, ...)
+- **Amount**: 0-0.001 BTC, 0.001-0.01, ..., 10k+ BTC
+- **Type**: P2PKH, P2SH, P2WPKH, P2WSH, P2TR
+- **Epoch**: By halving epoch
+
+## Recommended: mimalloc v3
+
+Use [mimalloc v3](https://crates.io/crates/mimalloc) as the global allocator to reduce memory usage.
+
+## Built On
+
+- `brk_indexer` for indexed blockchain data
+- `brk_fetcher` for price data
+- `brk_reader` for raw block access
+- `brk_grouper` for cohort filtering
+- `brk_traversable` for data export

crates/brk_computer/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_computer/examples/computer.rs

@@ -0,0 +1,68 @@
+use std::{
+    env,
+    path::Path,
+    thread::{self, sleep},
+    time::{Duration, Instant},
+};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_reader::Reader;
+use brk_rpc::{Auth, Client};
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()?;
+
+    Ok(())
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let bitcoin_dir = Client::default_bitcoin_path();
+    // let bitcoin_dir = Path::new("/Volumes/WD_BLACK/bitcoin");
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let client = Client::new(
+        Client::default_url(),
+        Auth::CookieFile(bitcoin_dir.join(".cookie")),
+    )?;
+
+    let reader = Reader::new(bitcoin_dir.join("blocks"), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let mut computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+    loop {
+        let i = Instant::now();
+        let starting_indexes = indexer.checked_index(&blocks, &client, &exit)?;
+        computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+        dbg!(i.elapsed());
+        sleep(Duration::from_secs(10));
+    }
+}

crates/brk_computer/examples/computer_bench.rs

@@ -0,0 +1,77 @@
+use std::{env, path::Path, thread, time::Instant};
+
+use brk_bencher::Bencher;
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_reader::Reader;
+use brk_rpc::{Auth, Client};
+use log::{debug, info};
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(None)?;
+
+    let bitcoin_dir = Client::default_bitcoin_path();
+    // let bitcoin_dir = Path::new("/Volumes/WD_BLACK/bitcoin");
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    let outputs_benches_dir = outputs_dir.join("benches");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let client = Client::new(
+        Client::default_url(),
+        Auth::CookieFile(bitcoin_dir.join(".cookie")),
+    )?;
+
+    let reader = Reader::new(bitcoin_dir.join("blocks"), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let mut computer = Computer::forced_import(&outputs_benches_dir, &indexer, Some(fetcher))?;
+
+    let mut bencher =
+        Bencher::from_cargo_env(env!("CARGO_PKG_NAME"), &outputs_dir.join("computed"))?;
+    bencher.start()?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+    let bencher_clone = bencher.clone();
+    exit.register_cleanup(move || {
+        let _ = bencher_clone.stop();
+        debug!("Bench stopped.");
+    });
+
+    let i = Instant::now();
+    let starting_indexes = indexer.index(&blocks, &client, &exit)?;
+    info!("Done in {:?}", i.elapsed());
+
+    let i = Instant::now();
+    computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+    info!("Done in {:?}", i.elapsed());
+
+    // We want to benchmark the drop too
+    drop(computer);
+    drop(indexer);
+
+    Ok(())
+}

crates/brk_computer/examples/computer_read.rs

@@ -0,0 +1,96 @@
+use std::{env, path::Path, thread};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_types::TxIndex;
+use mimalloc::MiMalloc;
+use vecdb::{Exit, GenericStoredVec};
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+    let txindex = TxIndex::new(134217893);
+
+    dbg!(
+        indexer
+            .vecs
+            .tx
+            .txindex_to_txid
+            .read_once(txindex)
+            .unwrap()
+            .to_string()
+    );
+    let first_txinindex = indexer
+        .vecs
+        .tx
+        .txindex_to_first_txinindex
+        .read_once(txindex)?;
+    dbg!(first_txinindex);
+    let first_txoutindex = indexer
+        .vecs
+        .tx
+        .txindex_to_first_txoutindex
+        .read_once(txindex)?;
+    dbg!(first_txoutindex);
+    let input_count = *computer.indexes.txindex_to_input_count.read_once(txindex)?;
+    dbg!(input_count);
+    let output_count = *computer
+        .indexes
+        .txindex_to_output_count
+        .read_once(txindex)?;
+    dbg!(output_count);
+
+    let _ = dbg!(computer.chain.txinindex_to_value.read_once(first_txinindex));
+    let _ = dbg!(
+        computer
+            .chain
+            .txinindex_to_value
+            .read_once(first_txinindex + 1)
+    );
+    let _ = dbg!(
+        indexer
+            .vecs
+            .txout
+            .txoutindex_to_value
+            .read_once(first_txoutindex)
+    );
+    let _ = dbg!(
+        indexer
+            .vecs
+            .txout
+            .txoutindex_to_value
+            .read_once(first_txoutindex + 1)
+    );
+    let _ = dbg!(computer.chain.txindex_to_input_value.read_once(txindex));
+    let _ = dbg!(computer.chain.txindex_to_input_value.read_once(txindex));
+    let _ = dbg!(computer.chain.txindex_to_output_value.read_once(txindex));
+    // dbg!(computer.indexes.txindex_to_txindex.ge(txindex));
+
+    Ok(())
+}

crates/brk_computer/examples/debug_indexer.rs

@@ -0,0 +1,141 @@
+use std::{env, path::Path};
+
+use brk_indexer::Indexer;
+use brk_types::{Height, P2PKHAddressIndex, P2SHAddressIndex, TxOutIndex, TypeIndex};
+use mimalloc::MiMalloc;
+use vecdb::GenericStoredVec;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+fn main() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+
+    let indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let reader_outputtype = indexer.vecs.txout.txoutindex_to_outputtype.create_reader();
+    let reader_typeindex = indexer.vecs.txout.txoutindex_to_typeindex.create_reader();
+    let reader_txindex = indexer.vecs.txout.txoutindex_to_txindex.create_reader();
+    let reader_txid = indexer.vecs.tx.txindex_to_txid.create_reader();
+    let reader_height_to_first_txoutindex = indexer
+        .vecs
+        .txout
+        .height_to_first_txoutindex
+        .create_reader();
+    let reader_p2pkh = indexer
+        .vecs
+        .address
+        .p2pkhaddressindex_to_p2pkhbytes
+        .create_reader();
+    let reader_p2sh = indexer
+        .vecs
+        .address
+        .p2shaddressindex_to_p2shbytes
+        .create_reader();
+
+    // Check what's stored at typeindex 254909199 in both P2PKH and P2SH vecs
+    let typeindex = TypeIndex::from(254909199_usize);
+
+    let p2pkh_bytes = indexer
+        .vecs
+        .address
+        .p2pkhaddressindex_to_p2pkhbytes
+        .read(P2PKHAddressIndex::from(typeindex), &reader_p2pkh);
+    println!("P2PKH at typeindex 254909199: {:?}", p2pkh_bytes);
+
+    let p2sh_bytes = indexer
+        .vecs
+        .address
+        .p2shaddressindex_to_p2shbytes
+        .read(P2SHAddressIndex::from(typeindex), &reader_p2sh);
+    println!("P2SH at typeindex 254909199: {:?}", p2sh_bytes);
+
+    // Check first P2SH index at height 476152
+    let reader_first_p2sh = indexer
+        .vecs
+        .address
+        .height_to_first_p2shaddressindex
+        .create_reader();
+    let reader_first_p2pkh = indexer
+        .vecs
+        .address
+        .height_to_first_p2pkhaddressindex
+        .create_reader();
+    let first_p2sh_at_476152 = indexer
+        .vecs
+        .address
+        .height_to_first_p2shaddressindex
+        .read(Height::from(476152_usize), &reader_first_p2sh);
+    let first_p2pkh_at_476152 = indexer
+        .vecs
+        .address
+        .height_to_first_p2pkhaddressindex
+        .read(Height::from(476152_usize), &reader_first_p2pkh);
+    println!(
+        "First P2SH index at height 476152: {:?}",
+        first_p2sh_at_476152
+    );
+    println!(
+        "First P2PKH index at height 476152: {:?}",
+        first_p2pkh_at_476152
+    );
+
+    // Check the problematic txoutindexes found during debugging
+    for txoutindex_usize in [653399433_usize, 653399443_usize] {
+        let txoutindex = TxOutIndex::from(txoutindex_usize);
+        let outputtype = indexer
+            .vecs
+            .txout
+            .txoutindex_to_outputtype
+            .read(txoutindex, &reader_outputtype)
+            .unwrap();
+        let typeindex = indexer
+            .vecs
+            .txout
+            .txoutindex_to_typeindex
+            .read(txoutindex, &reader_typeindex)
+            .unwrap();
+        let txindex = indexer
+            .vecs
+            .txout
+            .txoutindex_to_txindex
+            .read(txoutindex, &reader_txindex)
+            .unwrap();
+        let txid = indexer
+            .vecs
+            .tx
+            .txindex_to_txid
+            .read(txindex, &reader_txid)
+            .unwrap();
+
+        // Find height by binary search
+        let mut height = Height::from(0_usize);
+        for h in 0..900_000_usize {
+            let first_txoutindex = indexer
+                .vecs
+                .txout
+                .height_to_first_txoutindex
+                .read(Height::from(h), &reader_height_to_first_txoutindex);
+            if let Ok(first) = first_txoutindex {
+                if usize::from(first) > txoutindex_usize {
+                    break;
+                }
+                height = Height::from(h);
+            }
+        }
+
+        println!(
+            "txoutindex={}, outputtype={:?}, typeindex={}, txindex={}, txid={}, height={}",
+            txoutindex_usize,
+            outputtype,
+            usize::from(typeindex),
+            usize::from(txindex),
+            txid,
+            usize::from(height)
+        );
+    }
+
+    Ok(())
+}

crates/brk_computer/examples/pools.rs

@@ -0,0 +1,121 @@
+use std::{collections::BTreeMap, path::Path, thread};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_types::{Address, AddressBytes, OutputType, TxOutIndex, pools};
+use vecdb::{Exit, IterableVec, TypedVecIterator};
+
+fn main() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    thread::Builder::new()
+        .stack_size(256 * 1024 * 1024)
+        .spawn(move || -> Result<()> {
+            let outputs_dir = Path::new(&std::env::var("HOME").unwrap()).join(".brk");
+
+            let indexer = Indexer::forced_import(&outputs_dir)?;
+
+            let fetcher = Fetcher::import(true, None)?;
+
+            let computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+            let pools = pools();
+
+            let mut res: BTreeMap<&'static str, usize> = BTreeMap::default();
+
+            let vecs = indexer.vecs;
+            let stores = indexer.stores;
+
+            let mut height_to_first_txindex_iter = vecs.tx.height_to_first_txindex.iter()?;
+            let mut txindex_to_first_txoutindex_iter = vecs.tx.txindex_to_first_txoutindex.iter()?;
+            let mut txindex_to_output_count_iter = computer.indexes.txindex_to_output_count.iter();
+            let mut txoutindex_to_outputtype_iter = vecs.txout.txoutindex_to_outputtype.iter()?;
+            let mut txoutindex_to_typeindex_iter = vecs.txout.txoutindex_to_typeindex.iter()?;
+            let mut p2pk65addressindex_to_p2pk65bytes_iter =
+                vecs.address.p2pk65addressindex_to_p2pk65bytes.iter()?;
+            let mut p2pk33addressindex_to_p2pk33bytes_iter =
+                vecs.address.p2pk33addressindex_to_p2pk33bytes.iter()?;
+            let mut p2pkhaddressindex_to_p2pkhbytes_iter =
+                vecs.address.p2pkhaddressindex_to_p2pkhbytes.iter()?;
+            let mut p2shaddressindex_to_p2shbytes_iter =
+                vecs.address.p2shaddressindex_to_p2shbytes.iter()?;
+            let mut p2wpkhaddressindex_to_p2wpkhbytes_iter =
+                vecs.address.p2wpkhaddressindex_to_p2wpkhbytes.iter()?;
+            let mut p2wshaddressindex_to_p2wshbytes_iter =
+                vecs.address.p2wshaddressindex_to_p2wshbytes.iter()?;
+            let mut p2traddressindex_to_p2trbytes_iter =
+                vecs.address.p2traddressindex_to_p2trbytes.iter()?;
+            let mut p2aaddressindex_to_p2abytes_iter = vecs.address.p2aaddressindex_to_p2abytes.iter()?;
+
+            let unknown = pools.get_unknown();
+
+            stores
+                .height_to_coinbase_tag
+                .iter()
+                .for_each(|(height, coinbase_tag)| {
+                    let txindex = height_to_first_txindex_iter.get_unwrap(height);
+                    let txoutindex = txindex_to_first_txoutindex_iter.get_unwrap(txindex);
+                    let outputcount = txindex_to_output_count_iter.get_unwrap(txindex);
+
+                    let pool = (*txoutindex..(*txoutindex + *outputcount))
+                        .map(TxOutIndex::from)
+                        .find_map(|txoutindex| {
+                            let outputtype = txoutindex_to_outputtype_iter.get_unwrap(txoutindex);
+                            let typeindex = txoutindex_to_typeindex_iter.get_unwrap(txoutindex);
+
+                            match outputtype {
+                                OutputType::P2PK65 => Some(AddressBytes::from(
+                                    p2pk65addressindex_to_p2pk65bytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2PK33 => Some(AddressBytes::from(
+                                    p2pk33addressindex_to_p2pk33bytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2PKH => Some(AddressBytes::from(
+                                    p2pkhaddressindex_to_p2pkhbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2SH => Some(AddressBytes::from(
+                                    p2shaddressindex_to_p2shbytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2WPKH => Some(AddressBytes::from(
+                                    p2wpkhaddressindex_to_p2wpkhbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2WSH => Some(AddressBytes::from(
+                                    p2wshaddressindex_to_p2wshbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2TR => Some(AddressBytes::from(
+                                    p2traddressindex_to_p2trbytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2A => Some(AddressBytes::from(
+                                    p2aaddressindex_to_p2abytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                _ => None,
+                            }
+                            .map(|bytes| Address::try_from(&bytes).unwrap())
+                            .and_then(|address| pools.find_from_address(&address))
+                        })
+                        .or_else(|| pools.find_from_coinbase_tag(&coinbase_tag))
+                        .unwrap_or(unknown);
+
+                    *res.entry(pool.name).or_default() += 1;
+                });
+
+            let mut v = res.into_iter().map(|(k, v)| (v, k)).collect::<Vec<_>>();
+            v.sort_unstable();
+            println!("{:#?}", v);
+            println!("{:#?}", v.len());
+
+            Ok(())
+        })?
+        .join()
+        .unwrap()
+}

crates/brk_computer/flamegraph.sh

@@ -0,0 +1,2 @@
+cargo build --profile profiling
+flamegraph -- ../../target/profiling/examples/main

crates/brk_computer/samply.sh

@@ -0,0 +1,2 @@
+cargo build --example main --profile profiling
+samply record ../../target/profiling/examples/main

crates/brk_computer/src/blks.rs

@@ -0,0 +1,119 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_reader::Reader;
+use brk_traversable::Traversable;
+use brk_types::{BlkPosition, Height, TxIndex, Version};
+use vecdb::{
+    AnyStoredVec, AnyVec, Database, Exit, GenericStoredVec, ImportableVec, PAGE_SIZE, PcoVec,
+    TypedVecIterator,
+};
+
+use super::Indexes;
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub height_to_position: PcoVec<Height, BlkPosition>,
+    pub txindex_to_position: PcoVec<TxIndex, BlkPosition>,
+}
+
+impl Vecs {
+    pub fn forced_import(parent_path: &Path, parent_version: Version) -> Result<Self> {
+        let db = Database::open(&parent_path.join("blks"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let this = Self {
+            height_to_position: PcoVec::forced_import(&db, "position", version + Version::TWO)?,
+            txindex_to_position: PcoVec::forced_import(&db, "position", version + Version::TWO)?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: &Indexes,
+        reader: &Reader,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexer, starting_indexes, reader, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: &Indexes,
+        parser: &Reader,
+        exit: &Exit,
+    ) -> Result<()> {
+        let min_txindex =
+            TxIndex::from(self.txindex_to_position.len()).min(starting_indexes.txindex);
+
+        let Some(min_height) = indexer
+            .vecs
+            .tx.txindex_to_height
+            .iter()?
+            .get(min_txindex)
+            .map(|h| h.min(starting_indexes.height))
+        else {
+            return Ok(());
+        };
+
+        let mut height_to_first_txindex_iter = indexer.vecs.tx.height_to_first_txindex.iter()?;
+
+        parser
+            .read(
+                Some(min_height),
+                Some((indexer.vecs.tx.height_to_first_txindex.len() - 1).into()),
+            )
+            .iter()
+            .try_for_each(|block| -> Result<()> {
+                let height = block.height();
+
+                self.height_to_position
+                    .truncate_push(height, block.metadata().position())?;
+
+                let txindex = height_to_first_txindex_iter.get_unwrap(height);
+
+                block.tx_metadata().iter().enumerate().try_for_each(
+                    |(index, metadata)| -> Result<()> {
+                        let txindex = txindex + index;
+                        self.txindex_to_position
+                            .truncate_push(txindex, metadata.position())?;
+                        Ok(())
+                    },
+                )?;
+
+                if *height % 1_000 == 0 {
+                    let _lock = exit.lock();
+                    self.height_to_position.flush()?;
+                    self.txindex_to_position.flush()?;
+                }
+                Ok(())
+            })?;
+
+        let _lock = exit.lock();
+        self.height_to_position.flush()?;
+        self.txindex_to_position.flush()?;
+
+        Ok(())
+    }
+}

crates/brk_computer/src/chain/import.rs

@@ -0,0 +1,475 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{StoredBool, StoredU64, TxIndex, Version, Weight};
+use vecdb::{
+    Database, EagerVec, ImportableVec, IterableCloneableVec, LazyVecFrom1, LazyVecFrom2, PAGE_SIZE,
+    VecIndex,
+};
+
+use crate::{
+    grouped::{
+        ComputedValueVecsFromHeight, ComputedValueVecsFromTxindex, ComputedVecsFromDateIndex,
+        ComputedVecsFromHeight, ComputedVecsFromTxindex, Source, VecBuilderOptions,
+    },
+    indexes, price,
+};
+
+use super::{
+    Vecs, TARGET_BLOCKS_PER_DAY, TARGET_BLOCKS_PER_DECADE, TARGET_BLOCKS_PER_MONTH,
+    TARGET_BLOCKS_PER_QUARTER, TARGET_BLOCKS_PER_SEMESTER, TARGET_BLOCKS_PER_WEEK,
+    TARGET_BLOCKS_PER_YEAR,
+};
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("chain"))?;
+        db.set_min_len(PAGE_SIZE * 50_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let compute_dollars = price.is_some();
+        let v0 = Version::ZERO;
+        let v2 = Version::TWO;
+        let v4 = Version::new(4);
+        let v5 = Version::new(5);
+
+        // Helper macros for common patterns
+        macro_rules! eager {
+            ($name:expr) => {
+                EagerVec::forced_import(&db, $name, version + v0)?
+            };
+            ($name:expr, $v:expr) => {
+                EagerVec::forced_import(&db, $name, version + $v)?
+            };
+        }
+        macro_rules! computed_h {
+            ($name:expr, $source:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $source:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_di {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    version + $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_tx {
+            ($name:expr, $source:expr, $opts:expr) => {
+                ComputedVecsFromTxindex::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        let last = || VecBuilderOptions::default().add_last();
+        let sum = || VecBuilderOptions::default().add_sum();
+        let sum_cum = || VecBuilderOptions::default().add_sum().add_cumulative();
+        let stats = || {
+            VecBuilderOptions::default()
+                .add_average()
+                .add_minmax()
+                .add_percentiles()
+        };
+        let full_stats = || {
+            VecBuilderOptions::default()
+                .add_average()
+                .add_minmax()
+                .add_percentiles()
+                .add_sum()
+                .add_cumulative()
+        };
+
+        let txinindex_to_value = eager!("value");
+
+        let txindex_to_weight = LazyVecFrom2::init(
+            "weight",
+            version + Version::ZERO,
+            indexer.vecs.tx.txindex_to_base_size.boxed_clone(),
+            indexer.vecs.tx.txindex_to_total_size.boxed_clone(),
+            |index: TxIndex, txindex_to_base_size_iter, txindex_to_total_size_iter| {
+                let index = index.to_usize();
+                txindex_to_base_size_iter.get_at(index).map(|base_size| {
+                    let total_size = txindex_to_total_size_iter.get_at_unwrap(index);
+
+                    // This is the exact definition of a weight unit, as defined by BIP-141 (quote above).
+                    let wu = usize::from(base_size) * 3 + usize::from(total_size);
+
+                    Weight::from(bitcoin::Weight::from_wu_usize(wu))
+                })
+            },
+        );
+
+        let txindex_to_vsize = LazyVecFrom1::init(
+            "vsize",
+            version + Version::ZERO,
+            txindex_to_weight.boxed_clone(),
+            |index: TxIndex, iter| iter.get(index).map(brk_types::VSize::from),
+        );
+
+        let txindex_to_is_coinbase = LazyVecFrom2::init(
+            "is_coinbase",
+            version + Version::ZERO,
+            indexer.vecs.tx.txindex_to_height.boxed_clone(),
+            indexer.vecs.tx.height_to_first_txindex.boxed_clone(),
+            |index: TxIndex, txindex_to_height_iter, height_to_first_txindex_iter| {
+                txindex_to_height_iter.get(index).map(|height| {
+                    let txindex = height_to_first_txindex_iter.get_unwrap(height);
+                    StoredBool::from(index == txindex)
+                })
+            },
+        );
+
+        let txindex_to_input_value = eager!("input_value");
+        let txindex_to_output_value = eager!("output_value");
+        let txindex_to_fee = eager!("fee");
+        let txindex_to_fee_rate = eager!("fee_rate");
+
+        let dateindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.dateindex_to_dateindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_DAY)),
+        );
+        let weekindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.weekindex_to_weekindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_WEEK)),
+        );
+        let monthindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.monthindex_to_monthindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_MONTH)),
+        );
+        let quarterindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.quarterindex_to_quarterindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_QUARTER)),
+        );
+        let semesterindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.semesterindex_to_semesterindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_SEMESTER)),
+        );
+        let yearindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.yearindex_to_yearindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_YEAR)),
+        );
+        let decadeindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.decadeindex_to_decadeindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_DECADE)),
+        );
+
+        let this = Self {
+            dateindex_to_block_count_target,
+            weekindex_to_block_count_target,
+            monthindex_to_block_count_target,
+            quarterindex_to_block_count_target,
+            semesterindex_to_block_count_target,
+            yearindex_to_block_count_target,
+            decadeindex_to_block_count_target,
+            height_to_interval: eager!("interval"),
+            timeindexes_to_timestamp: computed_di!(
+                "timestamp",
+                VecBuilderOptions::default().add_first()
+            ),
+            indexes_to_block_interval: computed_h!("block_interval", Source::None, stats()),
+            indexes_to_block_count: computed_h!("block_count", Source::Compute, sum_cum()),
+            indexes_to_1w_block_count: computed_di!("1w_block_count", last()),
+            indexes_to_1m_block_count: computed_di!("1m_block_count", last()),
+            indexes_to_1y_block_count: computed_di!("1y_block_count", last()),
+            indexes_to_block_weight: computed_h!("block_weight", Source::None, full_stats()),
+            indexes_to_block_size: computed_h!("block_size", Source::None, full_stats()),
+            height_to_vbytes: eager!("vbytes"),
+            height_to_24h_block_count: eager!("24h_block_count"),
+            height_to_24h_coinbase_sum: eager!("24h_coinbase_sum"),
+            height_to_24h_coinbase_usd_sum: eager!("24h_coinbase_usd_sum"),
+            indexes_to_block_vbytes: computed_h!("block_vbytes", Source::None, full_stats()),
+            difficultyepoch_to_timestamp: eager!("timestamp"),
+            halvingepoch_to_timestamp: eager!("timestamp"),
+
+            dateindex_to_fee_dominance: eager!("fee_dominance"),
+            dateindex_to_subsidy_dominance: eager!("subsidy_dominance"),
+            indexes_to_difficulty: computed_h!("difficulty", Source::None, last()),
+            indexes_to_difficultyepoch: computed_di!("difficultyepoch", last()),
+            indexes_to_halvingepoch: computed_di!("halvingepoch", last()),
+            indexes_to_tx_count: computed_h!("tx_count", Source::Compute, full_stats()),
+            indexes_to_input_count: computed_tx!("input_count", Source::None, full_stats()),
+            indexes_to_output_count: computed_tx!("output_count", Source::None, full_stats()),
+            indexes_to_tx_v1: computed_h!("tx_v1", Source::Compute, sum_cum()),
+            indexes_to_tx_v2: computed_h!("tx_v2", Source::Compute, sum_cum()),
+            indexes_to_tx_v3: computed_h!("tx_v3", Source::Compute, sum_cum()),
+            indexes_to_sent_sum: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "sent_sum",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default().add_sum(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_fee: ComputedValueVecsFromTxindex::forced_import(
+                &db,
+                "fee",
+                indexer,
+                indexes,
+                Source::Vec(txindex_to_fee.boxed_clone()),
+                version + Version::ZERO,
+                price,
+                VecBuilderOptions::default()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_percentiles()
+                    .add_minmax()
+                    .add_average(),
+            )?,
+            indexes_to_fee_rate: computed_tx!("fee_rate", Source::None, stats()),
+            indexes_to_tx_vsize: computed_tx!("tx_vsize", Source::None, stats()),
+            indexes_to_tx_weight: computed_tx!("tx_weight", Source::None, stats()),
+            indexes_to_subsidy: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "subsidy",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default()
+                    .add_percentiles()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_minmax()
+                    .add_average(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_coinbase: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "coinbase",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_percentiles()
+                    .add_minmax()
+                    .add_average(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_unclaimed_rewards: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "unclaimed_rewards",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default().add_sum().add_cumulative(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_p2a_count: computed_h!("p2a_count", Source::Compute, full_stats()),
+            indexes_to_p2ms_count: computed_h!("p2ms_count", Source::Compute, full_stats()),
+            indexes_to_p2pk33_count: computed_h!("p2pk33_count", Source::Compute, full_stats()),
+            indexes_to_p2pk65_count: computed_h!("p2pk65_count", Source::Compute, full_stats()),
+            indexes_to_p2pkh_count: computed_h!("p2pkh_count", Source::Compute, full_stats()),
+            indexes_to_p2sh_count: computed_h!("p2sh_count", Source::Compute, full_stats()),
+            indexes_to_p2tr_count: computed_h!("p2tr_count", Source::Compute, full_stats()),
+            indexes_to_p2wpkh_count: computed_h!("p2wpkh_count", Source::Compute, full_stats()),
+            indexes_to_p2wsh_count: computed_h!("p2wsh_count", Source::Compute, full_stats()),
+            indexes_to_opreturn_count: computed_h!("opreturn_count", Source::Compute, full_stats()),
+            indexes_to_unknownoutput_count: computed_h!(
+                "unknownoutput_count",
+                Source::Compute,
+                full_stats()
+            ),
+            indexes_to_emptyoutput_count: computed_h!(
+                "emptyoutput_count",
+                Source::Compute,
+                full_stats()
+            ),
+            indexes_to_exact_utxo_count: computed_h!("exact_utxo_count", Source::Compute, last()),
+            indexes_to_subsidy_usd_1y_sma: compute_dollars
+                .then(|| {
+                    ComputedVecsFromDateIndex::forced_import(
+                        &db,
+                        "subsidy_usd_1y_sma",
+                        Source::Compute,
+                        version + v0,
+                        indexes,
+                        last(),
+                    )
+                })
+                .transpose()?,
+            indexes_to_puell_multiple: compute_dollars
+                .then(|| {
+                    ComputedVecsFromDateIndex::forced_import(
+                        &db,
+                        "puell_multiple",
+                        Source::Compute,
+                        version + v0,
+                        indexes,
+                        last(),
+                    )
+                })
+                .transpose()?,
+            indexes_to_hash_rate: computed_h!("hash_rate", Source::Compute, v5, last()),
+            indexes_to_hash_rate_1w_sma: computed_di!("hash_rate_1w_sma", last()),
+            indexes_to_hash_rate_1m_sma: computed_di!("hash_rate_1m_sma", last()),
+            indexes_to_hash_rate_2m_sma: computed_di!("hash_rate_2m_sma", last()),
+            indexes_to_hash_rate_1y_sma: computed_di!("hash_rate_1y_sma", last()),
+            indexes_to_difficulty_as_hash: computed_h!(
+                "difficulty_as_hash",
+                Source::Compute,
+                last()
+            ),
+            indexes_to_difficulty_adjustment: computed_h!(
+                "difficulty_adjustment",
+                Source::Compute,
+                sum()
+            ),
+            indexes_to_blocks_before_next_difficulty_adjustment: computed_h!(
+                "blocks_before_next_difficulty_adjustment",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_days_before_next_difficulty_adjustment: computed_h!(
+                "days_before_next_difficulty_adjustment",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_blocks_before_next_halving: computed_h!(
+                "blocks_before_next_halving",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_days_before_next_halving: computed_h!(
+                "days_before_next_halving",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_hash_price_ths: computed_h!("hash_price_ths", Source::Compute, v4, last()),
+            indexes_to_hash_price_phs: computed_h!("hash_price_phs", Source::Compute, v4, last()),
+            indexes_to_hash_value_ths: computed_h!("hash_value_ths", Source::Compute, v4, last()),
+            indexes_to_hash_value_phs: computed_h!("hash_value_phs", Source::Compute, v4, last()),
+            indexes_to_hash_price_ths_min: computed_h!(
+                "hash_price_ths_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_price_phs_min: computed_h!(
+                "hash_price_phs_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_price_rebound: computed_h!(
+                "hash_price_rebound",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_ths_min: computed_h!(
+                "hash_value_ths_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_phs_min: computed_h!(
+                "hash_value_phs_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_rebound: computed_h!(
+                "hash_value_rebound",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_inflation_rate: computed_di!("inflation_rate", last()),
+            indexes_to_annualized_volume: computed_di!("annualized_volume", last()),
+            indexes_to_annualized_volume_btc: computed_di!("annualized_volume_btc", last()),
+            indexes_to_annualized_volume_usd: computed_di!("annualized_volume_usd", last()),
+            indexes_to_tx_btc_velocity: computed_di!("tx_btc_velocity", last()),
+            indexes_to_tx_usd_velocity: computed_di!("tx_usd_velocity", last()),
+            indexes_to_tx_per_sec: computed_di!("tx_per_sec", v2, last()),
+            indexes_to_outputs_per_sec: computed_di!("outputs_per_sec", v2, last()),
+            indexes_to_inputs_per_sec: computed_di!("inputs_per_sec", v2, last()),
+
+            txindex_to_is_coinbase,
+            txinindex_to_value,
+            txindex_to_input_value,
+            txindex_to_output_value,
+            txindex_to_fee,
+            txindex_to_fee_rate,
+            txindex_to_vsize,
+            txindex_to_weight,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+}

crates/brk_computer/src/chain/mod.rs

@@ -0,0 +1,132 @@
+mod compute;
+mod import;
+
+use brk_traversable::Traversable;
+use brk_types::{
+    Bitcoin, DateIndex, DecadeIndex, DifficultyEpoch, Dollars, FeeRate, HalvingEpoch, Height,
+    MonthIndex, QuarterIndex, Sats, SemesterIndex, StoredBool, StoredF32, StoredF64, StoredU32,
+    StoredU64, Timestamp, TxInIndex, TxIndex, VSize, WeekIndex, Weight, YearIndex,
+};
+use vecdb::{Database, EagerVec, LazyVecFrom1, LazyVecFrom2, PcoVec};
+
+use crate::grouped::{
+    ComputedValueVecsFromHeight, ComputedValueVecsFromTxindex, ComputedVecsFromDateIndex,
+    ComputedVecsFromHeight, ComputedVecsFromTxindex,
+};
+
+pub(crate) const TARGET_BLOCKS_PER_DAY_F64: f64 = 144.0;
+pub(crate) const TARGET_BLOCKS_PER_DAY_F32: f32 = 144.0;
+pub(crate) const TARGET_BLOCKS_PER_DAY: u64 = 144;
+pub(crate) const TARGET_BLOCKS_PER_WEEK: u64 = 7 * TARGET_BLOCKS_PER_DAY;
+pub(crate) const TARGET_BLOCKS_PER_MONTH: u64 = 30 * TARGET_BLOCKS_PER_DAY;
+pub(crate) const TARGET_BLOCKS_PER_QUARTER: u64 = 3 * TARGET_BLOCKS_PER_MONTH;
+pub(crate) const TARGET_BLOCKS_PER_SEMESTER: u64 = 2 * TARGET_BLOCKS_PER_QUARTER;
+pub(crate) const TARGET_BLOCKS_PER_YEAR: u64 = 2 * TARGET_BLOCKS_PER_SEMESTER;
+pub(crate) const TARGET_BLOCKS_PER_DECADE: u64 = 10 * TARGET_BLOCKS_PER_YEAR;
+pub(crate) const ONE_TERA_HASH: f64 = 1_000_000_000_000.0;
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    pub(crate) db: Database,
+
+    pub dateindex_to_block_count_target: LazyVecFrom1<DateIndex, StoredU64, DateIndex, DateIndex>,
+    pub weekindex_to_block_count_target: LazyVecFrom1<WeekIndex, StoredU64, WeekIndex, WeekIndex>,
+    pub monthindex_to_block_count_target:
+        LazyVecFrom1<MonthIndex, StoredU64, MonthIndex, MonthIndex>,
+    pub quarterindex_to_block_count_target:
+        LazyVecFrom1<QuarterIndex, StoredU64, QuarterIndex, QuarterIndex>,
+    pub semesterindex_to_block_count_target:
+        LazyVecFrom1<SemesterIndex, StoredU64, SemesterIndex, SemesterIndex>,
+    pub yearindex_to_block_count_target: LazyVecFrom1<YearIndex, StoredU64, YearIndex, YearIndex>,
+    pub decadeindex_to_block_count_target:
+        LazyVecFrom1<DecadeIndex, StoredU64, DecadeIndex, DecadeIndex>,
+    pub height_to_interval: EagerVec<PcoVec<Height, Timestamp>>,
+    pub height_to_24h_block_count: EagerVec<PcoVec<Height, StoredU32>>,
+    pub height_to_24h_coinbase_sum: EagerVec<PcoVec<Height, Sats>>,
+    pub height_to_24h_coinbase_usd_sum: EagerVec<PcoVec<Height, Dollars>>,
+    pub height_to_vbytes: EagerVec<PcoVec<Height, StoredU64>>,
+    pub difficultyepoch_to_timestamp: EagerVec<PcoVec<DifficultyEpoch, Timestamp>>,
+    pub halvingepoch_to_timestamp: EagerVec<PcoVec<HalvingEpoch, Timestamp>>,
+    pub timeindexes_to_timestamp: ComputedVecsFromDateIndex<Timestamp>,
+    pub indexes_to_block_count: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_1w_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_1m_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_1y_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_block_interval: ComputedVecsFromHeight<Timestamp>,
+    pub indexes_to_block_size: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_block_vbytes: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_block_weight: ComputedVecsFromHeight<Weight>,
+    pub indexes_to_difficulty: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_difficultyepoch: ComputedVecsFromDateIndex<DifficultyEpoch>,
+    pub indexes_to_halvingepoch: ComputedVecsFromDateIndex<HalvingEpoch>,
+    pub indexes_to_coinbase: ComputedValueVecsFromHeight,
+    pub indexes_to_emptyoutput_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_fee: ComputedValueVecsFromTxindex,
+    pub indexes_to_fee_rate: ComputedVecsFromTxindex<FeeRate>,
+    /// Value == 0 when Coinbase
+    pub txindex_to_input_value: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub indexes_to_sent_sum: ComputedValueVecsFromHeight,
+    pub indexes_to_opreturn_count: ComputedVecsFromHeight<StoredU64>,
+    pub txindex_to_output_value: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub indexes_to_p2a_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2ms_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pk33_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pk65_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pkh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2sh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2tr_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2wpkh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2wsh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_subsidy: ComputedValueVecsFromHeight,
+    pub indexes_to_unclaimed_rewards: ComputedValueVecsFromHeight,
+    pub indexes_to_tx_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v1: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v2: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v3: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_vsize: ComputedVecsFromTxindex<VSize>,
+    pub indexes_to_tx_weight: ComputedVecsFromTxindex<Weight>,
+    pub indexes_to_unknownoutput_count: ComputedVecsFromHeight<StoredU64>,
+    pub txinindex_to_value: EagerVec<PcoVec<TxInIndex, Sats>>,
+    pub indexes_to_input_count: ComputedVecsFromTxindex<StoredU64>,
+    pub txindex_to_is_coinbase: LazyVecFrom2<TxIndex, StoredBool, TxIndex, Height, Height, TxIndex>,
+    pub indexes_to_output_count: ComputedVecsFromTxindex<StoredU64>,
+    pub txindex_to_vsize: LazyVecFrom1<TxIndex, VSize, TxIndex, Weight>,
+    pub txindex_to_weight: LazyVecFrom2<TxIndex, Weight, TxIndex, StoredU32, TxIndex, StoredU32>,
+    pub txindex_to_fee: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub txindex_to_fee_rate: EagerVec<PcoVec<TxIndex, FeeRate>>,
+    pub indexes_to_exact_utxo_count: ComputedVecsFromHeight<StoredU64>,
+    pub dateindex_to_fee_dominance: EagerVec<PcoVec<DateIndex, StoredF32>>,
+    pub dateindex_to_subsidy_dominance: EagerVec<PcoVec<DateIndex, StoredF32>>,
+    pub indexes_to_subsidy_usd_1y_sma: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub indexes_to_puell_multiple: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub indexes_to_hash_rate: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_hash_rate_1w_sma: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_hash_rate_1m_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_rate_2m_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_rate_1y_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_price_ths: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_ths_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_phs: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_phs_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_rebound: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_ths: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_ths_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_phs: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_phs_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_rebound: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_difficulty_as_hash: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_difficulty_adjustment: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_blocks_before_next_difficulty_adjustment: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_days_before_next_difficulty_adjustment: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_blocks_before_next_halving: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_days_before_next_halving: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_inflation_rate: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_annualized_volume: ComputedVecsFromDateIndex<Sats>,
+    pub indexes_to_annualized_volume_btc: ComputedVecsFromDateIndex<Bitcoin>,
+    pub indexes_to_annualized_volume_usd: ComputedVecsFromDateIndex<Dollars>,
+    pub indexes_to_tx_btc_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_tx_usd_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_tx_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_outputs_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_inputs_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+}

crates/brk_computer/src/cointime.rs

@@ -0,0 +1,577 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, CheckedSub, Dollars, StoredF32, StoredF64, Version};
+use vecdb::{Database, Exit, PAGE_SIZE, TypedVecIterator};
+
+use crate::{grouped::ComputedVecsFromDateIndex, utils::OptionExt};
+
+use super::{
+    Indexes, chain,
+    grouped::{
+        ComputedRatioVecsFromDateIndex, ComputedValueVecsFromHeight, ComputedVecsFromHeight,
+        Source, VecBuilderOptions,
+    },
+    indexes, price, stateful,
+};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub indexes_to_coinblocks_created: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_coinblocks_stored: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_liveliness: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_vaultedness: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_activity_to_vaultedness_ratio: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_vaulted_supply: ComputedValueVecsFromHeight,
+    pub indexes_to_active_supply: ComputedValueVecsFromHeight,
+    pub indexes_to_thermo_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_investor_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_active_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_active_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_active_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_true_market_mean: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_true_market_mean_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_cointime_value_destroyed: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_value_created: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_value_stored: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_cointime_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_cointime_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_cointime_adj_inflation_rate: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_cointime_adj_tx_btc_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_cointime_adj_tx_usd_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    // pub indexes_to_thermo_cap_rel_to_investor_cap: ComputedValueVecsFromHeight,
+}
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("cointime"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let compute_dollars = price.is_some();
+        let v0 = parent_version;
+        let v1 = parent_version + Version::ONE;
+
+        let last = || VecBuilderOptions::default().add_last();
+        let sum_cum = || VecBuilderOptions::default().add_sum().add_cumulative();
+
+        macro_rules! computed_h {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_di {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! ratio_di {
+            ($name:expr) => {
+                ComputedRatioVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::None,
+                    v0,
+                    indexes,
+                    true,
+                )?
+            };
+        }
+        macro_rules! value_h {
+            ($name:expr) => {
+                ComputedValueVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v1,
+                    last(),
+                    compute_dollars,
+                    indexes,
+                )?
+            };
+        }
+
+        let this = Self {
+            indexes_to_coinblocks_created: computed_h!("coinblocks_created", sum_cum()),
+            indexes_to_coinblocks_stored: computed_h!("coinblocks_stored", sum_cum()),
+            indexes_to_liveliness: computed_h!("liveliness", last()),
+            indexes_to_vaultedness: computed_h!("vaultedness", last()),
+            indexes_to_activity_to_vaultedness_ratio: computed_h!(
+                "activity_to_vaultedness_ratio",
+                last()
+            ),
+            indexes_to_vaulted_supply: value_h!("vaulted_supply"),
+            indexes_to_active_supply: value_h!("active_supply"),
+            indexes_to_thermo_cap: computed_h!("thermo_cap", v1, last()),
+            indexes_to_investor_cap: computed_h!("investor_cap", v1, last()),
+            indexes_to_vaulted_cap: computed_h!("vaulted_cap", v1, last()),
+            indexes_to_active_cap: computed_h!("active_cap", v1, last()),
+            indexes_to_vaulted_price: computed_h!("vaulted_price", last()),
+            indexes_to_vaulted_price_ratio: ratio_di!("vaulted_price"),
+            indexes_to_active_price: computed_h!("active_price", last()),
+            indexes_to_active_price_ratio: ratio_di!("active_price"),
+            indexes_to_true_market_mean: computed_h!("true_market_mean", last()),
+            indexes_to_true_market_mean_ratio: ratio_di!("true_market_mean"),
+            indexes_to_cointime_value_destroyed: computed_h!("cointime_value_destroyed", sum_cum()),
+            indexes_to_cointime_value_created: computed_h!("cointime_value_created", sum_cum()),
+            indexes_to_cointime_value_stored: computed_h!("cointime_value_stored", sum_cum()),
+            indexes_to_cointime_price: computed_h!("cointime_price", last()),
+            indexes_to_cointime_cap: computed_h!("cointime_cap", last()),
+            indexes_to_cointime_price_ratio: ratio_di!("cointime_price"),
+            indexes_to_cointime_adj_inflation_rate: computed_di!(
+                "cointime_adj_inflation_rate",
+                last()
+            ),
+            indexes_to_cointime_adj_tx_btc_velocity: computed_di!(
+                "cointime_adj_tx_btc_velocity",
+                last()
+            ),
+            indexes_to_cointime_adj_tx_usd_velocity: computed_di!(
+                "cointime_adj_tx_usd_velocity",
+                last()
+            ),
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    #[allow(clippy::too_many_arguments)]
+    pub fn compute(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        price: Option<&price::Vecs>,
+        chain: &chain::Vecs,
+        stateful: &stateful::Vecs,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexes, starting_indexes, price, chain, stateful, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    #[allow(clippy::too_many_arguments)]
+    fn compute_(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        price: Option<&price::Vecs>,
+        chain: &chain::Vecs,
+        stateful: &stateful::Vecs,
+        exit: &Exit,
+    ) -> Result<()> {
+        let circulating_supply = &stateful.utxo_cohorts.all.metrics.supply.height_to_supply;
+
+        self.indexes_to_coinblocks_created
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_transform(
+                    starting_indexes.height,
+                    circulating_supply,
+                    |(i, v, ..)| (i, StoredF64::from(Bitcoin::from(v))),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        let indexes_to_coinblocks_destroyed = &stateful
+            .utxo_cohorts
+            .all
+            .metrics
+            .activity
+            .indexes_to_coinblocks_destroyed;
+
+        self.indexes_to_coinblocks_stored
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                let mut coinblocks_destroyed_iter = indexes_to_coinblocks_destroyed
+                    .height
+                    .as_ref()
+                    .unwrap()
+                    .into_iter();
+                vec.compute_transform(
+                    starting_indexes.height,
+                    self.indexes_to_coinblocks_created.height.u(),
+                    |(i, created, ..)| {
+                        let destroyed = coinblocks_destroyed_iter.get_unwrap(i);
+                        (i, created.checked_sub(destroyed).unwrap())
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_liveliness
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_divide(
+                    starting_indexes.height,
+                    indexes_to_coinblocks_destroyed
+                        .height_extra
+                        .unwrap_cumulative(),
+                    self.indexes_to_coinblocks_created
+                        .height_extra
+                        .unwrap_cumulative(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        let liveliness = &self.indexes_to_liveliness;
+
+        self.indexes_to_vaultedness
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_transform(
+                    starting_indexes.height,
+                    liveliness.height.u(),
+                    |(i, v, ..)| (i, StoredF64::from(1.0).checked_sub(v).unwrap()),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        let vaultedness = &self.indexes_to_vaultedness;
+
+        self.indexes_to_activity_to_vaultedness_ratio.compute_all(
+            indexes,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_divide(
+                    starting_indexes.height,
+                    liveliness.height.u(),
+                    vaultedness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_vaulted_supply.compute_all(
+            indexes,
+            price,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_multiply(
+                    starting_indexes.height,
+                    circulating_supply,
+                    vaultedness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_active_supply.compute_all(
+            indexes,
+            price,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_multiply(
+                    starting_indexes.height,
+                    circulating_supply,
+                    liveliness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_cointime_adj_inflation_rate
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_multiply(
+                    starting_indexes.dateindex,
+                    self.indexes_to_activity_to_vaultedness_ratio
+                        .dateindex
+                        .unwrap_last(),
+                    chain.indexes_to_inflation_rate.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_cointime_adj_tx_btc_velocity
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_multiply(
+                    starting_indexes.dateindex,
+                    self.indexes_to_activity_to_vaultedness_ratio
+                        .dateindex
+                        .unwrap_last(),
+                    chain.indexes_to_tx_btc_velocity.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        if let Some(price) = price {
+            let realized_cap = &stateful
+                .utxo_cohorts
+                .all
+                .metrics
+                .realized
+                .u()
+                .height_to_realized_cap;
+            let realized_price = stateful
+                .utxo_cohorts
+                .all
+                .metrics
+                .realized
+                .u()
+                .indexes_to_realized_price
+                .height
+                .u();
+
+            self.indexes_to_thermo_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_transform(
+                        starting_indexes.height,
+                        chain
+                            .indexes_to_subsidy
+                            .dollars
+                            .as_ref()
+                            .unwrap()
+                            .height_extra
+                            .unwrap_cumulative(),
+                        |(i, v, ..)| (i, v),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_investor_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_subtract(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_thermo_cap.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_vaultedness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_active_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_liveliness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        realized_price,
+                        self.indexes_to_vaultedness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_vaulted_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_active_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        realized_price,
+                        self.indexes_to_liveliness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_active_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_active_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_true_market_mean.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        self.indexes_to_investor_cap.height.u(),
+                        self.indexes_to_active_supply
+                            .bitcoin
+                            .height
+                            .as_ref()
+                            .unwrap(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_true_market_mean_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_true_market_mean.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_cointime_value_destroyed.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    // TODO: Another example when the callback should be applied to each index, instead of to base then merging from more granular to less
+                    // The price taken won't be correct for time based indexes
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        indexes_to_coinblocks_destroyed.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_value_created.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        self.indexes_to_coinblocks_created.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_value_stored.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        self.indexes_to_coinblocks_stored.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        self.indexes_to_cointime_value_destroyed
+                            .height_extra
+                            .unwrap_cumulative(),
+                        self.indexes_to_coinblocks_stored
+                            .height_extra
+                            .unwrap_cumulative(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_cointime_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        self.indexes_to_cointime_price.height.u(),
+                        circulating_supply,
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_cointime_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_cointime_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_cointime_adj_tx_usd_velocity.compute_all(
+                starting_indexes,
+                exit,
+                |v| {
+                    v.compute_multiply(
+                        starting_indexes.dateindex,
+                        self.indexes_to_activity_to_vaultedness_ratio
+                            .dateindex
+                            .unwrap_last(),
+                        chain.indexes_to_tx_usd_velocity.dateindex.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/constants.rs

@@ -0,0 +1,256 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{StoredF32, StoredI16, StoredU16, Version};
+use vecdb::{AnyVec, Database, Exit, PAGE_SIZE};
+
+use crate::grouped::Source;
+
+use super::{
+    Indexes,
+    grouped::{ComputedVecsFromHeight, VecBuilderOptions},
+    indexes,
+};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub constant_0: ComputedVecsFromHeight<StoredU16>,
+    pub constant_1: ComputedVecsFromHeight<StoredU16>,
+    pub constant_2: ComputedVecsFromHeight<StoredU16>,
+    pub constant_3: ComputedVecsFromHeight<StoredU16>,
+    pub constant_4: ComputedVecsFromHeight<StoredU16>,
+    pub constant_38_2: ComputedVecsFromHeight<StoredF32>,
+    pub constant_50: ComputedVecsFromHeight<StoredU16>,
+    pub constant_61_8: ComputedVecsFromHeight<StoredF32>,
+    pub constant_100: ComputedVecsFromHeight<StoredU16>,
+    pub constant_600: ComputedVecsFromHeight<StoredU16>,
+    pub constant_minus_1: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_2: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_3: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_4: ComputedVecsFromHeight<StoredI16>,
+}
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexes: &indexes::Vecs,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("constants"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let this = Self {
+            constant_0: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_0",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_1: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_1",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_3: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_3",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_4: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_4",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_38_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_38_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_50: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_50",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_61_8: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_61_8",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_100: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_100",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_600: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_600",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_1: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_1",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_3: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_3",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_4: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_4",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexes, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        [
+            (&mut self.constant_0, 0),
+            (&mut self.constant_1, 1),
+            (&mut self.constant_2, 2),
+            (&mut self.constant_3, 3),
+            (&mut self.constant_4, 4),
+            (&mut self.constant_50, 50),
+            (&mut self.constant_100, 100),
+            (&mut self.constant_600, 600),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredU16::new(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        [
+            (&mut self.constant_minus_1, -1),
+            (&mut self.constant_minus_2, -2),
+            (&mut self.constant_minus_3, 3),
+            (&mut self.constant_minus_4, 4),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredI16::new(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        [
+            (&mut self.constant_38_2, 38.2),
+            (&mut self.constant_61_8, 61.8),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredF32::from(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        Ok(())
+    }
+}

crates/brk_computer/src/fetched.rs

@@ -0,0 +1,137 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{DateIndex, Height, OHLCCents, Version};
+use vecdb::{
+    AnyStoredVec, AnyVec, BytesVec, Database, Exit, GenericStoredVec, ImportableVec, IterableVec,
+    PAGE_SIZE, TypedVecIterator, VecIndex,
+};
+
+use super::{Indexes, indexes, utils::OptionExt};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+    fetcher: Fetcher,
+
+    pub dateindex_to_price_ohlc_in_cents: BytesVec<DateIndex, OHLCCents>,
+    pub height_to_price_ohlc_in_cents: BytesVec<Height, OHLCCents>,
+}
+
+impl Vecs {
+    pub fn forced_import(parent: &Path, fetcher: Fetcher, version: Version) -> Result<Self> {
+        let db = Database::open(&parent.join("fetched"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let this = Self {
+            fetcher,
+
+            dateindex_to_price_ohlc_in_cents: BytesVec::forced_import(
+                &db,
+                "price_ohlc_in_cents",
+                version + Version::ZERO,
+            )?,
+            height_to_price_ohlc_in_cents: BytesVec::forced_import(
+                &db,
+                "price_ohlc_in_cents",
+                version + Version::ZERO,
+            )?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexer, indexes, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        let height_to_timestamp = &indexer.vecs.block.height_to_timestamp;
+        let index = starting_indexes
+            .height
+            .min(Height::from(self.height_to_price_ohlc_in_cents.len()));
+        let mut prev_timestamp = index
+            .decremented()
+            .map(|prev_i| height_to_timestamp.iter().unwrap().get_unwrap(prev_i));
+        height_to_timestamp
+            .iter()?
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(i, v)| -> Result<()> {
+                self.height_to_price_ohlc_in_cents.truncate_push_at(
+                    i,
+                    self.fetcher
+                        .get_height(i.into(), v, prev_timestamp)
+                        .unwrap(),
+                )?;
+                prev_timestamp = Some(v);
+                Ok(())
+            })?;
+        self.height_to_price_ohlc_in_cents.safe_write(exit)?;
+
+        let index = starting_indexes
+            .dateindex
+            .min(DateIndex::from(self.dateindex_to_price_ohlc_in_cents.len()));
+        let mut prev = Some(index.decremented().map_or(OHLCCents::default(), |prev_i| {
+            self.dateindex_to_price_ohlc_in_cents
+                .iter()
+                .unwrap()
+                .get_unwrap(prev_i)
+        }));
+        indexes
+            .dateindex_to_date
+            .iter()
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(i, d)| -> Result<()> {
+                let ohlc = if i.to_usize() + 100 >= self.dateindex_to_price_ohlc_in_cents.len()
+                    && let Ok(mut ohlc) = self.fetcher.get_date(d)
+                {
+                    let prev_open = *prev.u().close;
+                    *ohlc.open = prev_open;
+                    *ohlc.high = (*ohlc.high).max(prev_open);
+                    *ohlc.low = (*ohlc.low).min(prev_open);
+                    ohlc
+                } else {
+                    prev.clone().unwrap()
+                };
+
+                prev.replace(ohlc.clone());
+
+                self.dateindex_to_price_ohlc_in_cents
+                    .truncate_push_at(i, ohlc)?;
+
+                Ok(())
+            })?;
+        self.dateindex_to_price_ohlc_in_cents.safe_write(exit)?;
+
+        Ok(())
+    }
+}