dist: enable for brk_cli

Add Docker support

.claude/prompts/changelog.md

@@ -0,0 +1,221 @@
+# Changelog Generation for Claude Code
+
+**TASK**: Update docs/CHANGELOG.md for ALL latest releases missing from the file.
+
+## ⚠️ CRITICAL FAILURE MODE TO AVOID ⚠️
+**THE #1 FAILURE**: Ignoring most changes and only documenting a few
+**ABSOLUTELY FORBIDDEN**: Skipping changes, summarizing with "and other updates", or being incomplete
+**YOU MUST DOCUMENT EVERY SINGLE MEANINGFUL CHANGE - NO EXCEPTIONS**
+
+## CORE WORKFLOW - EXECUTE EXACTLY:
+
+### Step 1: Get Release Information
+```bash
+git tag --list --sort=version:refname
+```
+
+### Step 2: Process ONE Release at a Time
+For each missing release, execute these commands to get complete information:
+
+**First, get the file list:**
+```bash
+git diff --name-only [previous-tag]..[current-tag]
+```
+
+**Then, get changes excluding Cargo.lock:**
+```bash
+git diff [previous-tag]..[current-tag] -- . ':(exclude)Cargo.lock'
+```
+
+**If output is too large, examine files individually:**
+```bash
+git diff [previous-tag]..[current-tag] -- path/to/specific/file.rs
+```
+
+**For summary of changes per file (if needed):**
+```bash
+git diff --stat [previous-tag]..[current-tag]
+```
+
+### Step 3: Analyze Before Writing
+**MANDATORY**: Before writing ANY changelog entry, analyze the diff output and explain:
+- **Use `git diff --name-only` to see ALL changed files** - this prevents truncation issues
+- **Use `git diff -- . ':(exclude)Cargo.lock'` to see actual changes** without Cargo.lock noise
+- **If output is large, examine key files individually** with `git diff [tags] -- path/to/file`
+- **Identify every functional change** - what new capabilities, fixes, or modifications were made
+- What each code change accomplishes functionally
+- The user-facing or system impact of modifications
+- **Which crate each change belongs to** (based on file paths)
+- How changes group together logically within and across crates
+- What functionality is added/removed/modified per crate
+
+**COMPLETENESS CHECK**: State "I have analyzed X files and identified Y distinct functional changes to document"
+
+### Step 4: Write Changelog Entry
+Only after analysis, update CHANGELOG.md with ONE release entry.
+
+**REQUIRED**: End each release entry with a comparison link:
+```markdown
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/vPREVIOUS...vCURRENT)
+```
+Where PREVIOUS is the previous release tag and CURRENT is the current release tag.
+
+### Step 5: Stop and Confirm
+**CRITICAL**: Process only ONE release, then ask for confirmation to continue.
+
+---
+
+## STRICT RULES
+
+### FORBIDDEN - NEVER MENTION:
+- **Cargo.lock** (ignore completely)
+- **Line counts** ("added 50 lines", "removed 200 lines")
+- **Dependency version bumps** (unless enabling major new features)
+- **Local crate version changes** (0.0.61 → 0.0.62)
+- **Vague words**: "Enhanced", "Improved", "Updated", "Refactored"
+
+### FORBIDDEN PHRASES:
+- "and other changes"
+- "various updates"
+- "along with minor improvements"
+- "among other enhancements"
+- **"along with other modifications"**
+- **"plus additional changes"**
+- **"including other updates"**
+- **"and more"**
+- **ANY phrase that suggests you're skipping changes**
+
+### REQUIRED - MUST INCLUDE:
+- **Every meaningful change** in the diff (no shortcuts)
+- **Specific functionality** descriptions
+- **Business impact** of each change
+- **Complete coverage** - if 20 changes exist, document all 20
+- **Source links** for significant changes (new features, breaking changes, major bug fixes)
+
+## WORKSPACE-SPECIFIC RULES
+
+### Crate Identification:
+- **Determine crate from file paths** in the diff (e.g., `crates/brk-core/src/lib.rs` → `brk-core`)
+- **Group all changes by their crate** before writing changelog entries
+- **Use crate name as subheading** under each change type section
+- **For root-level files**: Use `workspace` as the crate name
+
+### Cross-Crate Changes:
+- **When changes span multiple crates** for one feature, mention the relationship
+- **Example**: "Added new transaction API in `brk-core` with corresponding HTTP endpoints in `brk-api`"
+
+### Crate Naming:
+- **Use backticks** around crate names: `brk-core`, `brk-api`
+- **Use workspace structure** as shown in file paths, not display names
+
+### File Header (if missing):
+```markdown
+<!-- This changelog was generated by Claude Code -->
+```
+
+### Release Entry Format:
+```markdown
+## [vX.Y.Z](https://github.com/bitcoinresearchkit/brk/releases/tag/vX.Y.Z) - YYYY-MM-DD
+
+### Breaking Changes
+#### `crate-name`
+- Specific change with functional impact explanation ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/file.rs))
+
+### New Features
+#### `crate-name`
+- Feature description with user benefit ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/main/file.rs))
+- Another feature with implementation details
+
+#### `another-crate`
+- Feature specific to this crate
+
+### Bug Fixes
+#### `crate-name`
+- Specific problem that was resolved ([source](https://github.com/bitcoinresearchkit/brk/blob/vX.Y.Z/path/to/file.rs))
+- Another bug fix with impact description
+
+### Internal Changes
+#### `crate-name`
+- Architectural improvement with purpose
+- Code organization change with benefit
+
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/vPREVIOUS...vCURRENT)
+```
+
+---
+
+## ANALYSIS REQUIREMENTS
+
+**Before writing changelog text, you MUST state:**
+
+1. **File discovery**: "Using git diff --name-only, I see X files were modified"
+2. **Cargo.lock check**: "After excluding Cargo.lock, Y files contain actual changes"
+3. **Functional changes identified**: "I identified these distinct changes: A, B, C, D..."
+4. **What each change does**: "Change A: adds new API endpoints, Change B: fixes memory leak, etc."
+5. **Which crates are affected**: "Changes span crates: A, B, C"
+6. **What this means for users**: "Users can now do X, bug Y is fixed, etc."
+7. **How changes group together**: "Changes A and B work together to implement feature W"
+8. **Cross-crate dependencies**: "Crate A's new feature requires the new API in crate B"
+9. **Any unclear changes**: "I cannot determine the purpose of change X from the diff"
+10. **FINAL CHECK**: "I will now document all X functional changes without listing files"
+
+**MANDATORY**: The changelog should describe FUNCTIONALITY, not files. But you must capture ALL functionality changes.
+
+**Only after this analysis, write the changelog entry.**
+
+---
+
+## EXAMPLES
+
+### ❌ BAD (Vague, incomplete):
+```markdown
+### New Features
+- Enhanced blockchain processing capabilities
+- Improved error handling and various other updates
+```
+
+### ✅ GOOD (Specific, complete, grouped by crate, with source links):
+```markdown
+### New Features
+#### `brk-core`
+- Added `TransactionAnalyzer` struct with fee calculation and coinbase detection methods ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-core/src/analyzer.rs))
+- Implemented in-memory caching layer for blockchain queries using HashMap storage
+
+#### `brk-api`
+- Added three new API endpoints: `/api/blocks/{hash}`, `/api/transactions/search`, and `/api/stats/network` ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-api/src/routes.rs))
+- Implemented standardized error responses with error codes and descriptions
+
+### Bug Fixes
+#### `brk-core`
+- Fixed panic when processing blocks with zero transactions by adding explicit empty block validation ([source](https://github.com/bitcoinresearchkit/brk/blob/v0.0.108/crates/brk-core/src/block.rs))
+
+#### `brk-api`
+- Resolved memory leak in connection pool by implementing proper cleanup in Drop trait
+
+[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.107...v0.0.108)
+```
+
+---
+
+## SUCCESS CRITERIA
+
+✅ **COUNT files modified for internal verification only**
+✅ **Identify EVERY distinct functional change**
+✅ **Document ALL functionality changes (no "other changes")**
+✅ **Changelog describes WHAT users can do, not which files changed**
+✅ **Never mention Cargo.lock or line counts**
+✅ **Use specific, functional descriptions**
+✅ **Complete analysis before writing**
+✅ **Stop and ask for confirmation after each release**
+
+**FAILURE INDICATORS - If you do any of these, you FAILED:**
+❌ "and other changes"
+❌ "various updates"
+❌ "among other improvements"
+❌ **Diff shows new API endpoints but changelog doesn't mention them**
+❌ **Diff shows bug fixes but changelog misses some**
+❌ **Diff shows new structs/functions but changelog ignores them**
+❌ Missing obvious functional changes from the diff
+❌ Summarizing instead of listing each distinct functionality change
+
+**KEY PRINCIPLE**: Count files internally to ensure you don't miss changes, but write about FUNCTIONALITY for users.

.claude/prompts/readmes.md

@@ -0,0 +1,172 @@
+# README Generation Prompt
+
+Generate a professional, comprehensive README.md for each crate based SOLELY on code analysis. Use NO external documentation, commit messages, or existing READMEs.
+
+## MANDATORY PROCESS - FOLLOW EXACTLY:
+1. **IGNORE EXISTING DOCS**: Do NOT read any .md, .txt, .rst, or documentation files in the crate directory
+2. **CODE-ONLY ANALYSIS**: Examine ONLY these files:
+   - All .rs files in src/ directory and subdirectories
+   - Cargo.toml for dependencies and metadata
+   - Code structure and organization
+3. **MANDATORY CODE ANALYSIS**: Before writing ANY README content, you MUST:
+   - Examine all Rust files in src/ directory
+   - Identify the main structs, enums, traits, and functions
+   - Understand the crate's architecture and data flow
+   - Determine the crate's purpose from its implementation
+   - Map dependencies to understand external integrations
+4. **FRESH PERSPECTIVE**: Write the README as if you're the first person to document this crate
+5. Generate one complete README.md per crate
+6. Focus on one crate at a time for thorough analysis
+
+## ABSOLUTE REQUIREMENTS:
+- **SOURCE OF TRUTH**: Use ONLY the actual Rust code - no external docs, comments may provide hints but focus on implementation
+- **CRITICAL**: DO NOT read any existing README.md, CHANGELOG.md, or documentation files
+- **IGNORE ALL TEXT FILES**: .md, .txt, .rst files are FORBIDDEN sources - treat them as if they don't exist
+- **CODE ONLY**: Focus exclusively on .rs files, Cargo.toml, and code structure
+- **PROFESSIONAL GRADE**: Write as if this will be published on crates.io for other developers
+- **PROGRAMMER FOCUSED**: Assume audience knows Rust and relevant domain concepts
+- **IMPLEMENTATION-BASED**: Describe what the code actually does, not what comments claim it should do
+- **If you cannot determine functionality from code alone, state this explicitly**
+
+## README STRUCTURE (MANDATORY):
+
+### 1. CRATE HEADER
+```markdown
+# Crate Name
+
+Brief one-line description of what this crate does (max 80 chars).
+
+[![Crates.io](https://img.shields.io/crates/v/CRATE_NAME.svg)](https://crates.io/crates/CRATE_NAME)
+[![Documentation](https://docs.rs/CRATE_NAME/badge.svg)](https://docs.rs/CRATE_NAME)
+```
+
+### 2. OVERVIEW SECTION
+- **Purpose**: What problem does this crate solve?
+- **Key Features**: 3-5 bullet points of main capabilities (derived from code analysis)
+- **Target Use Cases**: Who would use this and for what?
+
+### 3. INSTALLATION
+```toml
+[dependencies]
+crate_name = "X.Y.Z"
+```
+
+### 4. QUICK START / USAGE
+- **Minimal working example** showing the primary API
+- **Common patterns** observed in the code
+- **Key structs/traits** that users will interact with
+
+### 5. API OVERVIEW
+- **Core Types**: Main structs, enums, traits with brief descriptions
+- **Key Methods**: Most important public functions
+- **Module Structure**: Brief overview of how code is organized
+
+### 6. FEATURES (if applicable)
+- Cargo features and what they enable
+- Optional dependencies and their purpose
+
+### 7. EXAMPLES
+- 2-3 practical code examples showing different use cases
+- Based on public API analysis, not existing examples
+
+## WRITING REQUIREMENTS:
+
+### TONE AND STYLE:
+- **Concise but comprehensive**: Every sentence must add value
+- **Technical precision**: Use exact terminology, avoid marketing speak
+- **Active voice**: "Provides X" not "X is provided"
+- **Present tense**: "The crate handles..." not "The crate will handle..."
+
+### FORBIDDEN PATTERNS:
+- **NEVER** use vague terms: "powerful", "flexible", "robust", "comprehensive", "advanced"
+- **NEVER** write marketing copy: "cutting-edge", "state-of-the-art", "enterprise-grade"
+- **NEVER** make claims you can't verify from code: "blazingly fast", "memory efficient"
+- **NEVER** copy-paste from existing documentation or comments
+- **NEVER** read or reference existing README.md files - pretend they don't exist
+- **NEVER** use phrases like "as mentioned in the documentation" or "according to the docs"
+- **NEVER** let existing documentation influence your analysis or writing
+
+### REQUIRED SPECIFICITY:
+- **Data structures**: Mention specific types (HashMap, Vec, etc.)
+- **Algorithms**: Reference actual implementations found in code
+- **Integration points**: Specific traits implemented, dependencies used
+- **Error handling**: How errors are represented and handled
+- **Async/sync**: Clearly state if operations are blocking or async
+
+### ANTI-BIAS PROTOCOL:
+
+### BEFORE STARTING ANY ANALYSIS:
+1. **Explicitly ignore**: Any README.md, CHANGELOG.md, docs/, documentation files
+2. **File filtering**: Only examine .rs and Cargo.toml files
+3. **Fresh eyes approach**: Analyze the code as if you've never seen this crate before
+4. **Independent thinking**: Form your own understanding purely from code inspection
+
+### IF YOU ACCIDENTALLY READ EXISTING DOCS:
+- Stop immediately and restart your analysis
+- Consciously disregard any information from documentation files
+- Base all descriptions solely on what you observe in the code
+- Ask yourself: "What would I think this code does if I had no documentation?"
+
+### VALIDATION CHECKS:
+- **Unique descriptions**: Your descriptions should differ significantly from any existing docs
+- **Code-derived insights**: Every feature mentioned must be visible in the source code
+- **Independent voice**: Write in your own technical style, not mimicking existing documentation
+- **Fresh examples**: Create new code examples based on API analysis, not existing samples
+
+## CODE ANALYSIS DEPTH:
+**You MUST analyze and understand:**
+1. **Public API surface**: All pub structs, functions, traits, modules
+2. **Core abstractions**: Main data types and their relationships
+3. **Error types**: Custom errors, Result patterns, panic conditions
+4. **Dependencies**: How external crates are integrated
+5. **Feature flags**: Conditional compilation and optional functionality
+6. **Async patterns**: Use of futures, tokio, async-std, etc.
+7. **Serialization**: Serde implementations, custom serialization
+8. **Performance characteristics**: Algorithm complexity where obvious
+
+### EXAMPLE STRUCTURE ANALYSIS OUTPUT:
+```markdown
+## Code Analysis Summary
+**Main Types**: `BlockProcessor`, `Transaction`, `ValidationError`
+**Core Trait**: `Validator` - implemented by `BasicValidator` and `StrictValidator`
+**Async Support**: All processing methods return `impl Future`
+**Error Handling**: Custom `ValidationError` enum with specific error types
+**Dependencies**: Uses `tokio` for async runtime, `serde` for serialization
+**Architecture**: Pipeline pattern with configurable validation stages
+```
+
+## EXAMPLES OF QUALITY:
+
+### ❌ BAD (VAGUE):
+```markdown
+# My Crate
+A powerful and flexible library for blockchain operations.
+
+## Features
+- Fast processing
+- Easy to use
+- Robust error handling
+```
+
+### ✅ GOOD (SPECIFIC):
+```markdown
+# brk-chain-analyzer
+Bitcoin blockchain analysis tools for transaction pattern detection.
+
+## Overview
+Provides utilities for analyzing Bitcoin transaction data, detecting address clustering patterns, and computing blockchain statistics. Built around a streaming parser that processes block data without loading entire blocks into memory.
+
+## Key Types
+- `TransactionAnalyzer`: Stateful analyzer for computing fees, detecting coinbase transactions
+- `ClusterDetector`: Implements common input ownership heuristics for address clustering
+- `BlockStream`: Async iterator over blockchain data with configurable batch sizes
+```
+
+## FINAL REQUIREMENTS:
+- **One README per crate** - don't combine multiple crates
+- **Minimum 200 words** - be thorough but concise
+- **Maximum 800 words** - stay focused and relevant
+- **Code examples must be syntactically correct** and compilable
+- **All claims must be verifiable** from the source code
+
+**PROCESS ONE CRATE AT A TIME. ANALYZE THE CODE THOROUGHLY BEFORE WRITING.**

.github/workflows/release.yml

@@ -1,4 +1,4 @@
-# This file was autogenerated by dist: https://opensource.axo.dev/cargo-dist/
+# This file was autogenerated by dist: https://axodotdev.github.io/cargo-dist
 #
 # Copyright 2022-2024, axodotdev
 # SPDX-License-Identifier: MIT or Apache-2.0
@@ -47,7 +47,7 @@ on:
 jobs:
   # Run 'dist plan' (or host) to determine what tasks we need to do
   plan:
-    runs-on: "ubuntu-latest"
+    runs-on: "ubuntu-22.04"
     outputs:
       val: ${{ steps.plan.outputs.manifest }}
       tag: ${{ !github.event.pull_request && github.ref_name || '' }}
@@ -58,12 +58,13 @@ jobs:
     steps:
       - uses: actions/checkout@v4
         with:
+          persist-credentials: false
           submodules: recursive
       - name: Install dist
         # we specify bash to get pipefail; it guards against the `curl` command
         # failing. otherwise `sh` won't catch that `curl` returned non-0
         shell: bash
-        run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.28.0/cargo-dist-installer.sh | sh"
+        run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.30.2/cargo-dist-installer.sh | sh"
       - name: Cache dist
         uses: actions/upload-artifact@v4
         with:
@@ -117,6 +118,7 @@ jobs:
           git config --global core.longpaths true
       - uses: actions/checkout@v4
         with:
+          persist-credentials: false
           submodules: recursive
       - name: Install Rust non-interactively if not already installed
         if: ${{ matrix.container }}
@@ -168,13 +170,14 @@ jobs:
     needs:
       - plan
       - build-local-artifacts
-    runs-on: "ubuntu-latest"
+    runs-on: "ubuntu-22.04"
     env:
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
       BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json
     steps:
       - uses: actions/checkout@v4
         with:
+          persist-credentials: false
           submodules: recursive
       - name: Install cached dist
         uses: actions/download-artifact@v4
@@ -214,16 +217,17 @@ jobs:
       - plan
       - build-local-artifacts
       - build-global-artifacts
-    # Only run if we're "publishing", and only if local and global didn't fail (skipped is fine)
-    if: ${{ always() && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
+    # Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
+    if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
     env:
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-    runs-on: "ubuntu-latest"
+    runs-on: "ubuntu-22.04"
     outputs:
       val: ${{ steps.host.outputs.manifest }}
     steps:
       - uses: actions/checkout@v4
         with:
+          persist-credentials: false
           submodules: recursive
       - name: Install cached dist
         uses: actions/download-artifact@v4
@@ -282,10 +286,11 @@ jobs:
     # still allowing individual publish jobs to skip themselves (for prereleases).
     # "host" however must run to completion, no skipping allowed!
     if: ${{ always() && needs.host.result == 'success' }}
-    runs-on: "ubuntu-latest"
+    runs-on: "ubuntu-22.04"
     env:
       GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
     steps:
       - uses: actions/checkout@v4
         with:
+          persist-credentials: false
           submodules: recursive

.gitignore

@@ -3,6 +3,9 @@
 
 # Builds
 target
+websites/dist
+bridge/
+/ids.txt
 
 # Copies
 *\ copy*
@@ -10,9 +13,23 @@ target
 # Ignored
 _*
 
-# Editors
-.vscode
-.zed
-
 # Logs
-.log
+*.log*
+
+# Environment variables/configs
+.env
+
+# Profiling
+profile.json.gz
+flamegraph.svg
+*.trace
+
+# AI
+.claude/settings*
+
+# Expand
+expand.rs
+
+# Benchmarks
+[0-9]/
+/benches

.zed/settings.json

@@ -0,0 +1,22 @@
+{
+  "file_scan_exclusions": [
+    // default
+    "**/.git",
+    "**/.svn",
+    "**/.hg",
+    "**/.jj",
+    "**/CVS",
+    "**/.DS_Store",
+    "**/Thumbs.db",
+    "**/.classpath",
+    "**/.settings",
+    // custom
+    "**/lean-qr/*/index.mjs",
+    "**/modern-screenshot/*/index.mjs",
+    "**/solidjs-signals/*/dist/prod.js",
+    "uFuzzy.mjs",
+    "lightweight-charts.standalone.production.mjs"
+    // "scripts/packages",
+    // "dist"
+  ]
+}

CHANGELOG.md

@@ -1,285 +0,0 @@
-<!--
-# v0.X.Y | WIP
-![Image of the kibo Web App version 0.X.Y](https://github.com/kibo-money/kibo/blob/main/_assets/v0.X.Y.jpg)
--->
-
-# v0.X.0 | WIP | A new beginning
-
-Full rewrite
-
-# [kibo-v0.5.0](https://github.com/kibo-money/kibo/tree/eea56d394bf92c62c81da8b78b8c47ea730683f5) | [873199](https://mempool.space/block/0000000000000000000270925aa6a565be92e13164565a3f7994ca1966e48050) - 2024/12/04
-
-![Image of the kibo Web App version 0.5.0](https://github.com/kibo-money/kibo/blob/main/_assets/v0.5.0.jpg)
-
-## Datasets
-
-- Added `Sell Side Risk Ratio` to all entities
-- Added `Open`, `High` and `Low` datasets
-- Added `Satoshis Per Dollar`
-- Added `All Time High`
-- Added `All Time High Date`
-- Added `Days Since All Time High`
-- Added `Max Days Between All Time Highs`
-- Added `Max Years Between All Time Highs`
-- Added `Drawdown`
-- Added `Adjusted Value Created`, `Adjusted Value Destroyed` and `Adjusted Spent Output Profit Ratio` to all entities
-- Added `Realized Profit To Loss Ratio` to all entities
-- Added `Hash Price Min`
-- Added `Hash Price Rebound`
-- Removed all year datasets (25) in favor for epoch datasets (5), the former was too granular to be really useful
-- Removed datasets split by liquidity for all datasets **already split by any address kind**, while fun to have, they took time to compute, ram, and space to store and no one was actually checking them
-- Fixed a lot of values in split by liquidity datasets
-
-## Website
-
-- Updated the design yet again which made the website for something more minimal and easier on the eyes
-- Added a *Save In Bitcoin* (DCA) simulation page
-- ~Added a dashboard~ Added the latest values to the tree next to each option instead, while less values are visible at a time, it's much more readable and organised
-- Added a library of PDFs
-- Fixed service worker not passing 304 (not modified) response and instead serving cached responses
-- Fixed history not being properly registered
-- Fixed window being moveable on iOS when in standalone mode when it shouldn't be
-- Added `Compare` section to all groups, to compare all datasets within a group
-- Updated `Solid Signals` library, which had an important breaking change on the `createEffect` function which might bring some bugs
-- Fixed some datasets paths
-- A lot of code reorg and file splits
-- Adopted a framework like approach to load pages while still being pure JS without a build step
-- Probably more that was forgotten
-
-## Parser
-
-- Added a `/datasets/last` json file with all the latest values
-- Added `--rpcconnect` parameter to the config
-- Added handling of SIGINT and SIGTERM terminal signals which menas you can now safely CTRL+C or kill the parser while it's exporting
-- Added config print at the start of the program
-- Compressed `empty_address_data` struct to save space (should shave of between up to 50% of the `address_index_to_empty_address_data` database)
-- Doubled the number of `txid_to_tx_data` databases from 4096 to 8192
-- ~Added `--recompute_computed true` argument, to allow recomputation of computed datasets in case of a bug~ Buggy for now
-- Fixed not saved arguments, not being processed properly
-- Fixed bug in `generic_map.multi_insert_simple_average`
-- Added defragmentation option `--first-defragment true` of databases to save space (which can save up to 50%)
-- Fixed bug in the computation of averages in `GenericMap`
-- Added support and paramer for cookie files with `--rpccookiefile`, and auto find if the path is `--datadir/.cookie`
-- Increased number of retries and time between them when fetching price from exchanges APIs
-
-## Server
-
-- Fixed links in several places missing the `/api` part and thus not working
-- Fixed broken last values routes
-- Added support for the `/datasets/last` file via the `/api/last` route
-- Added support for `.json` (won't change anything) and `.csv` (will download a csv file) extension at the end of datasets routes
-- Added `all=true` query parameter to dataset routes to get to full history
-
-## Biter
-
-- Moved back to this repo
-
-# [kibo-v0.4.0](https://github.com/kibo-money/kibo/tree/a64c544815d9ef785e2fc1323582f774f16b9200) | [861950](https://mempool.space/block/00000000000000000000530d0e30ccf7deeace122dcc99f2668a06c6dad83629) - 2024/09/19
-
-![Image of the kibo Web App version 0.4.0](https://github.com/kibo-money/kibo/blob/main/_assets/v0.4.0.jpg)
-
-## Brand
-
-- **Satonomics** is now **kibo** 🎉
-
-## Website
-
-- Complete redesign of the website
-- Rewrote the whole application and removed `node`/`npm`/`pnpm` dependencies in favor for pure `HTML`/`CSS`/`Javascript`
-- Website is now served by the server
-- Added Trading View attribution link to the settings frame and file in the lightweight charts folder
-- Many other changes
-
-## Parser
-
-- Changed the block iterator from a custom version of [bitcoin-explorer](https://crates.io/crates/bitcoin-explorer) to the homemade [biter](https://crates.io/crates/biter) which allows the parser to run alongside `bitcoind`
-- Added datasets compression thanks to [zstd](https://crates.io/crates/zstd) to reduce disk usage
-- Use the Bitcoin RPC server for various calls instead of running cli commands and then parsing the JSON from the output
-- **Important database changes that will need a full rescan**:
-  - Changed databases page size from 1MB to 4KB for improved disk usage
-  - Split txid_to_tx_data database in 4096 chunks (from 256) for improved disk usage
-  - Split address_index_to_X databases to chunks of 25_000 instead of 50_000
-  - Removed local Multisig database
-- Updated the config, run with `-h` to see possible args
-- Moved outputs from `/target/outputs` to `/out` to allow to run commands like `cargo clean` without side effects
-- Various first run fixes
-- Added to `-h` which arguments are saved, which is all of them at the time of writing
-
-## Server
-
-- Updated the code to support compressed binaries
-- Added serving of the website
-- Improved `Cache-Control` behavior
-
-# [kibo-v0.3.0](https://github.com/kibo-money/kibo/tree/b68b016091c45b071218fba01bac5b76e8eaf18c) | [853930](https://mempool.space/block/00000000000000000002eb5e9a7950ca2d5d98bd1ed28fc9098aa630d417985d) - 2024/07/26
-
-![Image of the Satonomics Web App version 0.3.0](https://github.com/kibo-money/kibo/blob/main/_assets/v0.3.0.jpg)
-
-## Parser
-
-- Global
-  - Improved self-hosting by:
-    - Fixing an incredibly annoying bug that made the program panic because of a wrong utxo/address durable state after a or many new datasets were added/changed after a first successful parse of the chain
-    - Fixing a bug that would crash the program if launched for the first time ever
-    - Auto fetch prices from the main Satonomics instance if missing instead of only trying Kraken's and Binance's API which are limited to the last 16 hours
-  - Merged the core of `HeightMap` and `DateMap` structs into `GenericMap`
-  - Added `Height` struct and many others
-  - Reorganized outputs of both the parser and the server for ease of use and easier sync compatibility
-- CLI
-  - Added an argument parser for improved UX with several options
-- Datasets
-  - Added the following datasets for all entities:
-    - Value destroyed
-    - Value created
-    - Spent Output Profit Ratio (SOPR)
-  - Added the following ratio datasets and their variations to all prices {realized, moving average, any cointime, etc}:
-    - Market Price to {X}
-    - Market Price to {X} Ratio
-    - Market Price to {X} Ratio 1 Week SMA
-    - Market Price to {X} Ratio 1 Month SMA
-    - Market Price to {X} Ratio 1 Year SMA
-    - Market Price to {X} Ratio 1 Year SMA Momentum Oscillator
-    - Market Price to {X} Ratio 99th Percentile
-    - Market Price to {X} Ratio 99.5th Percentile
-    - Market Price to {X} Ratio 99.9th Percentile
-    - Market Price to {X} Ratio 1st Percentile
-    - Market Price to {X} Ratio 0.5th Percentile
-    - {X} 1% Top Probability
-    - {X} 0.5% Top Probability
-    - {X} 0.1% Top Probability
-    - {X} 1% Bottom Probability
-    - {X} 0.5% Bottom Probability
-    - {X} 0.1% Bottom Probability
-  - Added block metadatasets and their variants (raw/sum/average/min/max/percentiles):
-    - Block size
-    - Block weight
-    - Block VBytes
-    - Block interval
-- Price
-  - Improved error message when price cannot be found
-
-## App
-
-- General
-  - Added chart scroll button for nice animations à la Wicked
-  - Added scale mode switch (Linear/Logarithmic) at the bottom right of all charts
-  - Added unit at the top left of all charts
-  - Added a backup API in case the main one fails or is offline
-  - Complete redesign of the datasets object
-  - Removed import of routes in JSON in favor for hardcoded typed routes in string format which resulted in:
-    - \+ A much lighter app
-    - \+ Better Lighthouse score
-    - \- Slower Typescript server
-  - Fixed datasets with null values crashing their fetch function
-  - Added a 'Go to a random chart' button in several places
-- Chart
-  - Fixed series color being set to default ones after hovering the legend
-  - Fixed chart starting showing candlesticks and quickly switching to a line when it should've started directly with the line
-  - Separated the QRCode generator library from the main chunk and made it imported on click
-  - Fixed timescale changing on small screen after changing charts
-- Folders
-  - Added the size in the "filename" of address cohorts grouped by size
-- Favorites
-  - Added a 'favorite' and 'unfavorite' button at the bottom
-- Settings
-  - Removed the horizontal scroll bar which was unintended
-
-## Server
-
-- Run file
-  - Only run with a watcher if `cargo watch` is available
-  - Removed id_to_path file in favor for only `paths.d.ts` in `app/src/types`
-
-# [kibo-v0.2.0](https://github.com/kibo-money/kibo/tree/248187889283597c5dbb806292297453c25e97b8) | [851286](https://mempool.space/block/0000000000000000000281ca7f1bf8c50702bfca168c7af1bdc67c977c1ac8ed) - 2024/07/08
-
-![Image of the Satonomics Web App version 0.2.0](https://github.com/kibo-money/kibo/blob/main/_assets/v0.2.0.jpg)
-
-## App
-
-- General
-  - Added the height version of all datasets and many optimizations to make them usable but only available on desktop and tablets for now
-  - Added a light theme
-- Charts
-  - Added split panes in order to have the vertical axis visible for all datasets
-  - Added min and max values on the charts
-  - Fixed legend hovering on mobile not resetting on touch end
-  - Added "3 months" and yearly time scale setters (from year 2009 to today)
-  - Hide scrollbar of timescale setters and instead added scroll buttons to the legend only visible on desktop
-  - Improved Share/QR Code screen
-  - Changed all Area series to Line series
-  - Fixed horizontal scrollable legend not updating on preset change
-- Performance
-  - Improved app's reactivity
-  - Added some chunk splitting for a faster initial load
-  - Global improvements that increased the Lighthouse's performance score
-- Settings
-  - Finally made a proper component where you can chose the app's theme, between a moving or static background and its text opacity
-  - Added donations section with a leaderboard
-  - Added various links that are visible on the bottom side of the strip on desktop to mobile users
-  - Added install instructions when not installed for Apple users
-- Misc
-  - Support mini window size, could be useful for embedded views
-  - Hopefully made scrollbars a little more subtle on WIndows and Linux, can't test
-  - Generale style updates
-
-## Parser
-
-- Fixed ulimit only being run in Mac OS instead of whenever the program is detected
-
-# [kibo-v0.1.1](https://github.com/kibo-money/kibo/tree/e55b5195a9de9aea306903c94ed63cb1720fda5f) | [849240](https://mempool.space/block/000000000000000000002b8653988655071c07bb5f7181c038f9326bc86db741) - 2024/06/24
-
-![Image of the Satonomics Web App version 0.1.1](https://github.com/kibo-money/kibo/blob/main/_assets/v0.1.1.jpg)
-
-## Parser
-
-- Fixed overflow in `Price` struct which caused many Realized Caps and Realized Prices to have completely bogus data
-- Fixed Realized Cap computation which was using rounded prices instead normal ones
-
-## Server
-
-- Added the chunk, date and time of the request to the terminal logs
-
-## App
-
-- Chart
-  - Added double click option on a legend to toggle the visibility of all other series
-  - Added highlight effect to a legend by darkening the color of all the other series on the chart while hovering it with the mouse
-  - Added an API link in the legend for each dataset where applicable (when isn't generated locally)
-  - Save fullscreen preference in local storage and url
-  - Improved resize bar on desktop
-  - Changed resize button logo
-  - Changed the share button to visible on small screen too
-  - Improved share screen
-  - Fixed time range shifting not being the one in url params or saved in local storage
-  - Fixed time range shifting on series toggling via the legend
-  - Fixed time range shifting on fullscreen
-  - Fixed time range shifting on resize of the sidebar
-  - Set default view at first load to last 6 months
-  - Added some padding around the datasets (year 1970 to 2100)
-- History
-  - Changed background for the sticky dates from blur to a solid color as it didn't appear properly in Firefox
-- Build
-  - Tried to add lazy loads to have split chunks after build, to have much faster load times and they worked great ! But they completely broke Safari on iOS, we can't have nice things
-  - Removed many libraries and did some things manually instead to improve build size
-- Strip
-  - Temporarily removed the Home button on the strip bar on desktop as there is no landing page yet
-- Settings
-  - Added version
-- PWA
-  - Fixed background update
-  - Changed update check frequency to 1 minute (~1kb to fetch every minute which is very reasonable)
-  - Added a nice banner to ask the user to install the update
-- Misc
-  - Removed tracker even though it was a very privacy friendly as it appeared to not be working properly
-
-## Price
-
-- Deleted old price datasets and their backups
-
-# [kibo-v0.1.0](https://github.com/kibo-money/kibo/tree/a1a576d088c8f83ed32d48753a7611f70a964574) | [848642](https://mempool.space/block/000000000000000000020be5761d70751252219a9557f55e91ecdfb86c4e026a) - 2024/06/19
-
-![Image of the Satonomics Web App version 0.1.0](https://github.com/kibo-money/kibo/blob/main/_assets/v0.1.0.jpg)
-
-# kibo-v0.0.1 | [835444](https://mempool.space/block/000000000000000000009f93907a0dd83c080d5585cc7ec82c076d45f6d7c872) - 2024/03/20
-
-![Image of the Satonomics Web App version 0.0.X](https://github.com/kibo-money/kibo/blob/main/_assets/v0.0.X.jpg)

Cargo.toml

@@ -4,55 +4,84 @@ members = ["crates/*"]
 package.description = "The Bitcoin Research Kit is a suite of tools designed to extract, compute and display data stored on a Bitcoin Core node"
 package.license = "MIT"
 package.edition = "2024"
-package.version = "0.0.46"
+package.version = "0.1.0-alpha.1"
 package.homepage = "https://bitcoinresearchkit.org"
 package.repository = "https://github.com/bitcoinresearchkit/brk"
+package.readme = "README.md"
+
+[profile.dev]
+lto = "thin"
+codegen-units = 16
+opt-level = 2
+split-debuginfo = "unpacked"
 
 [profile.release]
 lto = "fat"
 codegen-units = 1
 panic = "abort"
+strip = true
+overflow-checks = false
+
+[profile.bloaty]
+debug = true
+lto = false
+strip = false
+inherits = "release"
 
 [profile.dist]
 inherits = "release"
 
+[profile.profiling]
+inherits = "release"
+debug = true
+
 [workspace.dependencies]
-arc-swap = "1.7.1"
-axum = "0.8.4"
-bincode = { version = "2.0.1", features = ["serde"] }
-bitcoin = { version = "0.32.6", features = ["serde"] }
+aide = { version = "0.16.0-alpha.1", features = ["axum-json", "axum-query"] }
+axum = "0.8.8"
+bitcoin = { version = "0.32.8", features = ["serde"] }
 bitcoincore-rpc = "0.19.0"
-brk_cli = { version = "0", path = "crates/brk_cli" }
-brk_computer = { version = "0", path = "crates/brk_computer" }
-brk_core = { version = "0", path = "crates/brk_core" }
-brk_exit = { version = "0", path = "crates/brk_exit" }
-brk_fetcher = { version = "0", path = "crates/brk_fetcher" }
-brk_indexer = { version = "0", path = "crates/brk_indexer" }
-brk_logger = { version = "0", path = "crates/brk_logger" }
-brk_parser = { version = "0", path = "crates/brk_parser" }
-brk_query = { version = "0", path = "crates/brk_query" }
-brk_server = { version = "0", path = "crates/brk_server" }
-brk_state = { version = "0", path = "crates/brk_state" }
-brk_store = { version = "0", path = "crates/brk_store" }
-brk_vec = { version = "0", path = "crates/brk_vec" }
-byteview = "=0.6.1"
-clap = { version = "4.5.39", features = ["string"] }
-clap_derive = "4.5.32"
+brk_bencher = { version = "0.1.0-alpha.1", path = "crates/brk_bencher" }
+brk_binder = { version = "0.1.0-alpha.1", path = "crates/brk_binder" }
+brk_bundler = { version = "0.1.0-alpha.1", path = "crates/brk_bundler" }
+brk_cli = { version = "0.1.0-alpha.1", path = "crates/brk_cli" }
+brk_computer = { version = "0.1.0-alpha.1", path = "crates/brk_computer" }
+brk_error = { version = "0.1.0-alpha.1", path = "crates/brk_error" }
+brk_fetcher = { version = "0.1.0-alpha.1", path = "crates/brk_fetcher" }
+brk_grouper = { version = "0.1.0-alpha.1", path = "crates/brk_grouper" }
+brk_indexer = { version = "0.1.0-alpha.1", path = "crates/brk_indexer" }
+brk_query = { version = "0.1.0-alpha.1", path = "crates/brk_query", features = ["tokio"] }
+brk_iterator = { version = "0.1.0-alpha.1", path = "crates/brk_iterator" }
+brk_logger = { version = "0.1.0-alpha.1", path = "crates/brk_logger" }
+brk_mcp = { version = "0.1.0-alpha.1", path = "crates/brk_mcp" }
+brk_mempool = { version = "0.1.0-alpha.1", path = "crates/brk_mempool" }
+brk_reader = { version = "0.1.0-alpha.1", path = "crates/brk_reader" }
+brk_rpc = { version = "0.1.0-alpha.1", path = "crates/brk_rpc" }
+brk_server = { version = "0.1.0-alpha.1", path = "crates/brk_server" }
+brk_store = { version = "0.1.0-alpha.1", path = "crates/brk_store" }
+brk_types = { version = "0.1.0-alpha.1", path = "crates/brk_types" }
+brk_traversable = { version = "0.1.0-alpha.1", path = "crates/brk_traversable", features = ["pco", "derive"] }
+brk_traversable_derive = { version = "0.1.0-alpha.1", path = "crates/brk_traversable_derive" }
+byteview = "0.9.1"
 color-eyre = "0.6.5"
 derive_deref = "1.1.1"
-fjall = "2.11.0"
-jiff = "0.2.14"
-log = { version = "0.4.27" }
-minreq = { version = "2.13.4", features = ["https", "serde_json"] }
-rayon = "1.10.0"
-serde = { version = "1.0.219" }
-serde_bytes = "0.11.17"
-serde_derive = "1.0.219"
-serde_json = { version = "1.0.140", features = ["float_roundtrip"] }
-tabled = "0.20.0"
-tokio = { version = "1.45.1", features = ["rt-multi-thread"] }
-zerocopy = { version = "0.8.25" }
-zerocopy-derive = "0.8.25"
+fjall = "3.0.0-rc.6"
+jiff = "0.2.16"
+log = "0.4.29"
+mimalloc = { version = "0.1.48", features = ["v3"] }
+minreq = { version = "2.14.1", features = ["https", "serde_json"] }
+parking_lot = "0.12.5"
+rayon = "1.11.0"
+rustc-hash = "2.1.1"
+schemars = "1.1.0"
+serde = "1.0.228"
+serde_bytes = "0.11.19"
+serde_derive = "1.0.228"
+serde_json = { version = "1.0.145", features = ["float_roundtrip"] }
+smallvec = "1.15.1"
+tokio = { version = "1.48.0", features = ["rt-multi-thread"] }
+vecdb = { version = "0.4.4", features = ["derive", "serde_json", "pco"] }
+# vecdb = { path = "../anydb/crates/vecdb", features = ["derive", "serde_json", "pco", "schemars"] }
+# vecdb = { git = "https://github.com/anydb-rs/anydb", features = ["derive", "serde_json", "pco"] }
 
 [workspace.metadata.release]
 shared-version = true
@@ -61,20 +90,8 @@ pre-release-commit-message = "release: v{{version}}"
 tag-message = "release: v{{version}}"
 
 [workspace.metadata.dist]
-cargo-dist-version = "0.28.0"
+cargo-dist-version = "0.30.2"
 ci = "github"
+allow-dirty = ["ci"]
 installers = []
-targets = [
-    "aarch64-apple-darwin",
-    "aarch64-unknown-linux-gnu",
-    "x86_64-apple-darwin",
-    "x86_64-unknown-linux-gnu",
-]
-
-[workspace.metadata.dist.github-custom-runners]
-global = "ubuntu-latest"
-aarch64-apple-darwin.runner = "macos-14"
-x86_64-unknown-linux-gnu.runner = "ubuntu-latest"
-x86_64-unknown-linux-gnu.container = { image = "quay.io/pypa/manylinux_2_28_x86_64", host = "x86_64-unknown-linux-musl" }
-aarch64-unknown-linux-gnu.runner = "ubuntu-latest"
-aarch64-unknown-linux-gnu.container = { image = "quay.io/pypa/manylinux_2_28_x86_64", host = "x86_64-unknown-linux-musl" }
+targets = ["aarch64-apple-darwin", "aarch64-unknown-linux-gnu", "x86_64-unknown-linux-gnu"]

README.md

@@ -1,102 +0,0 @@
-# Bitcoin Research Kit
-
-<p align="left">
-  <a href="https://github.com/bitcoinresearchkit/brk">
-    <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
-  </a>
-  <a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
-    <img src="https://img.shields.io/crates/l/brk" alt="License" />
-  </a>
-  <a href="https://crates.io/crates/brk">
-    <img src="https://img.shields.io/crates/v/brk" alt="Version" />
-  </a>
-  <a href="https://docs.rs/brk">
-    <img src="https://img.shields.io/docsrs/brk" alt="Documentation" />
-  </a>
-  <img src="https://img.shields.io/crates/size/brk" alt="Size" />
-  <a href="https://deps.rs/crate/brk">
-    <img src="https://deps.rs/crate/brk/latest/status.svg" alt="Dependency status">
-  </a>
-  <a href="https://discord.gg/HaR3wpH3nr">
-    <img src="https://img.shields.io/discord/1350431684562124850?label=discord" alt="Discord" />
-  </a>
-  <a href="https://primal.net/p/nprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6">
-    <img src="https://img.shields.io/badge/nostr-purple?link=https%3A%2F%2Fprimal.net%2Fp%2Fnprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6" alt="Nostr" />
-  </a>
-  <a href="https://bsky.app/profile/bitcoinresearchkit.org">
-    <img src="https://img.shields.io/badge/bluesky-blue?link=https%3A%2F%2Fbsky.app%2Fprofile%2Fbitcoinresearchkit.org" alt="Bluesky" />
-  </a>
-  <a href="https://x.com/brkdotorg">
-    <img src="https://img.shields.io/badge/x.com-black" alt="X" />
-  </a>
-</p>
-
-The Bitcoin Research Kit is a high-performance toolchain designed to parse, index, compute, serve and visualize data from a Bitcoin Core node, enabling users to gain deeper insights into the Bitcoin network.
-
-In other words it's an alternative to [Glassnode](https://glassnode.com), [mempool.space](https://mempool.space/) (soon) and [electrs](https://github.com/romanz/electrs) (soon) all in one package with a particular focus on simplicity and the self-hosting experience.
-
-The toolkit can be used in various ways to accommodate as many needs as possible:
-
-- **[Website](https://kibo.money)** \
-  Everyone is welcome to visit [kibo.money](https://kibo.money) which is the official showcase of the suite's capabilities and served by default when running BRK. \
-  Researchers and developers are free to use the API which endpoints documentation can be found [here](https://github.com/bitcoinresearchkit/brk/tree/main/crates/brk_server#endpoints). \
-  As a token of gratitude to the community and to stimulate curiosity, both the website and the API are entirely free, allowing anyone to use them.
-- **[CLI](https://crates.io/crates/brk_cli)** \
-  Node runners are strongly encouraged to try out and self-host their own instance. \
-  A lot of effort has gone into making this as easy as possible. \
-  For more information visit: [`brk_cli`](https://crates.io/crates/brk_cli)
-- **[Crates](https://crates.io/crates/brk)** \
-  Rust developers have access to a wide range crates, each built upon one another with its own specific purpose, enabling independent use and offering great flexibility.
-  PRs are welcome, especially if their goal is to introduce additional datasets.
-
-The primary goal of this project is to be fully-featured and accessible for everyone, regardless of their background or financial situation - whether that person is an enthusiast, researcher, miner, analyst, or simply curious.
-
-In contrast, existing alternatives tend to be either [very costly](https://studio.glassnode.com/pricing) or missing essential features, with the vast majority being closed-source and unverifiable, which fundamentally undermines the principles of Bitcoin.
-
-## Crates
-
-- [`brk`](https://crates.io/crates/brk): Wrapper around all other `brk-*` crates
-- [`brk_cli`](https://crates.io/crates/brk_cli): A standalone command line interface to interact with the Bitcoin Research Kit
-- [`brk_computer`](https://crates.io/crates/brk_computer): A Bitcoin dataset computer, built on top of brk_indexer
-- [`brk_core`](https://crates.io/crates/brk_core): The Core (Structs and Errors) of the Bitcoin Research Kit
-- [`brk_exit`](https://crates.io/crates/brk_exit): An exit blocker built on top of ctrlc
-- [`brk_fetcher`](https://crates.io/crates/brk_fetcher): A Bitcoin price fetcher
-- [`brk_indexer`](https://crates.io/crates/brk_indexer): A Bitcoin Core indexer built on top of brk_parser
-- [`brk_logger`](https://crates.io/crates/brk_logger): A clean logger used in the Bitcoin Research Kit.
-- [`brk_parser`](https://crates.io/crates/brk_parser): A very fast Bitcoin Core block parser and iterator built on top of bitcoin-rust
-- [`brk_query`](https://crates.io/crates/brk_query): A library that finds requested datasets.
-- [`brk_server`](https://crates.io/crates/brk_server): A server that serves Bitcoin data and swappable front-ends, built on top of `brk_indexer`, `brk_fetcher` and `brk_computer`
-- [`brk_state`](https://crates.io/crates/brk_state): Various states used mainly by the computer
-- [`brk_store`](https://crates.io/crates/brk_store): A thin wrapper around [`fjall`](https://crates.io/crates/fjall)
-- [`brk_vec`](https://crates.io/crates/brk_vec): A push-only, truncable, compressable, saveable Vec
-
-## Acknowledgments
-
-Deepest gratitude to the [Open Sats](https://opensats.org/) public charity. Their grant — from December 2024 to the present — has been critical in sustaining this project.
-
-Heartfelt thanks go out to every donor on [Nostr](https://primal.net/p/npub1jagmm3x39lmwfnrtvxcs9ac7g300y3dusv9lgzhk2e4x5frpxlrqa73v44) and [Geyser.fund](https://geyser.fund/project/brk) whose support has ensured the availability of the [kibo.money](https://kibo.money) public instance.
-
-## Hosting as a service
-
-*Soon™*
-
-If you'd like to have your own instance hosted for you please contact [hosting@bitcoinresearchkit.org](mailto:hosting@bitcoinresearchkit.org).
-
-- 2 separate dedicated servers (1 GB/s each) with different ISPs and Cloudflare integration for enhanced performance and optimal availability
-- 99.99% SLA
-- Configurated for speed (`raw + eager`)
-- Updates delivered at your convenience
-- Direct communication for feature requests and support
-- Bitcoin Core or Knots with desired version
-- Optional subdomains: `*.bitcoinresearchkit.org`, `*.kibo.money` and `*.satonomics.xyz`
-- Logo featured in the Readme if desired
-
-Pricing: `0.01 BTC / month` *or* `0.1 BTC / year`
-
-## Donate
-
-[`bc1q09 8zsm89 m7kgyz e338vf ejhpdt 92ua9p 3peuve`](bitcoin:bc1q098zsm89m7kgyze338vfejhpdt92ua9p3peuve)
-
-[`lnurl1dp68gurn8ghj7ampd3kx2ar0veekzar0wd5xjtnrdakj7tnhv4kxctttdehhwm30d3h82unvwqhkxmmww3jkuar8d35kgetj8yuq363hv4`](lightning:lnurl1dp68gurn8ghj7ampd3kx2ar0veekzar0wd5xjtnrdakj7tnhv4kxctttdehhwm30d3h82unvwqhkxmmww3jkuar8d35kgetj8yuq363hv4)
-
-[Geyser Fund](https://geyser.fund/project/brk)

crates/brk/Cargo.toml

@@ -2,54 +2,74 @@
 name = "brk"
 description.workspace = true
 license.workspace = true
-readme.workspace = true
 homepage.workspace = true
 repository.workspace = true
 edition.workspace = true
 version.workspace = true
+build = "build.rs"
 
 [features]
 full = [
-    "core",
+    "bencher",
+    "binder",
+    "bundler",
     "computer",
-    "exit",
+    "error",
     "fetcher",
+    "grouper",
     "indexer",
+    "iterator",
     "logger",
-    "parser",
+    "mcp",
+    "mempool",
     "query",
+    "reader",
+    "rpc",
     "server",
-    "state",
     "store",
-    "vec",
+    "traversable",
+    "types",
 ]
-core = ["brk_core"]
+bencher = ["brk_bencher"]
+binder = ["brk_binder"]
+bundler = ["brk_bundler"]
 computer = ["brk_computer"]
-exit = ["brk_exit"]
+error = ["brk_error"]
 fetcher = ["brk_fetcher"]
+grouper = ["brk_grouper"]
 indexer = ["brk_indexer"]
+iterator = ["brk_iterator"]
 logger = ["brk_logger"]
-parser = ["brk_parser"]
+mcp = ["brk_mcp"]
+mempool = ["brk_mempool"]
 query = ["brk_query"]
+reader = ["brk_reader"]
+rpc = ["brk_rpc"]
 server = ["brk_server"]
-state = ["brk_state"]
 store = ["brk_store"]
-vec = ["brk_vec"]
+traversable = ["brk_traversable"]
+types = ["brk_types"]
 
 [dependencies]
-brk_cli = { workspace = true }
-brk_core = { workspace = true, optional = true }
+brk_bencher = { workspace = true, optional = true }
+brk_binder = { workspace = true, optional = true }
+brk_bundler = { workspace = true, optional = true }
 brk_computer = { workspace = true, optional = true }
-brk_exit = { workspace = true, optional = true }
+brk_error = { workspace = true, optional = true }
 brk_fetcher = { workspace = true, optional = true }
+brk_grouper = { workspace = true, optional = true }
 brk_indexer = { workspace = true, optional = true }
+brk_iterator = { workspace = true, optional = true }
 brk_logger = { workspace = true, optional = true }
-brk_parser = { workspace = true, optional = true }
+brk_mcp = { workspace = true, optional = true }
+brk_mempool = { workspace = true, optional = true }
 brk_query = { workspace = true, optional = true }
+brk_reader = { workspace = true, optional = true }
+brk_rpc = { workspace = true, optional = true }
 brk_server = { workspace = true, optional = true }
-brk_state = { workspace = true, optional = true }
 brk_store = { workspace = true, optional = true }
-brk_vec = { workspace = true, optional = true }
+brk_traversable = { workspace = true, optional = true }
+brk_types = { workspace = true, optional = true }
 
 [package.metadata.docs.rs]
 all-features = true

crates/brk/README.md

@@ -0,0 +1,42 @@
+# brk
+
+Umbrella crate for the Bitcoin Research Kit.
+
+## What It Enables
+
+Single dependency to access any BRK component. Enable only what you need via feature flags.
+
+## Usage
+
+```toml
+[dependencies]
+brk = { version = "0.x", features = ["query", "types"] }
+```
+
+```rust,ignore
+use brk::query::Query;
+use brk::types::Height;
+```
+
+## Feature Flags
+
+| Feature | Crate | Description |
+|---------|-------|-------------|
+| `binder` | `brk_binder` | Client code generation |
+| `bundler` | `brk_bundler` | JS bundling |
+| `computer` | `brk_computer` | Metric computation |
+| `error` | `brk_error` | Error types |
+| `fetcher` | `brk_fetcher` | Price data fetching |
+| `grouper` | `brk_grouper` | Cohort filtering |
+| `indexer` | `brk_indexer` | Blockchain indexing |
+| `iterator` | `brk_iterator` | Block iteration |
+| `logger` | `brk_logger` | Logging setup |
+| `mcp` | `brk_mcp` | MCP server |
+| `mempool` | `brk_mempool` | Mempool monitoring |
+| `query` | `brk_query` | Query interface |
+| `reader` | `brk_reader` | Raw block reading |
+| `rpc` | `brk_rpc` | Bitcoin RPC client |
+| `server` | `brk_server` | HTTP API server |
+| `store` | `brk_store` | Key-value storage |
+| `traversable` | `brk_traversable` | Data traversal |
+| `types` | `brk_types` | Domain types |

crates/brk/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk/flamegraph.sh

@@ -0,0 +1 @@
+sudo cargo flamegraph --profile profiling --root

crates/brk/samply.sh

@@ -0,0 +1,2 @@
+cargo build --profile profiling
+samply record ../../target/profiling/brk

crates/brk/src/lib.rs

@@ -1,49 +1,77 @@
-#![doc = include_str!(concat!("../", env!("CARGO_PKG_README")))]
+#![doc = include_str!("../README.md")]
 
-#[cfg(feature = "core")]
+#[cfg(feature = "bencher")]
 #[doc(inline)]
-pub use brk_core as core;
+pub use brk_bencher as bencher;
+
+#[cfg(feature = "binder")]
+#[doc(inline)]
+pub use brk_binder as binder;
+
+#[cfg(feature = "bundler")]
+#[doc(inline)]
+pub use brk_bundler as bundler;
 
 #[cfg(feature = "computer")]
 #[doc(inline)]
 pub use brk_computer as computer;
 
-#[cfg(feature = "exit")]
+#[cfg(feature = "error")]
 #[doc(inline)]
-pub use brk_exit as exit;
+pub use brk_error as error;
 
 #[cfg(feature = "fetcher")]
 #[doc(inline)]
 pub use brk_fetcher as fetcher;
 
+#[cfg(feature = "grouper")]
+#[doc(inline)]
+pub use brk_grouper as grouper;
+
 #[cfg(feature = "indexer")]
 #[doc(inline)]
 pub use brk_indexer as indexer;
 
+#[cfg(feature = "iterator")]
+#[doc(inline)]
+pub use brk_iterator as iterator;
+
 #[cfg(feature = "logger")]
 #[doc(inline)]
 pub use brk_logger as logger;
 
-#[cfg(feature = "parser")]
+#[cfg(feature = "mcp")]
 #[doc(inline)]
-pub use brk_parser as parser;
+pub use brk_mcp as mcp;
+
+#[cfg(feature = "mempool")]
+#[doc(inline)]
+pub use brk_mempool as mempool;
 
 #[cfg(feature = "query")]
 #[doc(inline)]
 pub use brk_query as query;
 
+#[cfg(feature = "reader")]
+#[doc(inline)]
+pub use brk_reader as reader;
+
+#[cfg(feature = "rpc")]
+#[doc(inline)]
+pub use brk_rpc as rpc;
+
 #[cfg(feature = "server")]
 #[doc(inline)]
 pub use brk_server as server;
 
-#[cfg(feature = "state")]
-#[doc(inline)]
-pub use brk_state as state;
-
 #[cfg(feature = "store")]
 #[doc(inline)]
 pub use brk_store as store;
 
-#[cfg(feature = "vec")]
+#[cfg(feature = "traversable")]
 #[doc(inline)]
-pub use brk_vec as vec;
+pub use brk_traversable as traversable;
+
+#[cfg(feature = "types")]
+#[doc(inline)]
+pub use brk_types as types;

crates/brk/src/main.rs

@@ -1 +0,0 @@
-use brk_cli::main;

crates/brk_bencher/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "brk_bencher"
+description = "A simple benchmarker for testing other crates."
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+brk_error = { workspace = true }
+brk_logger = { workspace = true }
+parking_lot = { workspace = true }
+
+[target.'cfg(target_os = "macos")'.dependencies]
+libproc = "0.14"

crates/brk_bencher/README.md

@@ -0,0 +1,43 @@
+# brk_bencher
+
+Resource monitoring for long-running Bitcoin indexing operations.
+
+## What It Enables
+
+Track disk usage, memory consumption (current + peak), and I/O throughput during indexing runs. Progress tracking hooks into brk_logger to record processing milestones automatically.
+
+## Key Features
+
+- **Multi-metric monitoring**: Disk, memory (RSS + peak), I/O read/write
+- **Progress tracking**: Integrates with logging to capture block heights as they're processed
+- **Run comparison**: Outputs timestamped CSVs for comparing multiple runs
+- **macOS optimized**: Uses libproc for accurate process metrics on macOS
+- **Non-blocking**: Monitors in background thread with 5-second sample interval
+
+## Core API
+
+```rust,ignore
+let mut bencher = Bencher::from_cargo_env("brk_indexer", &data_path)?;
+bencher.start()?;
+
+// ... run indexing ...
+
+bencher.stop()?;
+```
+
+## Output Structure
+
+```
+benches/
+└── brk_indexer/
+    └── 1703001234/
+        ├── disk.csv      # timestamp_ms, bytes
+        ├── memory.csv    # timestamp_ms, current, peak
+        ├── io.csv        # timestamp_ms, read, written
+        └── progress.csv  # timestamp_ms, height
+```
+
+## Built On
+
+- `brk_error` for error handling
+- `brk_logger` for progress hook integration

crates/brk_bencher/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bencher/src/disk.rs

@@ -0,0 +1,64 @@
+use std::collections::HashMap;
+use std::fs::{self, File};
+use std::io::{self, Write};
+use std::os::unix::fs::MetadataExt;
+use std::path::{Path, PathBuf};
+use std::time::SystemTime;
+
+pub struct DiskMonitor {
+    cache: HashMap<PathBuf, (u64, SystemTime)>, // path -> (bytes_used, mtime)
+    monitored_path: PathBuf,
+    writer: File,
+}
+
+impl DiskMonitor {
+    pub fn new(monitored_path: &Path, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,disk_usage")?;
+
+        Ok(Self {
+            cache: HashMap::new(),
+            monitored_path: monitored_path.to_path_buf(),
+            writer,
+        })
+    }
+
+    /// Record disk usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok(bytes) = self.scan_recursive(&self.monitored_path.clone()) {
+            writeln!(self.writer, "{},{}", elapsed_ms, bytes)?;
+        }
+        Ok(())
+    }
+
+    fn scan_recursive(&mut self, path: &Path) -> io::Result<u64> {
+        let mut total = 0;
+
+        for entry in fs::read_dir(path)? {
+            let entry = entry?;
+            let path = entry.path();
+            let metadata = entry.metadata()?;
+
+            if metadata.is_file() {
+                let mtime = metadata.modified()?;
+
+                // Check cache: if mtime unchanged, use cached value
+                if let Some((cached_bytes, cached_mtime)) = self.cache.get(&path)
+                    && *cached_mtime == mtime
+                {
+                    total += cached_bytes;
+                    continue;
+                }
+
+                // File is new or modified - get actual disk usage
+                let bytes = metadata.blocks() * 512;
+                self.cache.insert(path, (bytes, mtime));
+                total += bytes;
+            } else if metadata.is_dir() {
+                total += self.scan_recursive(&path)?;
+            }
+        }
+
+        Ok(total)
+    }
+}

crates/brk_bencher/src/io.rs

@@ -0,0 +1,81 @@
+use std::fs::File;
+use std::io::{self, Write};
+use std::path::Path;
+
+#[cfg(target_os = "linux")]
+use std::fs;
+
+#[cfg(target_os = "macos")]
+use libproc::pid_rusage::{pidrusage, RUsageInfoV2};
+
+pub struct IoMonitor {
+    pid: u32,
+    writer: File,
+}
+
+impl IoMonitor {
+    pub fn new(pid: u32, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,bytes_read,bytes_written")?;
+
+        Ok(Self { pid, writer })
+    }
+
+    /// Record I/O usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok((read, written)) = self.get_io_usage() {
+            writeln!(self.writer, "{},{},{}", elapsed_ms, read, written)?;
+        }
+        Ok(())
+    }
+
+    /// Get I/O usage in bytes
+    /// Returns (bytes_read, bytes_written)
+    fn get_io_usage(&self) -> io::Result<(u64, u64)> {
+        #[cfg(target_os = "linux")]
+        {
+            self.get_io_usage_linux()
+        }
+
+        #[cfg(target_os = "macos")]
+        {
+            self.get_io_usage_macos()
+        }
+    }
+
+    #[cfg(target_os = "linux")]
+    fn get_io_usage_linux(&self) -> io::Result<(u64, u64)> {
+        let io_content = fs::read_to_string(format!("/proc/{}/io", self.pid))?;
+
+        let mut read_bytes = None;
+        let mut write_bytes = None;
+
+        for line in io_content.lines() {
+            if line.starts_with("read_bytes:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    read_bytes = value_str.parse::<u64>().ok();
+                }
+            } else if line.starts_with("write_bytes:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    write_bytes = value_str.parse::<u64>().ok();
+                }
+            }
+        }
+
+        match (read_bytes, write_bytes) {
+            (Some(r), Some(w)) => Ok((r, w)),
+            _ => Err(io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse I/O stats from /proc/[pid]/io",
+            )),
+        }
+    }
+
+    #[cfg(target_os = "macos")]
+    fn get_io_usage_macos(&self) -> io::Result<(u64, u64)> {
+        match pidrusage::<RUsageInfoV2>(self.pid as i32) {
+            Ok(info) => Ok((info.ri_diskio_bytesread, info.ri_diskio_byteswritten)),
+            Err(_) => Err(io::Error::other("Failed to get process I/O stats")),
+        }
+    }
+}

crates/brk_bencher/src/lib.rs

@@ -0,0 +1,160 @@
+use std::{
+    fs,
+    path::{Path, PathBuf},
+    sync::{
+        Arc,
+        atomic::{AtomicBool, Ordering},
+    },
+    thread::{self, JoinHandle},
+    time::{Duration, Instant, SystemTime, UNIX_EPOCH},
+};
+
+use brk_error::{Error, Result};
+
+mod disk;
+mod io;
+mod memory;
+mod progression;
+
+use disk::*;
+use io::*;
+use memory::*;
+use parking_lot::Mutex;
+use progression::*;
+
+#[derive(Clone)]
+pub struct Bencher(Arc<BencherInner>);
+
+struct BencherInner {
+    bench_dir: PathBuf,
+    monitored_path: PathBuf,
+    stop_flag: Arc<AtomicBool>,
+    monitor_thread: Mutex<Option<JoinHandle<Result<()>>>>,
+    progression: Arc<ProgressionMonitor>,
+}
+
+impl Bencher {
+    /// Create a new bencher for the given crate name
+    /// Creates directory structure: workspace_root/benches/{crate_name}/{timestamp}/
+    pub fn new(crate_name: &str, workspace_root: &Path, monitored_path: &Path) -> Result<Self> {
+        let timestamp = SystemTime::now().duration_since(UNIX_EPOCH)?.as_secs();
+
+        let bench_dir = workspace_root
+            .join("benches")
+            .join(crate_name)
+            .join(timestamp.to_string());
+
+        fs::create_dir_all(&bench_dir)?;
+
+        let progress_csv = bench_dir.join("progress.csv");
+        let progression = Arc::new(ProgressionMonitor::new(&progress_csv)?);
+        let progression_clone = progression.clone();
+
+        // Register hook with logger
+        brk_logger::register_hook(move |message| {
+            progression_clone.check_and_record(message);
+        })
+        .map_err(|e| std::io::Error::new(std::io::ErrorKind::AlreadyExists, e))?;
+
+        Ok(Self(Arc::new(BencherInner {
+            bench_dir,
+            monitored_path: monitored_path.to_path_buf(),
+            stop_flag: Arc::new(AtomicBool::new(false)),
+            progression,
+            monitor_thread: Mutex::new(None),
+        })))
+    }
+
+    /// Create a bencher using CARGO_MANIFEST_DIR to find workspace root
+    pub fn from_cargo_env(crate_name: &str, monitored_path: &Path) -> Result<Self> {
+        let mut current = std::env::current_dir()
+            .map_err(|e| format!("Failed to get current directory: {}", e))
+            .unwrap();
+
+        let workspace_root = loop {
+            let cargo_toml = current.join("Cargo.toml");
+            if cargo_toml.exists() {
+                let contents = std::fs::read_to_string(&cargo_toml)
+                    .map_err(|e| format!("Failed to read Cargo.toml: {}", e))
+                    .unwrap();
+                if contents.contains("[workspace]") {
+                    break current;
+                }
+            }
+
+            current = current
+                .parent()
+                .ok_or(Error::NotFound("Workspace root not found".into()))?
+                .to_path_buf();
+        };
+
+        Self::new(crate_name, &workspace_root, monitored_path)
+    }
+
+    /// Start monitoring disk usage and memory footprint
+    pub fn start(&mut self) -> Result<()> {
+        if self.0.monitor_thread.lock().is_some() {
+            return Err(Error::Internal("Bencher already started"));
+        }
+
+        let stop_flag = self.0.stop_flag.clone();
+        let bench_dir = self.0.bench_dir.clone();
+        let monitored_path = self.0.monitored_path.clone();
+
+        let handle =
+            thread::spawn(move || monitor_resources(&monitored_path, &bench_dir, stop_flag));
+
+        *self.0.monitor_thread.lock() = Some(handle);
+        Ok(())
+    }
+
+    /// Stop monitoring and wait for the thread to finish
+    pub fn stop(&self) -> Result<()> {
+        self.0.stop_flag.store(true, Ordering::Relaxed);
+
+        if let Some(handle) = self.0.monitor_thread.lock().take() {
+            handle.join().map_err(|_| Error::Internal("Monitor thread panicked"))??;
+        }
+
+        self.0.progression.flush()?;
+
+        Ok(())
+    }
+}
+
+impl Drop for Bencher {
+    fn drop(&mut self) {
+        let _ = self.stop();
+    }
+}
+
+fn monitor_resources(
+    monitored_path: &Path,
+    bench_dir: &Path,
+    stop_flag: Arc<AtomicBool>,
+) -> Result<()> {
+    let pid = std::process::id();
+    let start = Instant::now();
+
+    let mut disk_monitor = DiskMonitor::new(monitored_path, &bench_dir.join("disk.csv"))?;
+    let mut memory_monitor = MemoryMonitor::new(pid, &bench_dir.join("memory.csv"))?;
+    let mut io_monitor = IoMonitor::new(pid, &bench_dir.join("io.csv"))?;
+
+    'l: loop {
+        let elapsed_ms = start.elapsed().as_millis();
+
+        disk_monitor.record(elapsed_ms)?;
+        memory_monitor.record(elapsed_ms)?;
+        io_monitor.record(elapsed_ms)?;
+
+        for _ in 0..50 {
+            // 50 * 100ms = 5 seconds
+            if stop_flag.load(Ordering::Relaxed) {
+                break 'l;
+            }
+            thread::sleep(Duration::from_millis(100));
+        }
+    }
+
+    Ok(())
+}

crates/brk_bencher/src/memory.rs

@@ -0,0 +1,141 @@
+use std::fs::File;
+use std::io::{self, Write};
+use std::path::Path;
+
+#[cfg(target_os = "linux")]
+use std::fs;
+
+#[cfg(target_os = "macos")]
+use std::process::Command;
+
+pub struct MemoryMonitor {
+    pid: u32,
+    writer: File,
+}
+
+impl MemoryMonitor {
+    pub fn new(pid: u32, csv_path: &Path) -> io::Result<Self> {
+        let mut writer = File::create(csv_path)?;
+        writeln!(writer, "timestamp_ms,phys_footprint,phys_footprint_peak")?;
+
+        Ok(Self { pid, writer })
+    }
+
+    /// Record memory usage at the given timestamp
+    pub fn record(&mut self, elapsed_ms: u128) -> io::Result<()> {
+        if let Ok((footprint, peak)) = self.get_memory_usage() {
+            writeln!(self.writer, "{},{},{}", elapsed_ms, footprint, peak)?;
+        }
+        Ok(())
+    }
+
+    /// Get memory usage in bytes
+    /// Returns (current_bytes, peak_bytes)
+    fn get_memory_usage(&self) -> io::Result<(u64, u64)> {
+        #[cfg(target_os = "linux")]
+        {
+            self.get_memory_usage_linux()
+        }
+
+        #[cfg(target_os = "macos")]
+        {
+            self.get_memory_usage_macos()
+        }
+    }
+
+    #[cfg(target_os = "linux")]
+    fn get_memory_usage_linux(&self) -> io::Result<(u64, u64)> {
+        let status_content = fs::read_to_string(format!("/proc/{}/status", self.pid))?;
+
+        let mut vm_rss = None;
+        let mut vm_hwm = None;
+
+        for line in status_content.lines() {
+            if line.starts_with("VmRSS:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    if let Ok(kb) = value_str.parse::<u64>() {
+                        vm_rss = Some(kb * 1024); // KiB to bytes
+                    }
+                }
+            } else if line.starts_with("VmHWM:") {
+                if let Some(value_str) = line.split_whitespace().nth(1) {
+                    if let Ok(kb) = value_str.parse::<u64>() {
+                        vm_hwm = Some(kb * 1024); // KiB to bytes
+                    }
+                }
+            }
+        }
+
+        match (vm_rss, vm_hwm) {
+            (Some(rss), Some(hwm)) => Ok((rss, hwm)),
+            _ => Err(io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse memory info from /proc/[pid]/status",
+            )),
+        }
+    }
+
+    #[cfg(target_os = "macos")]
+    fn get_memory_usage_macos(&self) -> io::Result<(u64, u64)> {
+        let output = Command::new("footprint")
+            .args(["-p", &self.pid.to_string()])
+            .output()?;
+
+        let stdout = String::from_utf8(output.stdout).map_err(|_| {
+            io::Error::new(io::ErrorKind::InvalidData, "Invalid UTF-8 from footprint")
+        })?;
+
+        parse_footprint_output(&stdout).ok_or_else(|| {
+            io::Error::new(
+                io::ErrorKind::InvalidData,
+                "Failed to parse footprint output",
+            )
+        })
+    }
+}
+
+#[cfg(target_os = "macos")]
+fn parse_footprint_output(output: &str) -> Option<(u64, u64)> {
+    let mut phys_footprint = None;
+    let mut phys_footprint_peak = None;
+
+    for line in output.lines() {
+        let line = line.trim();
+
+        if line.starts_with("phys_footprint:") {
+            // Format: "phys_footprint: 7072 KB"
+            let parts: Vec<&str> = line.split_whitespace().collect();
+            if parts.len() >= 3 {
+                // parts[0] = "phys_footprint:"
+                // parts[1] = "7072"
+                // parts[2] = "KB"
+                phys_footprint = parse_size_to_bytes(parts[1], parts[2]);
+            }
+        } else if line.starts_with("phys_footprint_peak:") {
+            // Format: "phys_footprint_peak: 15 MB"
+            let parts: Vec<&str> = line.split_whitespace().collect();
+            if parts.len() >= 3 {
+                phys_footprint_peak = parse_size_to_bytes(parts[1], parts[2]);
+            }
+        }
+    }
+
+    match (phys_footprint, phys_footprint_peak) {
+        (Some(f), Some(p)) => Some((f, p)),
+        _ => None,
+    }
+}
+
+#[cfg(target_os = "macos")]
+fn parse_size_to_bytes(value: &str, unit: &str) -> Option<u64> {
+    let value: f64 = value.parse().ok()?;
+
+    let multiplier = match unit.to_uppercase().as_str() {
+        "KB" => 1024.0,                   // KiB to bytes
+        "MB" => 1024.0 * 1024.0,          // MiB to bytes
+        "GB" => 1024.0 * 1024.0 * 1024.0, // GiB to bytes
+        _ => return None,
+    };
+
+    Some((value * multiplier) as u64)
+}

crates/brk_bencher/src/progression.rs

@@ -0,0 +1,74 @@
+use parking_lot::Mutex;
+use std::{
+    fs,
+    io::{self, BufWriter, Write},
+    path::Path,
+    time::Instant,
+};
+
+/// Patterns to match for progress tracking.
+const PROGRESS_PATTERNS: &[&str] = &[
+    "block ",    // "Indexing block 123..."
+    "chain at ", // "Processing chain at 456..."
+];
+
+pub struct ProgressionMonitor {
+    csv_file: Mutex<BufWriter<fs::File>>,
+    start_time: Instant,
+}
+
+impl ProgressionMonitor {
+    pub fn new(csv_path: &Path) -> io::Result<Self> {
+        let mut csv_file = BufWriter::new(fs::File::create(csv_path)?);
+        writeln!(csv_file, "timestamp_ms,value")?;
+
+        Ok(Self {
+            csv_file: Mutex::new(csv_file),
+            start_time: Instant::now(),
+        })
+    }
+
+    /// Check message for progress patterns and record if found
+    #[inline]
+    pub fn check_and_record(&self, message: &str) {
+        let Some(value) = parse_progress(message) else {
+            return;
+        };
+
+        if value % 10 != 0 {
+            return;
+        }
+
+        let elapsed_ms = self.start_time.elapsed().as_millis();
+        let _ = writeln!(self.csv_file.lock(), "{},{}", elapsed_ms, value);
+    }
+
+    pub fn flush(&self) -> io::Result<()> {
+        self.csv_file.lock().flush()
+    }
+}
+
+/// Parse progress value from message
+#[inline]
+fn parse_progress(message: &str) -> Option<u64> {
+    PROGRESS_PATTERNS
+        .iter()
+        .find_map(|pattern| parse_number_after(message, pattern))
+}
+
+/// Extract number immediately following the pattern
+#[inline]
+fn parse_number_after(message: &str, pattern: &str) -> Option<u64> {
+    let start = message.find(pattern)?;
+    let after = &message[start + pattern.len()..];
+
+    let end = after
+        .find(|c: char| !c.is_ascii_digit())
+        .unwrap_or(after.len());
+
+    if end == 0 {
+        return None;
+    }
+
+    after[..end].parse().ok()
+}

crates/brk_bencher_visualizer/Cargo.toml

@@ -0,0 +1,13 @@
+[package]
+name = "brk_bencher_visualizer"
+description = "A generator of charts for brk_bencher"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+publish = false
+build = "build.rs"
+
+[dependencies]
+plotters = "0.3.7"

crates/brk_bencher_visualizer/README.md

@@ -0,0 +1,34 @@
+# brk_bencher_visualizer
+
+SVG chart generation for benchmark visualization.
+
+## What It Enables
+
+Turn benchmark CSV data into publication-ready SVG charts showing disk usage, memory (current/peak), progress, and I/O over time. Compare multiple runs side-by-side with automatic color coding.
+
+## Key Features
+
+- **Multi-run comparison**: Overlay multiple benchmark runs with distinct colors
+- **Dual-axis charts**: Memory charts show both current and peak usage (solid vs dashed lines)
+- **Smart scaling**: Automatic unit conversion for bytes (KB/MB/GB) and time (seconds/minutes/hours)
+- **Per-run trimming**: Aligns data by progress cutoffs for fair comparison
+- **Dark theme**: Clean, readable charts with monospace fonts
+
+## Core API
+
+```rust,ignore
+let viz = Visualizer::from_cargo_env()?;
+viz.generate_all_charts()?;  // Process all crates in benches/
+```
+
+## Chart Types
+
+- `disk.svg` - Storage consumption over time
+- `memory.svg` - Current + peak memory usage
+- `progress.svg` - Processing progress (e.g., blocks indexed)
+- `io_read.svg` / `io_write.svg` - I/O throughput
+
+## Input Format
+
+Reads CSV files from `benches/<crate>/<run_id>/`:
+- `disk.csv`, `memory.csv`, `progress.csv`, `io.csv`

crates/brk_bencher_visualizer/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bencher_visualizer/src/chart.rs

@@ -0,0 +1,251 @@
+use crate::data::{DataPoint, DualRun, Result, Run};
+use crate::format;
+use plotters::prelude::*;
+use std::path::Path;
+
+const FONT: &str = "monospace";
+const FONT_SIZE: i32 = 20;
+const FONT_SIZE_BIG: i32 = 30;
+const SIZE: (u32, u32) = (2000, 1000);
+const TIME_BUFFER_MS: u64 = 10_000;
+
+const BG_COLOR: RGBColor = RGBColor(18, 18, 24);
+const TEXT_COLOR: RGBColor = RGBColor(230, 230, 240);
+const COLORS: [RGBColor; 6] = [
+    RGBColor(255, 99, 132),  // Pink/Red
+    RGBColor(54, 162, 235),  // Blue
+    RGBColor(75, 192, 192),  // Teal
+    RGBColor(255, 206, 86),  // Yellow
+    RGBColor(153, 102, 255), // Purple
+    RGBColor(255, 159, 64),  // Orange
+];
+
+pub enum YAxisFormat {
+    Bytes,
+    Number,
+}
+
+pub struct ChartConfig<'a> {
+    pub output_path: &'a Path,
+    pub title: String,
+    pub y_label: String,
+    pub y_format: YAxisFormat,
+}
+
+/// Generate a simple line chart from runs
+pub fn generate(config: ChartConfig, runs: &[Run]) -> Result<()> {
+    if runs.is_empty() {
+        return Ok(());
+    }
+
+    let max_time_ms = runs.iter().map(|r| r.max_timestamp()).max().unwrap_or(1000) + TIME_BUFFER_MS;
+    let max_time_s = max_time_ms as f64 / 1000.0;
+    let max_value = runs.iter().map(|r| r.max_value()).fold(0.0, f64::max);
+
+    let (time_scaled, time_divisor, time_label) = format::time(max_time_s);
+    let (value_scaled, scale_factor, y_label) = scale_y_axis(max_value, &config.y_label, &config.y_format);
+    let x_labels = label_count(time_scaled);
+
+    let root = SVGBackend::new(config.output_path, SIZE).into_drawing_area();
+    root.fill(&BG_COLOR)?;
+
+    let mut chart = ChartBuilder::on(&root)
+        .caption(&config.title, (FONT, FONT_SIZE_BIG).into_font().color(&TEXT_COLOR))
+        .margin(20)
+        .margin_right(40)
+        .x_label_area_size(50)
+        .margin_left(50)
+        .right_y_label_area_size(75)
+        .build_cartesian_2d(0.0..time_scaled * 1.025, 0.0..value_scaled * 1.1)?;
+
+    configure_mesh(&mut chart, time_label, &y_label, &config.y_format, x_labels)?;
+
+    for (idx, run) in runs.iter().enumerate() {
+        let color = COLORS[idx % COLORS.len()];
+        draw_series(&mut chart, &run.data, &run.id, color, time_divisor, scale_factor)?;
+    }
+
+    configure_legend(&mut chart)?;
+    root.present()?;
+    println!("Generated: {}", config.output_path.display());
+    Ok(())
+}
+
+/// Generate a chart with dual series per run (e.g., current + peak memory)
+pub fn generate_dual(
+    config: ChartConfig,
+    runs: &[DualRun],
+    primary_suffix: &str,
+    secondary_suffix: &str,
+) -> Result<()> {
+    if runs.is_empty() {
+        return Ok(());
+    }
+
+    let max_time_ms = runs
+        .iter()
+        .flat_map(|r| r.primary.iter().chain(r.secondary.iter()))
+        .map(|d| d.timestamp_ms)
+        .max()
+        .unwrap_or(1000)
+        + TIME_BUFFER_MS;
+    let max_time_s = max_time_ms as f64 / 1000.0;
+    let max_value = runs.iter().map(|r| r.max_value()).fold(0.0, f64::max);
+
+    let (time_scaled, time_divisor, time_label) = format::time(max_time_s);
+    let (value_scaled, scale_factor, y_label) = scale_y_axis(max_value, &config.y_label, &config.y_format);
+    let x_labels = label_count(time_scaled);
+
+    let root = SVGBackend::new(config.output_path, SIZE).into_drawing_area();
+    root.fill(&BG_COLOR)?;
+
+    let mut chart = ChartBuilder::on(&root)
+        .caption(&config.title, (FONT, FONT_SIZE_BIG).into_font().color(&TEXT_COLOR))
+        .margin(20)
+        .margin_right(40)
+        .x_label_area_size(50)
+        .margin_left(50)
+        .right_y_label_area_size(75)
+        .build_cartesian_2d(0.0..time_scaled * 1.025, 0.0..value_scaled * 1.1)?;
+
+    configure_mesh(&mut chart, time_label, &y_label, &config.y_format, x_labels)?;
+
+    for (idx, run) in runs.iter().enumerate() {
+        let color = COLORS[idx % COLORS.len()];
+
+        // Primary series (solid)
+        draw_series(
+            &mut chart,
+            &run.primary,
+            &format!("{} {}", run.id, primary_suffix),
+            color,
+            time_divisor,
+            scale_factor,
+        )?;
+
+        // Secondary series (dashed)
+        draw_dashed_series(
+            &mut chart,
+            &run.secondary,
+            &format!("{} {}", run.id, secondary_suffix),
+            color.mix(0.5),
+            time_divisor,
+            scale_factor,
+        )?;
+    }
+
+    configure_legend(&mut chart)?;
+    root.present()?;
+    println!("Generated: {}", config.output_path.display());
+    Ok(())
+}
+
+fn scale_y_axis(max_value: f64, base_label: &str, y_format: &YAxisFormat) -> (f64, f64, String) {
+    match y_format {
+        YAxisFormat::Bytes => {
+            let (scaled, unit) = format::bytes(max_value);
+            let factor = max_value / scaled;
+            (scaled, factor, format!("{} ({})", base_label, unit))
+        }
+        YAxisFormat::Number => (max_value, 1.0, base_label.to_string()),
+    }
+}
+
+/// Calculate appropriate label count to avoid duplicates when rounding to integers
+fn label_count(max_value: f64) -> usize {
+    let max_int = max_value.ceil() as usize;
+    // Don't exceed the range, cap at 12 for readability
+    max_int.clamp(2, 12)
+}
+
+type Chart<'a, 'b> = ChartContext<
+    'a,
+    SVGBackend<'b>,
+    Cartesian2d<plotters::coord::types::RangedCoordf64, plotters::coord::types::RangedCoordf64>,
+>;
+
+fn configure_mesh(chart: &mut Chart, x_label: &str, y_label: &str, y_format: &YAxisFormat, x_labels: usize) -> Result<()> {
+    let y_formatter: Box<dyn Fn(&f64) -> String> = match y_format {
+        YAxisFormat::Bytes => Box::new(|y: &f64| {
+            if y.fract() == 0.0 {
+                format!("{:.0}", y)
+            } else {
+                format!("{:.1}", y)
+            }
+        }),
+        YAxisFormat::Number => Box::new(|y: &f64| format::axis_number(*y)),
+    };
+
+    chart
+        .configure_mesh()
+        .disable_mesh()
+        .x_desc(x_label)
+        .y_desc(y_label)
+        .x_label_formatter(&|x| format!("{:.0}", x))
+        .y_label_formatter(&y_formatter)
+        .x_labels(x_labels)
+        .y_labels(10)
+        .x_label_style((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.7)))
+        .y_label_style((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.7)))
+        .axis_style(TEXT_COLOR.mix(0.3))
+        .draw()?;
+    Ok(())
+}
+
+fn draw_series(
+    chart: &mut Chart,
+    data: &[DataPoint],
+    label: &str,
+    color: RGBColor,
+    time_divisor: f64,
+    scale_factor: f64,
+) -> Result<()> {
+    let points = data
+        .iter()
+        .map(|d| (d.timestamp_ms as f64 / 1000.0 / time_divisor, d.value / scale_factor));
+
+    chart
+        .draw_series(LineSeries::new(points, color.stroke_width(1)))?
+        .label(label)
+        .legend(move |(x, y)| PathElement::new(vec![(x, y), (x + 20, y)], color.stroke_width(1)));
+    Ok(())
+}
+
+fn draw_dashed_series(
+    chart: &mut Chart,
+    data: &[DataPoint],
+    label: &str,
+    color: RGBAColor,
+    time_divisor: f64,
+    scale_factor: f64,
+) -> Result<()> {
+    let points: Vec<_> = data
+        .iter()
+        .map(|d| (d.timestamp_ms as f64 / 1000.0 / time_divisor, d.value / scale_factor))
+        .collect();
+
+    // Draw dashed line by skipping every other segment
+    chart
+        .draw_series(
+            points
+                .windows(2)
+                .enumerate()
+                .filter(|(i, _)| i % 2 == 0)
+                .map(|(_, w)| PathElement::new(vec![w[0], w[1]], color.stroke_width(2))),
+        )?
+        .label(label)
+        .legend(move |(x, y)| PathElement::new(vec![(x, y), (x + 10, y), (x + 20, y)], color.stroke_width(2)));
+    Ok(())
+}
+
+fn configure_legend<'a>(chart: &mut Chart<'a, 'a>) -> Result<()> {
+    chart
+        .configure_series_labels()
+        .position(SeriesLabelPosition::UpperLeft)
+        .label_font((FONT, FONT_SIZE).into_font().color(&TEXT_COLOR.mix(0.9)))
+        .background_style(BG_COLOR.mix(0.98))
+        .border_style(BG_COLOR)
+        .margin(10)
+        .draw()?;
+    Ok(())
+}

crates/brk_bencher_visualizer/src/data.rs

@@ -0,0 +1,238 @@
+use std::{collections::HashMap, fs, path::Path};
+
+pub type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
+
+#[derive(Debug, Clone)]
+pub struct DataPoint {
+    pub timestamp_ms: u64,
+    pub value: f64,
+}
+
+/// Per-run cutoff timestamps for fair comparison
+pub struct Cutoffs {
+    by_id: HashMap<String, u64>,
+    default: u64,
+}
+
+impl Cutoffs {
+    /// Calculate cutoffs from progress runs.
+    /// Finds the common max progress, then returns when each run reached it.
+    pub fn from_progress(progress_runs: &[Run]) -> Self {
+        const TIME_BUFFER_MS: u64 = 10_000;
+
+        if progress_runs.is_empty() {
+            return Self {
+                by_id: HashMap::new(),
+                default: u64::MAX,
+            };
+        }
+
+        // Find the minimum of max progress values (the common point all runs reached)
+        let common_progress = progress_runs
+            .iter()
+            .map(|r| r.max_value())
+            .fold(f64::MAX, f64::min);
+
+        let by_id: HashMap<_, _> = progress_runs
+            .iter()
+            .map(|run| {
+                let cutoff = run
+                    .data
+                    .iter()
+                    .find(|d| d.value >= common_progress)
+                    .map(|d| d.timestamp_ms)
+                    .unwrap_or_else(|| run.max_timestamp())
+                    .saturating_add(TIME_BUFFER_MS);
+                (run.id.clone(), cutoff)
+            })
+            .collect();
+
+        let default = by_id.values().copied().max().unwrap_or(u64::MAX);
+
+        Self { by_id, default }
+    }
+
+    pub fn get(&self, id: &str) -> u64 {
+        self.by_id.get(id).copied().unwrap_or(self.default)
+    }
+
+    pub fn trim_runs(&self, runs: &[Run]) -> Vec<Run> {
+        runs.iter().map(|r| r.trimmed(self.get(&r.id))).collect()
+    }
+
+    pub fn trim_dual_runs(&self, runs: &[DualRun]) -> Vec<DualRun> {
+        runs.iter().map(|r| r.trimmed(self.get(&r.id))).collect()
+    }
+}
+
+#[derive(Debug, Clone)]
+pub struct Run {
+    pub id: String,
+    pub data: Vec<DataPoint>,
+}
+
+impl Run {
+    pub fn max_timestamp(&self) -> u64 {
+        self.data.iter().map(|d| d.timestamp_ms).max().unwrap_or(0)
+    }
+
+    pub fn max_value(&self) -> f64 {
+        self.data.iter().map(|d| d.value).fold(0.0, f64::max)
+    }
+
+    pub fn trimmed(&self, max_timestamp_ms: u64) -> Self {
+        Self {
+            id: self.id.clone(),
+            data: self
+                .data
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+        }
+    }
+}
+
+/// Two data series from a single run (e.g., memory footprint + peak, or io read + write)
+#[derive(Debug, Clone)]
+pub struct DualRun {
+    pub id: String,
+    pub primary: Vec<DataPoint>,
+    pub secondary: Vec<DataPoint>,
+}
+
+impl DualRun {
+    pub fn trimmed(&self, max_timestamp_ms: u64) -> Self {
+        Self {
+            id: self.id.clone(),
+            primary: self
+                .primary
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+            secondary: self
+                .secondary
+                .iter()
+                .filter(|d| d.timestamp_ms <= max_timestamp_ms)
+                .cloned()
+                .collect(),
+        }
+    }
+
+    pub fn max_value(&self) -> f64 {
+        self.primary
+            .iter()
+            .chain(self.secondary.iter())
+            .map(|d| d.value)
+            .fold(0.0, f64::max)
+    }
+}
+
+pub fn read_runs(crate_path: &Path, filename: &str) -> Result<Vec<Run>> {
+    let mut runs = Vec::new();
+
+    for entry in fs::read_dir(crate_path)? {
+        let run_path = entry?.path();
+        if !run_path.is_dir() {
+            continue;
+        }
+
+        let run_id = run_path
+            .file_name()
+            .and_then(|n| n.to_str())
+            .ok_or("Invalid run ID")?
+            .to_string();
+
+        // Skip underscore-prefixed or numeric-only directories
+        if run_id.starts_with('_') || run_id.chars().all(|c| c.is_ascii_digit()) {
+            continue;
+        }
+
+        let csv_path = run_path.join(filename);
+        if csv_path.exists()
+            && let Ok(data) = read_csv(&csv_path)
+        {
+            runs.push(Run { id: run_id, data });
+        }
+    }
+
+    Ok(runs)
+}
+
+pub fn read_dual_runs(crate_path: &Path, filename: &str) -> Result<Vec<DualRun>> {
+    let mut runs = Vec::new();
+
+    for entry in fs::read_dir(crate_path)? {
+        let run_path = entry?.path();
+        if !run_path.is_dir() {
+            continue;
+        }
+
+        let run_id = run_path
+            .file_name()
+            .and_then(|n| n.to_str())
+            .ok_or("Invalid run ID")?
+            .to_string();
+
+        if run_id.starts_with('_') || run_id.chars().all(|c| c.is_ascii_digit()) {
+            continue;
+        }
+
+        let csv_path = run_path.join(filename);
+        if csv_path.exists()
+            && let Ok((primary, secondary)) = read_dual_csv(&csv_path)
+        {
+            runs.push(DualRun {
+                id: run_id,
+                primary,
+                secondary,
+            });
+        }
+    }
+
+    Ok(runs)
+}
+
+fn read_csv(path: &Path) -> Result<Vec<DataPoint>> {
+    let content = fs::read_to_string(path)?;
+    let data = content
+        .lines()
+        .skip(1) // header
+        .filter_map(|line| {
+            let mut parts = line.split(',');
+            let timestamp_ms = parts.next()?.parse().ok()?;
+            let value = parts.next()?.parse().ok()?;
+            Some(DataPoint {
+                timestamp_ms,
+                value,
+            })
+        })
+        .collect();
+    Ok(data)
+}
+
+fn read_dual_csv(path: &Path) -> Result<(Vec<DataPoint>, Vec<DataPoint>)> {
+    let content = fs::read_to_string(path)?;
+    let mut primary = Vec::new();
+    let mut secondary = Vec::new();
+
+    for line in content.lines().skip(1) {
+        let mut parts = line.split(',');
+        if let (Some(ts), Some(v1), Some(v2)) = (parts.next(), parts.next(), parts.next())
+            && let (Ok(timestamp_ms), Ok(val1), Ok(val2)) =
+                (ts.parse(), v1.parse::<f64>(), v2.parse::<f64>())
+        {
+            primary.push(DataPoint {
+                timestamp_ms,
+                value: val1,
+            });
+            secondary.push(DataPoint {
+                timestamp_ms,
+                value: val2,
+            });
+        }
+    }
+
+    Ok((primary, secondary))
+}

crates/brk_bencher_visualizer/src/format.rs

@@ -0,0 +1,45 @@
+const KIB: f64 = 1024.0;
+const MIB: f64 = KIB * 1024.0;
+const GIB: f64 = MIB * 1024.0;
+
+const MINUTE: f64 = 60.0;
+const HOUR: f64 = 3600.0;
+
+/// Returns (scaled_value, unit_suffix)
+pub fn bytes(bytes: f64) -> (f64, &'static str) {
+    if bytes >= GIB {
+        (bytes / GIB, "GiB")
+    } else if bytes >= MIB {
+        (bytes / MIB, "MiB")
+    } else if bytes >= KIB {
+        (bytes / KIB, "KiB")
+    } else {
+        (bytes, "bytes")
+    }
+}
+
+/// Returns (scaled_value, divisor, axis_label)
+pub fn time(seconds: f64) -> (f64, f64, &'static str) {
+    if seconds >= HOUR * 2.0 {
+        (seconds / HOUR, HOUR, "Time (h)")
+    } else if seconds >= MINUTE * 2.0 {
+        (seconds / MINUTE, MINUTE, "Time (min)")
+    } else {
+        (seconds, 1.0, "Time (s)")
+    }
+}
+
+pub fn axis_number(value: f64) -> String {
+    if value >= 1000.0 {
+        let k = value / 1000.0;
+        if k.fract() == 0.0 || k >= 100.0 {
+            format!("{:.0}k", k)
+        } else if k >= 10.0 {
+            format!("{:.1}k", k)
+        } else {
+            format!("{:.2}k", k)
+        }
+    } else {
+        format!("{:.0}", value)
+    }
+}

crates/brk_bencher_visualizer/src/lib.rs

@@ -0,0 +1,249 @@
+mod chart;
+mod data;
+mod format;
+
+use data::{read_dual_runs, read_runs, Cutoffs, DualRun, Result, Run};
+use std::{
+    fs,
+    path::{Path, PathBuf},
+};
+
+pub struct Visualizer {
+    workspace_root: PathBuf,
+}
+
+impl Visualizer {
+    pub fn new(workspace_root: impl AsRef<Path>) -> Self {
+        Self {
+            workspace_root: workspace_root.as_ref().to_path_buf(),
+        }
+    }
+
+    pub fn from_cargo_env() -> Result<Self> {
+        let workspace_root = Path::new(env!("CARGO_MANIFEST_DIR"))
+            .parent()
+            .and_then(|p| p.parent())
+            .ok_or("Failed to find workspace root")?
+            .to_path_buf();
+        Ok(Self { workspace_root })
+    }
+
+    pub fn generate_all_charts(&self) -> Result<()> {
+        let benches_dir = self.workspace_root.join("benches");
+        if !benches_dir.exists() {
+            return Err("Benches directory does not exist".into());
+        }
+
+        for entry in fs::read_dir(&benches_dir)? {
+            let path = entry?.path();
+            if path.is_dir() {
+                let crate_name = path
+                    .file_name()
+                    .and_then(|n| n.to_str())
+                    .ok_or("Invalid crate name")?;
+
+                println!("Generating charts for crate: {}", crate_name);
+                self.generate_crate_charts(&path, crate_name)?;
+            }
+        }
+
+        Ok(())
+    }
+
+    fn generate_crate_charts(&self, crate_path: &Path, crate_name: &str) -> Result<()> {
+        let disk_runs = read_runs(crate_path, "disk.csv")?;
+        let memory_runs = read_dual_runs(crate_path, "memory.csv")?;
+        let progress_runs = read_runs(crate_path, "progress.csv")?;
+        let io_runs = read_dual_runs(crate_path, "io.csv")?;
+
+        // Combined charts (all runs)
+        self.generate_combined_charts(crate_path, crate_name, &disk_runs, &memory_runs, &progress_runs, &io_runs)?;
+
+        // Individual charts (one per run)
+        self.generate_individual_charts(crate_path, crate_name, &disk_runs, &memory_runs, &progress_runs, &io_runs)?;
+
+        Ok(())
+    }
+
+    fn generate_combined_charts(
+        &self,
+        crate_path: &Path,
+        crate_name: &str,
+        disk_runs: &[Run],
+        memory_runs: &[DualRun],
+        progress_runs: &[Run],
+        io_runs: &[DualRun],
+    ) -> Result<()> {
+        let cutoffs = Cutoffs::from_progress(progress_runs);
+
+        // Trim data to per-run cutoffs for fair comparison
+        let disk_trimmed = cutoffs.trim_runs(disk_runs);
+        let memory_trimmed = cutoffs.trim_dual_runs(memory_runs);
+        let io_trimmed = cutoffs.trim_dual_runs(io_runs);
+
+        if !disk_trimmed.is_empty() {
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("disk.svg"),
+                    title: format!("{} — Disk Usage", crate_name),
+                    y_label: "Disk Usage".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &disk_trimmed,
+            )?;
+        }
+
+        if !memory_trimmed.is_empty() {
+            chart::generate_dual(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("memory.svg"),
+                    title: format!("{} — Memory", crate_name),
+                    y_label: "Memory".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &memory_trimmed,
+                "(current)",
+                "(peak)",
+            )?;
+        }
+
+        if !progress_runs.is_empty() {
+            let progress_trimmed = cutoffs.trim_runs(progress_runs);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("progress.svg"),
+                    title: format!("{} — Progress", crate_name),
+                    y_label: "Progress".to_string(),
+                    y_format: chart::YAxisFormat::Number,
+                },
+                &progress_trimmed,
+            )?;
+        }
+
+        if !io_trimmed.is_empty() {
+            // I/O Read (primary column)
+            let io_read: Vec<_> = io_trimmed
+                .iter()
+                .map(|r| Run {
+                    id: r.id.clone(),
+                    data: r.primary.clone(),
+                })
+                .collect();
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("io_read.svg"),
+                    title: format!("{} — I/O Read", crate_name),
+                    y_label: "Bytes Read".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &io_read,
+            )?;
+
+            // I/O Write (secondary column)
+            let io_write: Vec<_> = io_trimmed
+                .iter()
+                .map(|r| Run {
+                    id: r.id.clone(),
+                    data: r.secondary.clone(),
+                })
+                .collect();
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &crate_path.join("io_write.svg"),
+                    title: format!("{} — I/O Write", crate_name),
+                    y_label: "Bytes Written".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                &io_write,
+            )?;
+        }
+
+        Ok(())
+    }
+
+    fn generate_individual_charts(
+        &self,
+        crate_path: &Path,
+        crate_name: &str,
+        disk_runs: &[Run],
+        memory_runs: &[DualRun],
+        progress_runs: &[Run],
+        io_runs: &[DualRun],
+    ) -> Result<()> {
+        for run in disk_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("disk.svg"),
+                    title: format!("{} — Disk Usage", crate_name),
+                    y_label: "Disk Usage".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(run),
+            )?;
+        }
+
+        for run in memory_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate_dual(
+                chart::ChartConfig {
+                    output_path: &run_path.join("memory.svg"),
+                    title: format!("{} — Memory", crate_name),
+                    y_label: "Memory".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(run),
+                "(current)",
+                "(peak)",
+            )?;
+        }
+
+        for run in progress_runs {
+            let run_path = crate_path.join(&run.id);
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("progress.svg"),
+                    title: format!("{} — Progress", crate_name),
+                    y_label: "Progress".to_string(),
+                    y_format: chart::YAxisFormat::Number,
+                },
+                std::slice::from_ref(run),
+            )?;
+        }
+
+        for run in io_runs {
+            let run_path = crate_path.join(&run.id);
+
+            let read_run = Run {
+                id: run.id.clone(),
+                data: run.primary.clone(),
+            };
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("io_read.svg"),
+                    title: format!("{} — I/O Read", crate_name),
+                    y_label: "Bytes Read".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(&read_run),
+            )?;
+
+            let write_run = Run {
+                id: run.id.clone(),
+                data: run.secondary.clone(),
+            };
+            chart::generate(
+                chart::ChartConfig {
+                    output_path: &run_path.join("io_write.svg"),
+                    title: format!("{} — I/O Write", crate_name),
+                    y_label: "Bytes Written".to_string(),
+                    y_format: chart::YAxisFormat::Bytes,
+                },
+                std::slice::from_ref(&write_run),
+            )?;
+        }
+
+        Ok(())
+    }
+}
+

crates/brk_bencher_visualizer/src/main.rs

@@ -0,0 +1,6 @@
+use brk_bencher_visualizer::Visualizer;
+
+fn main() {
+    let v = Visualizer::from_cargo_env().unwrap();
+    v.generate_all_charts().unwrap();
+}

crates/brk_binder/.gitignore

@@ -0,0 +1 @@
+clients/

crates/brk_binder/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "brk_binder"
+description = "A generator of binding files for other languages"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+brk_query = { workspace = true }
+brk_types = { workspace = true }
+oas3 = "0.20"
+schemars = { workspace = true }
+serde_json = { workspace = true }
+vecdb = { workspace = true }

crates/brk_binder/DESIGN.md

@@ -0,0 +1,296 @@
+# brk_binder Design Document
+
+## Goal
+
+Generate typed API clients for **Rust, JavaScript, and Python** with:
+- **Discoverability**: Full IDE autocomplete for 20k+ metrics
+- **Ease of use**: Fluent API with `.fetch()` on each metric node
+
+## Current State
+
+### What's Working ✅
+
+1. **JS + JSDoc generator**: Generates `client.js` with full JSDoc type annotations
+2. **Python generator**: Generates `client.py` with type hints and httpx
+3. **Rust generator**: Generates `client.rs` with strong typing and reqwest
+4. **schemars integration**: JSON schemas embedded in `MetricLeafWithSchema` for type info
+5. **Tree navigation**: `client.tree.blocks.difficulty.fetch()` pattern
+6. **OpenAPI integration**: All GET endpoints generate typed methods
+7. **Server integration**: brk_server calls brk_binder on startup (when clients/ dir exists)
+
+### Generated Output
+
+When `crates/brk_binder/clients/` directory exists, running the server generates:
+
+```
+crates/brk_binder/clients/
+├── javascript/
+│   └── client.js      # JS + JSDoc with tree + API methods
+├── python/
+│   └── client.py      # Python with type hints + httpx
+└── rust/
+    └── client.rs      # Rust with reqwest + strong typing
+```
+
+## Target Architecture
+
+### Input Sources
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                      Input Sources                          │
+├─────────────────────────────────────────────────────────────┤
+│  1. OpenAPI spec (from aide) - endpoint definitions         │
+│  2. brk_query catalog - metric tree structure               │
+│  3. brk_types - Rust types for responses (Rust client only) │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Output: Fluent Client
+
+```javascript
+// JavaScript (with JSDoc for IDE support)
+const client = new BrkClient("http://localhost:3000");
+const data = await client.tree.supply.active.by_date.fetch();
+//                        ^^^^ autocomplete all the way down
+```
+
+```python
+# Python
+client = BrkClient("http://localhost:3000")
+data = client.tree.supply.active.by_date.fetch()
+```
+
+```rust
+// Rust
+let client = BrkClient::new("http://localhost:3000")?;
+let data = client.tree().supply.active.by_date.fetch()?;
+```
+
+## Implementation Details
+
+### Smart Metric Nodes
+
+Each tree leaf becomes a "smart node" holding a client reference:
+
+```javascript
+// JavaScript + JSDoc
+/**
+ * Metric node with fetch capability
+ * @template T
+ */
+class MetricNode {
+  constructor(client, path) {
+    this._client = client;
+    this._path = path;
+  }
+
+  async fetch() {
+    return this._client.get(this._path);
+  }
+}
+```
+
+```python
+# Python
+class MetricNode(Generic[T]):
+    def __init__(self, client: BrkClientBase, path: str):
+        self._client = client
+        self._path = path
+
+    def fetch(self) -> T:
+        return self._client.get(self._path)
+```
+
+```rust
+// Rust
+pub struct MetricNode<'a, T> {
+    client: &'a BrkClientBase,
+    path: &'static str,
+    _marker: PhantomData<T>,
+}
+
+impl<'a, T: DeserializeOwned> MetricNode<'a, T> {
+    pub fn fetch(&self) -> Result<T> {
+        self.client.get(self.path)
+    }
+}
+```
+
+### Pattern Reuse
+
+To avoid 20k+ individual types, reuse structural patterns:
+
+```rust
+// Shared pattern for metrics with same index groupings
+struct ByDateHeightMonth<T> {
+    by_date: MetricNode<T>,
+    by_height: MetricNode<T>,
+    by_month: MetricNode<T>,
+}
+
+// Composed into full tree
+struct Supply {
+    active: ByDateHeightMonth<Vec<f64>>,
+    total: ByDateHeightMonth<Vec<f64>>,
+}
+```
+
+## Type Discovery Solution ✅ IMPLEMENTED
+
+### The Problem
+
+Type information was erased at runtime because metrics are stored as `&dyn AnyExportableVec` trait objects.
+
+### The Solution
+
+Use `std::any::type_name::<T>()` with caching to extract short type names.
+
+#### Implementation (vecdb)
+
+Added `short_type_name<T>()` helper and `value_type_to_string()` to `AnyVec` trait.
+
+### Result
+
+`brk_query` now exposes:
+
+```rust
+for (metric_name, index_to_vec) in &vecs.metric_to_index_to_vec {
+    for (index, vec) in index_to_vec {
+        println!("{} @ {} -> {}",
+            metric_name,                    // "difficulty"
+            vec.index_type_to_string(),     // "Height"
+            vec.value_type_to_string(),     // "StoredF64"
+        );
+    }
+}
+```
+
+## TreeNode Enhancement ✅ IMPLEMENTED
+
+Changed `TreeNode::Leaf(String)` to `TreeNode::Leaf(MetricLeafWithSchema)` where:
+
+```rust
+#[derive(Debug, Clone, Serialize, JsonSchema)]
+pub struct MetricLeafWithSchema {
+    #[serde(flatten)]
+    pub leaf: MetricLeaf,
+    #[serde(skip)]
+    pub schema: serde_json::Value,  // JSON Schema from schemars
+}
+```
+
+## OpenAPI Integration ✅ IMPLEMENTED
+
+### Flow
+
+1. brk_server creates OpenAPI spec via aide
+2. On startup, serializes spec to JSON string
+3. Passes JSON to `brk_binder::generate_clients()`
+4. brk_binder parses with `oas3` crate (supports OpenAPI 3.1)
+5. Generates typed methods for all GET endpoints
+
+### Why oas3?
+
+aide generates OpenAPI 3.1 specs. The `openapiv3` crate only supports 3.0.x.
+The `oas3` crate supports OpenAPI 3.1.x parsing.
+
+## Tasks
+
+### Phase 0: Type Infrastructure ✅ COMPLETE
+
+- [x] vecdb: Add `short_type_name<T>()` and `value_type_to_string()`
+- [x] vecdb: Add optional `schemars` feature with `AnySchemaVec` trait
+- [x] brk_types: Enhance `TreeNode::Leaf` to include `MetricLeafWithSchema`
+- [x] brk_traversable: Update all `to_tree_node()` with schemars integration
+- [x] brk_binder: Set up generator module structure
+
+### Phase 1: JavaScript Client ✅ COMPLETE
+
+- [x] Define `MetricNode` class with JSDoc generics
+- [x] Define `BrkClient` with base HTTP functionality
+- [x] Generate `client.js` with full JSDoc type annotations
+- [x] Tree navigation: `client.tree.category.metric.fetch()`
+- [x] API methods from OpenAPI endpoints
+
+### Phase 2: OpenAPI Integration ✅ COMPLETE
+
+- [x] Add `oas3` crate dependency (OpenAPI 3.1 support)
+- [x] brk_server passes OpenAPI JSON to brk_binder on startup
+- [x] Parse OpenAPI spec and extract endpoint definitions
+- [x] Generate typed methods for each GET endpoint
+
+### Phase 3: Python Client ✅ COMPLETE
+
+- [x] Define `MetricNode` class with type hints
+- [x] Define `BrkClient` with httpx
+- [x] Generate typed methods from OpenAPI
+- [x] Generate tree navigation
+
+### Phase 4: Rust Client ✅ COMPLETE
+
+- [x] Define `MetricNode<T>` struct with lifetimes
+- [x] Define `BrkClient` with reqwest (blocking)
+- [x] Generate tree navigation with proper lifetimes
+- [x] Generate typed methods from OpenAPI
+
+### Phase 5: Polish
+
+- [x] Switch from `openapiv3` to `oas3` crate
+- [ ] Error types per language
+- [ ] Documentation generation
+- [ ] Tests
+- [ ] Example usage in each language
+- [ ] Async Rust client variant
+
+## File Structure
+
+```
+crates/brk_binder/
+├── src/
+│   ├── lib.rs
+│   ├── js.rs              # JS constants generation (existing)
+│   └── generator/
+│       ├── mod.rs         # generate_clients() entry point
+│       ├── types.rs       # ClientMetadata, MetricInfo, IndexPattern
+│       ├── openapi.rs     # OpenAPI 3.1 spec parsing (oas3)
+│       ├── javascript.rs  # JavaScript + JSDoc client ✅
+│       ├── python.rs      # Python client ✅
+│       └── rust.rs        # Rust client ✅
+├── clients/               # Generated output (gitignored)
+│   ├── javascript/
+│   ├── python/
+│   └── rust/
+├── Cargo.toml
+├── README.md
+└── DESIGN.md
+
+crates/brk_server/
+└── src/
+    ├── lib.rs             # Calls brk_binder::generate_clients() on startup
+    └── api/
+        └── openapi.rs     # create_openapi() for aide
+```
+
+## Dependencies
+
+```toml
+[dependencies]
+brk_query = { workspace = true }
+brk_types = { workspace = true }
+oas3 = "0.20"                # OpenAPI 3.1 spec parsing
+schemars = { workspace = true }
+serde_json = { workspace = true }
+```
+
+## Usage
+
+To generate clients:
+
+```bash
+# Create the output directory
+mkdir -p crates/brk_binder/clients
+
+# Run the server (generates clients on startup)
+cargo run -p brk_server
+```

crates/brk_binder/README.md

@@ -0,0 +1,42 @@
+# brk_binder
+
+Code generation for BRK client libraries.
+
+## What It Enables
+
+Generate typed metric catalogs and constants for JavaScript/TypeScript clients. Keeps frontend code in sync with available metrics without manual maintenance.
+
+## Key Features
+
+- **Metric catalog**: Generates `metrics.js` with all metric IDs and their supported indexes
+- **Compression**: Metric names compressed via word-to-base62 mapping for smaller bundles
+- **Mining pools**: Generates `pools.js` with pool ID to name mapping
+- **Version sync**: Generates `version.js` matching server version
+
+## Core API
+
+```rust,ignore
+generate_js_files(&query, &modules_path)?;
+```
+
+## Generated Files
+
+```
+modules/brk-client/generated/
+├── version.js    # export const VERSION = "vX.Y.Z"
+├── metrics.js    # INDEXES, COMPRESSED_METRIC_TO_INDEXES
+└── pools.js      # POOL_ID_TO_POOL_NAME
+```
+
+## Metric Compression
+
+To minimize bundle size, metric names are compressed:
+1. Extract all words from metric names
+2. Sort by frequency
+3. Map to base52 codes (A-Z, a-z)
+4. Store compressed metric → index group mapping
+
+## Built On
+
+- `brk_query` for metric enumeration
+- `brk_types` for mining pool data

crates/brk_binder/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_binder/src/javascript.rs

@@ -0,0 +1,988 @@
+use std::collections::HashSet;
+use std::fmt::Write as FmtWrite;
+use std::fs;
+use std::io;
+use std::path::Path;
+
+use brk_types::{Index, TreeNode};
+use serde_json::Value;
+
+use crate::{
+    ClientMetadata, Endpoint, FieldNamePosition, IndexSetPattern, PatternField, StructuralPattern,
+    TypeSchemas, extract_inner_type, get_fields_with_child_info, get_first_leaf_name,
+    get_node_fields, get_pattern_instance_base, to_camel_case, to_pascal_case,
+};
+
+/// Generate JavaScript + JSDoc client from metadata and OpenAPI endpoints
+pub fn generate_javascript_client(
+    metadata: &ClientMetadata,
+    endpoints: &[Endpoint],
+    schemas: &TypeSchemas,
+    output_dir: &Path,
+) -> io::Result<()> {
+    let mut output = String::new();
+
+    // Header
+    writeln!(output, "// Auto-generated BRK JavaScript client").unwrap();
+    writeln!(output, "// Do not edit manually\n").unwrap();
+
+    // Generate type definitions from OpenAPI schemas
+    generate_type_definitions(&mut output, schemas);
+
+    // Generate the base client class
+    generate_base_client(&mut output);
+
+    // Generate index accessor factory functions
+    generate_index_accessors(&mut output, &metadata.index_set_patterns);
+
+    // Generate structural pattern factory functions
+    generate_structural_patterns(&mut output, &metadata.structural_patterns, metadata);
+
+    // Generate tree JSDoc typedefs
+    generate_tree_typedefs(&mut output, &metadata.catalog, metadata);
+
+    // Generate the main client class with tree and API methods
+    generate_main_client(&mut output, &metadata.catalog, metadata, endpoints);
+
+    fs::write(output_dir.join("client.js"), output)?;
+
+    Ok(())
+}
+
+/// Generate JSDoc type definitions from OpenAPI schemas
+fn generate_type_definitions(output: &mut String, schemas: &TypeSchemas) {
+    if schemas.is_empty() {
+        return;
+    }
+
+    writeln!(output, "// Type definitions\n").unwrap();
+
+    for (name, schema) in schemas {
+        let js_type = schema_to_js_type_ctx(schema, Some(name));
+
+        if is_primitive_alias(schema) {
+            // Simple type alias: @typedef {number} Height
+            writeln!(output, "/** @typedef {{{}}} {} */", js_type, name).unwrap();
+        } else if let Some(props) = schema.get("properties").and_then(|p| p.as_object()) {
+            // Object type with properties
+            writeln!(output, "/**").unwrap();
+            writeln!(output, " * @typedef {{Object}} {}", name).unwrap();
+            for (prop_name, prop_schema) in props {
+                let prop_type = schema_to_js_type_ctx(prop_schema, Some(name));
+                let required = schema
+                    .get("required")
+                    .and_then(|r| r.as_array())
+                    .map(|arr| arr.iter().any(|v| v.as_str() == Some(prop_name)))
+                    .unwrap_or(false);
+                let optional = if required { "" } else { "=" };
+                writeln!(
+                    output,
+                    " * @property {{{}{}}} {}",
+                    prop_type, optional, prop_name
+                )
+                .unwrap();
+            }
+            writeln!(output, " */").unwrap();
+        } else {
+            // Other schemas - just typedef
+            writeln!(output, "/** @typedef {{{}}} {} */", js_type, name).unwrap();
+        }
+    }
+    writeln!(output).unwrap();
+}
+
+/// Check if schema represents a primitive type alias (like Height = number)
+fn is_primitive_alias(schema: &Value) -> bool {
+    schema.get("properties").is_none()
+        && schema.get("items").is_none()
+        && schema.get("anyOf").is_none()
+        && schema.get("oneOf").is_none()
+        && schema.get("enum").is_none()
+}
+
+/// Convert a single JSON type string to JavaScript type
+fn json_type_to_js(ty: &str, schema: &Value, current_type: Option<&str>) -> String {
+    match ty {
+        "integer" | "number" => "number".to_string(),
+        "boolean" => "boolean".to_string(),
+        "string" => "string".to_string(),
+        "null" => "null".to_string(),
+        "array" => {
+            let item_type = schema
+                .get("items")
+                .map(|s| schema_to_js_type_ctx(s, current_type))
+                .unwrap_or_else(|| "*".to_string());
+            format!("{}[]", item_type)
+        }
+        "object" => {
+            // Check if it has additionalProperties (dict-like)
+            if let Some(add_props) = schema.get("additionalProperties") {
+                let value_type = schema_to_js_type_ctx(add_props, current_type);
+                // Use TypeScript index signature syntax for recursive types
+                return format!("{{ [key: string]: {} }}", value_type);
+            }
+            "Object".to_string()
+        }
+        _ => "*".to_string(),
+    }
+}
+
+/// Convert JSON Schema to JavaScript/JSDoc type with context for recursive types
+fn schema_to_js_type_ctx(schema: &Value, current_type: Option<&str>) -> String {
+    // Handle allOf (try each element until we find a resolvable type)
+    if let Some(all_of) = schema.get("allOf").and_then(|v| v.as_array()) {
+        for item in all_of {
+            let resolved = schema_to_js_type_ctx(item, current_type);
+            if resolved != "*" {
+                return resolved;
+            }
+        }
+    }
+
+    // Handle $ref
+    if let Some(ref_path) = schema.get("$ref").and_then(|r| r.as_str()) {
+        return ref_path.rsplit('/').next().unwrap_or("*").to_string();
+    }
+
+    // Handle enum (array of string values)
+    if let Some(enum_values) = schema.get("enum").and_then(|e| e.as_array()) {
+        let literals: Vec<String> = enum_values
+            .iter()
+            .filter_map(|v| v.as_str())
+            .map(|s| format!("\"{}\"", s))
+            .collect();
+        if !literals.is_empty() {
+            return format!("({})", literals.join("|"));
+        }
+    }
+
+    // Handle type field (can be string or array of strings)
+    if let Some(ty) = schema.get("type") {
+        // Handle array of types like ["string", "null"] for Optional
+        if let Some(type_array) = ty.as_array() {
+            let types: Vec<String> = type_array
+                .iter()
+                .filter_map(|t| t.as_str())
+                .filter(|t| *t != "null")
+                .map(|t| json_type_to_js(t, schema, current_type))
+                .collect();
+            let has_null = type_array.iter().any(|t| t.as_str() == Some("null"));
+
+            if types.len() == 1 {
+                let base_type = &types[0];
+                return if has_null {
+                    format!("?{}", base_type)
+                } else {
+                    base_type.clone()
+                };
+            } else if !types.is_empty() {
+                let union = format!("({})", types.join("|"));
+                return if has_null {
+                    format!("?{}", union)
+                } else {
+                    union
+                };
+            }
+        }
+
+        // Handle single type string
+        if let Some(ty_str) = ty.as_str() {
+            return json_type_to_js(ty_str, schema, current_type);
+        }
+    }
+
+    // Handle anyOf/oneOf
+    if let Some(variants) = schema
+        .get("anyOf")
+        .or_else(|| schema.get("oneOf"))
+        .and_then(|v| v.as_array())
+    {
+        let types: Vec<String> = variants
+            .iter()
+            .map(|v| schema_to_js_type_ctx(v, current_type))
+            .collect();
+        // Filter out * and null for cleaner unions
+        let filtered: Vec<_> = types.iter().filter(|t| *t != "*").collect();
+        if !filtered.is_empty() {
+            return format!(
+                "({})",
+                filtered
+                    .iter()
+                    .map(|s| s.as_str())
+                    .collect::<Vec<_>>()
+                    .join("|")
+            );
+        }
+        return format!("({})", types.join("|"));
+    }
+
+    // Check for format hint without type (common in OpenAPI)
+    if let Some(format) = schema.get("format").and_then(|f| f.as_str()) {
+        return match format {
+            "int32" | "int64" => "number".to_string(),
+            "float" | "double" => "number".to_string(),
+            "date" | "date-time" => "string".to_string(),
+            _ => "*".to_string(),
+        };
+    }
+
+    "*".to_string()
+}
+
+/// Generate the base BrkClient class with HTTP functionality
+fn generate_base_client(output: &mut String) {
+    writeln!(
+        output,
+        r#"/**
+ * @typedef {{Object}} BrkClientOptions
+ * @property {{string}} baseUrl - Base URL for the API
+ * @property {{number}} [timeout] - Request timeout in milliseconds
+ */
+
+const _isBrowser = typeof window !== 'undefined' && 'caches' in window;
+const _runIdle = (fn) => (globalThis.requestIdleCallback ?? setTimeout)(fn);
+
+/** @type {{Promise<Cache | null>}} */
+const _cachePromise = _isBrowser
+  ? caches.open('__BRK_CLIENT__').catch(() => null)
+  : Promise.resolve(null);
+
+/**
+ * Custom error class for BRK client errors
+ */
+class BrkError extends Error {{
+  /**
+   * @param {{string}} message
+   * @param {{number}} [status]
+   */
+  constructor(message, status) {{
+    super(message);
+    this.name = 'BrkError';
+    this.status = status;
+  }}
+}}
+
+/**
+ * A metric node that can fetch data for different indexes.
+ * @template T
+ */
+class MetricNode {{
+  /**
+   * @param {{BrkClientBase}} client
+   * @param {{string}} path
+   */
+  constructor(client, path) {{
+    this._client = client;
+    this._path = path;
+  }}
+
+  /**
+   * Fetch all data points for this metric.
+   * @param {{(value: T[]) => void}} [onUpdate] - Called when data is available (may be called twice: cache then fresh)
+   * @returns {{Promise<T[] | null>}}
+   */
+  get(onUpdate) {{
+    return this._client.get(this._path, onUpdate);
+  }}
+
+  /**
+   * Fetch data points within a range.
+   * @param {{string | number}} from
+   * @param {{string | number}} to
+   * @param {{(value: T[]) => void}} [onUpdate] - Called when data is available (may be called twice: cache then fresh)
+   * @returns {{Promise<T[] | null>}}
+   */
+  getRange(from, to, onUpdate) {{
+    return this._client.get(`${{this._path}}?from=${{from}}&to=${{to}}`, onUpdate);
+  }}
+}}
+
+/**
+ * Base HTTP client for making requests with caching support
+ */
+class BrkClientBase {{
+  /**
+   * @param {{BrkClientOptions|string}} options
+   */
+  constructor(options) {{
+    const isString = typeof options === 'string';
+    this.baseUrl = isString ? options : options.baseUrl;
+    this.timeout = isString ? 5000 : (options.timeout ?? 5000);
+  }}
+
+  /**
+   * Make a GET request with stale-while-revalidate caching
+   * @template T
+   * @param {{string}} path
+   * @param {{(value: T) => void}} [onUpdate] - Called when data is available
+   * @returns {{Promise<T | null>}}
+   */
+  async get(path, onUpdate) {{
+    const url = `${{this.baseUrl}}${{path}}`;
+    const cache = await _cachePromise;
+    const cachedRes = await cache?.match(url);
+    const cachedJson = cachedRes ? await cachedRes.json() : null;
+
+    if (cachedJson) onUpdate?.(cachedJson);
+    if (!globalThis.navigator?.onLine) return cachedJson;
+
+    try {{
+      const res = await fetch(url, {{ signal: AbortSignal.timeout(this.timeout) }});
+      if (!res.ok) throw new BrkError(`HTTP ${{res.status}}`, res.status);
+      if (cachedRes?.headers.get('ETag') === res.headers.get('ETag')) return cachedJson;
+
+      const cloned = res.clone();
+      const json = await res.json();
+      onUpdate?.(json);
+      if (cache) _runIdle(() => cache.put(url, cloned));
+      return json;
+    }} catch (e) {{
+      if (cachedJson) return cachedJson;
+      throw e;
+    }}
+  }}
+}}
+
+"#
+    )
+    .unwrap();
+}
+
+/// Generate index accessor factory functions
+fn generate_index_accessors(output: &mut String, patterns: &[IndexSetPattern]) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "// Index accessor factory functions\n").unwrap();
+
+    for pattern in patterns {
+        // Generate JSDoc typedef for the accessor
+        writeln!(output, "/**").unwrap();
+        writeln!(output, " * @template T").unwrap();
+        writeln!(output, " * @typedef {{Object}} {}", pattern.name).unwrap();
+        for index in &pattern.indexes {
+            let field_name = index_to_camel_case(index);
+            writeln!(output, " * @property {{MetricNode<T>}} {}", field_name).unwrap();
+        }
+        writeln!(output, " */\n").unwrap();
+
+        // Generate factory function
+        writeln!(output, "/**").unwrap();
+        writeln!(output, " * Create a {} accessor", pattern.name).unwrap();
+        writeln!(output, " * @template T").unwrap();
+        writeln!(output, " * @param {{BrkClientBase}} client").unwrap();
+        writeln!(output, " * @param {{string}} basePath").unwrap();
+        writeln!(output, " * @returns {{{}<T>}}", pattern.name).unwrap();
+        writeln!(output, " */").unwrap();
+        writeln!(
+            output,
+            "function create{}(client, basePath) {{",
+            pattern.name
+        )
+        .unwrap();
+        writeln!(output, "  return {{").unwrap();
+
+        for (i, index) in pattern.indexes.iter().enumerate() {
+            let field_name = index_to_camel_case(index);
+            let path_segment = index.serialize_long();
+            let comma = if i < pattern.indexes.len() - 1 {
+                ","
+            } else {
+                ""
+            };
+            writeln!(
+                output,
+                "    {}: new MetricNode(client, `${{basePath}}/{}`){}",
+                field_name, path_segment, comma
+            )
+            .unwrap();
+        }
+
+        writeln!(output, "  }};").unwrap();
+        writeln!(output, "}}\n").unwrap();
+    }
+}
+
+/// Convert an Index to a camelCase field name (e.g., DateIndex -> byDateIndex)
+fn index_to_camel_case(index: &Index) -> String {
+    format!("by{}", to_pascal_case(index.serialize_long()))
+}
+
+/// Generate structural pattern factory functions
+fn generate_structural_patterns(
+    output: &mut String,
+    patterns: &[StructuralPattern],
+    metadata: &ClientMetadata,
+) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "// Reusable structural pattern factories\n").unwrap();
+
+    for pattern in patterns {
+        // Check if this pattern is parameterizable (has field positions detected)
+        let is_parameterizable = pattern.is_parameterizable();
+
+        // Generate JSDoc typedef
+        writeln!(output, "/**").unwrap();
+        if pattern.is_generic {
+            writeln!(output, " * @template T").unwrap();
+        }
+        writeln!(output, " * @typedef {{Object}} {}", pattern.name).unwrap();
+        for field in &pattern.fields {
+            let js_type = field_to_js_type_generic(field, metadata, pattern.is_generic);
+            writeln!(
+                output,
+                " * @property {{{}}} {}",
+                js_type,
+                to_camel_case(&field.name)
+            )
+            .unwrap();
+        }
+        writeln!(output, " */\n").unwrap();
+
+        // Generate factory function
+        writeln!(output, "/**").unwrap();
+        writeln!(output, " * Create a {} pattern node", pattern.name).unwrap();
+        writeln!(output, " * @param {{BrkClientBase}} client").unwrap();
+        if is_parameterizable {
+            writeln!(output, " * @param {{string}} acc - Accumulated metric name").unwrap();
+        } else {
+            writeln!(output, " * @param {{string}} basePath").unwrap();
+        }
+        writeln!(output, " * @returns {{{}}}", pattern.name).unwrap();
+        writeln!(output, " */").unwrap();
+
+        let param_name = if is_parameterizable {
+            "acc"
+        } else {
+            "basePath"
+        };
+        writeln!(
+            output,
+            "function create{}(client, {}) {{",
+            pattern.name, param_name
+        )
+        .unwrap();
+        writeln!(output, "  return {{").unwrap();
+
+        for (i, field) in pattern.fields.iter().enumerate() {
+            let comma = if i < pattern.fields.len() - 1 {
+                ","
+            } else {
+                ""
+            };
+
+            if is_parameterizable {
+                generate_parameterized_field(output, field, pattern, metadata, comma);
+            } else {
+                generate_tree_path_field(output, field, metadata, comma);
+            }
+        }
+
+        writeln!(output, "  }};").unwrap();
+        writeln!(output, "}}\n").unwrap();
+    }
+}
+
+/// Generate a field using parameterized (prepend/append) metric name construction
+fn generate_parameterized_field(
+    output: &mut String,
+    field: &PatternField,
+    pattern: &StructuralPattern,
+    metadata: &ClientMetadata,
+    comma: &str,
+) {
+    let field_name_js = to_camel_case(&field.name);
+
+    // For branch fields, pass the accumulated name to nested pattern
+    if metadata.is_pattern_type(&field.rust_type) {
+        // Get the field position to determine how to transform the accumulated name
+        let child_acc = if let Some(pos) = pattern.get_field_position(&field.name) {
+            match pos {
+                FieldNamePosition::Append(suffix) => format!("`${{acc}}{}`", suffix),
+                FieldNamePosition::Prepend(prefix) => format!("`{}{}`", prefix, "${acc}"),
+                FieldNamePosition::Identity => "acc".to_string(),
+                FieldNamePosition::SetBase(base) => format!("'{}'", base),
+            }
+        } else {
+            // Fallback: append field name
+            format!("`${{acc}}_{}`", field.name)
+        };
+
+        writeln!(
+            output,
+            "    {}: create{}(client, {}){}",
+            field_name_js, field.rust_type, child_acc, comma
+        )
+        .unwrap();
+        return;
+    }
+
+    // For leaf fields, construct the metric path based on position
+    let metric_expr = if let Some(pos) = pattern.get_field_position(&field.name) {
+        match pos {
+            FieldNamePosition::Append(suffix) => format!("`/${{acc}}{suffix}`"),
+            FieldNamePosition::Prepend(prefix) => format!("`/{prefix}${{acc}}`"),
+            FieldNamePosition::Identity => "`/${acc}`".to_string(),
+            FieldNamePosition::SetBase(base) => format!("'/{base}'"),
+        }
+    } else {
+        // Fallback: use field name appended
+        format!("`/${{acc}}_{}`", field.name)
+    };
+
+    if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "    {}: create{}(client, {}){}",
+            field_name_js, accessor.name, metric_expr, comma
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "    {}: new MetricNode(client, {}){}",
+            field_name_js, metric_expr, comma
+        )
+        .unwrap();
+    }
+}
+
+/// Generate a field using tree path construction (fallback for non-parameterizable patterns)
+fn generate_tree_path_field(
+    output: &mut String,
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    comma: &str,
+) {
+    let field_name_js = to_camel_case(&field.name);
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        writeln!(
+            output,
+            "    {}: create{}(client, `${{basePath}}/{}`){}",
+            field_name_js, field.rust_type, field.name, comma
+        )
+        .unwrap();
+    } else if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "    {}: create{}(client, `${{basePath}}/{}`){}",
+            field_name_js, accessor.name, field.name, comma
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "    {}: new MetricNode(client, `${{basePath}}/{}`){}",
+            field_name_js, field.name, comma
+        )
+        .unwrap();
+    }
+}
+
+/// Convert pattern field to JavaScript/JSDoc type, with optional generic support
+fn field_to_js_type_generic(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+) -> String {
+    field_to_js_type_with_generic_value(field, metadata, is_generic, None)
+}
+
+/// Convert pattern field to JavaScript/JSDoc type.
+/// - `is_generic`: If true and field.rust_type is "T", use T in the output
+/// - `generic_value_type`: For branch fields that reference a generic pattern, this is the concrete type to substitute
+fn field_to_js_type_with_generic_value(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+    generic_value_type: Option<&str>,
+) -> String {
+    // For generic patterns, use T instead of concrete value type
+    // Also extract inner type from wrappers like Close<Dollars> -> Dollars
+    let value_type = if is_generic && field.rust_type == "T" {
+        "T".to_string()
+    } else {
+        extract_inner_type(&field.rust_type)
+    };
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        // Check if this pattern is generic and we have a value type
+        if metadata.is_pattern_generic(&field.rust_type)
+            && let Some(vt) = generic_value_type
+        {
+            return format!("{}<{}>", field.rust_type, vt);
+        }
+        field.rust_type.clone()
+    } else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
+        // Leaf with accessor - use value_type as the generic
+        format!("{}<{}>", accessor.name, value_type)
+    } else {
+        // Leaf - use value_type as the generic
+        format!("MetricNode<{}>", value_type)
+    }
+}
+
+
+/// Generate tree typedefs
+fn generate_tree_typedefs(output: &mut String, catalog: &TreeNode, metadata: &ClientMetadata) {
+    writeln!(output, "// Catalog tree typedefs\n").unwrap();
+
+    let pattern_lookup = metadata.pattern_lookup();
+    let mut generated = HashSet::new();
+    generate_tree_typedef(
+        output,
+        "CatalogTree",
+        catalog,
+        &pattern_lookup,
+        metadata,
+        &mut generated,
+    );
+}
+
+/// Recursively generate tree typedefs
+fn generate_tree_typedef(
+    output: &mut String,
+    name: &str,
+    node: &TreeNode,
+    pattern_lookup: &std::collections::HashMap<Vec<PatternField>, String>,
+    metadata: &ClientMetadata,
+    generated: &mut HashSet<String>,
+) {
+    let TreeNode::Branch(children) = node else {
+        return;
+    };
+
+    let fields_with_child_info = get_fields_with_child_info(children, name, pattern_lookup);
+    let fields: Vec<PatternField> = fields_with_child_info
+        .iter()
+        .map(|(f, _)| f.clone())
+        .collect();
+
+    // Skip if this matches a pattern (already generated)
+    if pattern_lookup.contains_key(&fields) && pattern_lookup.get(&fields) != Some(&name.to_string())
+    {
+        return;
+    }
+
+    if generated.contains(name) {
+        return;
+    }
+    generated.insert(name.to_string());
+
+    writeln!(output, "/**").unwrap();
+    writeln!(output, " * @typedef {{Object}} {}", name).unwrap();
+
+    for (field, child_fields) in &fields_with_child_info {
+        let generic_value_type = child_fields
+            .as_ref()
+            .and_then(|cf| metadata.get_generic_value_type(&field.rust_type, cf));
+        let js_type = field_to_js_type_with_generic_value(
+            field,
+            metadata,
+            false,
+            generic_value_type.as_deref(),
+        );
+        writeln!(
+            output,
+            " * @property {{{}}} {}",
+            js_type,
+            to_camel_case(&field.name)
+        )
+        .unwrap();
+    }
+
+    writeln!(output, " */\n").unwrap();
+
+    // Generate child typedefs
+    for (child_name, child_node) in children {
+        if let TreeNode::Branch(grandchildren) = child_node {
+            let child_fields = get_node_fields(grandchildren, pattern_lookup);
+            if !pattern_lookup.contains_key(&child_fields) {
+                let child_type_name = format!("{}_{}", name, to_pascal_case(child_name));
+                generate_tree_typedef(
+                    output,
+                    &child_type_name,
+                    child_node,
+                    pattern_lookup,
+                    metadata,
+                    generated,
+                );
+            }
+        }
+    }
+}
+
+/// Generate main client
+fn generate_main_client(
+    output: &mut String,
+    catalog: &TreeNode,
+    metadata: &ClientMetadata,
+    endpoints: &[Endpoint],
+) {
+    let pattern_lookup = metadata.pattern_lookup();
+
+    writeln!(output, "/**").unwrap();
+    writeln!(
+        output,
+        " * Main BRK client with catalog tree and API methods"
+    )
+    .unwrap();
+    writeln!(output, " * @extends BrkClientBase").unwrap();
+    writeln!(output, " */").unwrap();
+    writeln!(output, "class BrkClient extends BrkClientBase {{").unwrap();
+    writeln!(output, "  /**").unwrap();
+    writeln!(output, "   * @param {{BrkClientOptions|string}} options").unwrap();
+    writeln!(output, "   */").unwrap();
+    writeln!(output, "  constructor(options) {{").unwrap();
+    writeln!(output, "    super(options);").unwrap();
+    writeln!(output, "    /** @type {{CatalogTree}} */").unwrap();
+    writeln!(output, "    this.tree = this._buildTree('');").unwrap();
+    writeln!(output, "  }}\n").unwrap();
+
+    // Generate _buildTree method
+    writeln!(output, "  /**").unwrap();
+    writeln!(output, "   * @private").unwrap();
+    writeln!(output, "   * @param {{string}} basePath").unwrap();
+    writeln!(output, "   * @returns {{CatalogTree}}").unwrap();
+    writeln!(output, "   */").unwrap();
+    writeln!(output, "  _buildTree(basePath) {{").unwrap();
+    writeln!(output, "    return {{").unwrap();
+    generate_tree_initializer(output, catalog, "", 3, &pattern_lookup, metadata);
+    writeln!(output, "    }};").unwrap();
+    writeln!(output, "  }}\n").unwrap();
+
+    // Generate API methods
+    generate_api_methods(output, endpoints);
+
+    writeln!(output, "}}\n").unwrap();
+
+    // Export
+    writeln!(
+        output,
+        "export {{ BrkClient, BrkClientBase, BrkError, MetricNode }};"
+    )
+    .unwrap();
+}
+
+/// Generate tree initializer
+fn generate_tree_initializer(
+    output: &mut String,
+    node: &TreeNode,
+    accumulated_name: &str,
+    indent: usize,
+    pattern_lookup: &std::collections::HashMap<Vec<PatternField>, String>,
+    metadata: &ClientMetadata,
+) {
+    let indent_str = "  ".repeat(indent);
+
+    if let TreeNode::Branch(children) = node {
+        for (i, (child_name, child_node)) in children.iter().enumerate() {
+            let field_name = to_camel_case(child_name);
+            let comma = if i < children.len() - 1 { "," } else { "" };
+
+            match child_node {
+                TreeNode::Leaf(leaf) => {
+                    // Use leaf.name() (vec.name()) for API path, not tree path
+                    let metric_path = format!("/{}", leaf.name());
+                    if let Some(accessor) = metadata.find_index_set_pattern(leaf.indexes()) {
+                        writeln!(
+                            output,
+                            "{}{}: create{}(this, '{}'){}",
+                            indent_str, field_name, accessor.name, metric_path, comma
+                        )
+                        .unwrap();
+                    } else {
+                        writeln!(
+                            output,
+                            "{}{}: new MetricNode(this, '{}'){}",
+                            indent_str, field_name, metric_path, comma
+                        )
+                        .unwrap();
+                    }
+                }
+                TreeNode::Branch(grandchildren) => {
+                    let child_fields = get_node_fields(grandchildren, pattern_lookup);
+                    if let Some(pattern_name) = pattern_lookup.get(&child_fields) {
+                        // For parameterized patterns, derive accumulated metric name from first leaf
+                        let pattern = metadata
+                            .structural_patterns
+                            .iter()
+                            .find(|p| &p.name == pattern_name);
+                        let is_parameterizable =
+                            pattern.map(|p| p.is_parameterizable()).unwrap_or(false);
+
+                        let arg = if is_parameterizable {
+                            // Get the metric base from the first leaf descendant
+                            get_pattern_instance_base(child_node, child_name)
+                        } else {
+                            // Fallback to tree path for non-parameterizable patterns
+                            if accumulated_name.is_empty() {
+                                format!("/{}", child_name)
+                            } else {
+                                format!("{}/{}", accumulated_name, child_name)
+                            }
+                        };
+
+                        writeln!(
+                            output,
+                            "{}{}: create{}(this, '{}'){}",
+                            indent_str, field_name, pattern_name, arg, comma
+                        )
+                        .unwrap();
+                    } else {
+                        // Not a pattern - recurse with accumulated name
+                        let child_acc =
+                            infer_child_accumulated_name(child_node, accumulated_name, child_name);
+                        writeln!(output, "{}{}: {{", indent_str, field_name).unwrap();
+                        generate_tree_initializer(
+                            output,
+                            child_node,
+                            &child_acc,
+                            indent + 1,
+                            pattern_lookup,
+                            metadata,
+                        );
+                        writeln!(output, "{}}}{}", indent_str, comma).unwrap();
+                    }
+                }
+            }
+        }
+    }
+}
+
+/// Infer the accumulated metric name for a child node
+fn infer_child_accumulated_name(node: &TreeNode, parent_acc: &str, field_name: &str) -> String {
+    // Try to infer from first leaf descendant
+    if let Some(leaf_name) = get_first_leaf_name(node) {
+        // Look for field_name in the leaf metric name
+        if let Some(pos) = leaf_name.find(field_name) {
+            // The field_name appears in the metric - use it as base
+            if pos == 0 {
+                // At start - this is the base
+                return field_name.to_string();
+            } else if leaf_name.chars().nth(pos - 1) == Some('_') {
+                // After underscore - likely an append pattern
+                if parent_acc.is_empty() {
+                    return field_name.to_string();
+                }
+                return format!("{}_{}", parent_acc, field_name);
+            }
+        }
+    }
+
+    // Fallback: append field name
+    if parent_acc.is_empty() {
+        field_name.to_string()
+    } else {
+        format!("{}_{}", parent_acc, field_name)
+    }
+}
+
+/// Generate API methods
+fn generate_api_methods(output: &mut String, endpoints: &[Endpoint]) {
+    for endpoint in endpoints {
+        if !endpoint.should_generate() {
+            continue;
+        }
+
+        let method_name = endpoint_to_method_name(endpoint);
+        let return_type = endpoint.response_type.as_deref().unwrap_or("*");
+
+        writeln!(output, "  /**").unwrap();
+        if let Some(summary) = &endpoint.summary {
+            writeln!(output, "   * {}", summary).unwrap();
+        }
+
+        for param in &endpoint.path_params {
+            let desc = param.description.as_deref().unwrap_or("");
+            writeln!(
+                output,
+                "   * @param {{{}}} {} {}",
+                param.param_type, param.name, desc
+            )
+            .unwrap();
+        }
+        for param in &endpoint.query_params {
+            let optional = if param.required { "" } else { "=" };
+            let desc = param.description.as_deref().unwrap_or("");
+            writeln!(
+                output,
+                "   * @param {{{}{}}} [{}] {}",
+                param.param_type, optional, param.name, desc
+            )
+            .unwrap();
+        }
+
+        writeln!(output, "   * @returns {{Promise<{}>}}", return_type).unwrap();
+        writeln!(output, "   */").unwrap();
+
+        let params = build_method_params(endpoint);
+        writeln!(output, "  async {}({}) {{", method_name, params).unwrap();
+
+        let path = build_path_template(&endpoint.path, &endpoint.path_params);
+
+        if endpoint.query_params.is_empty() {
+            writeln!(output, "    return this.get(`{}`);", path).unwrap();
+        } else {
+            writeln!(output, "    const params = new URLSearchParams();").unwrap();
+            for param in &endpoint.query_params {
+                if param.required {
+                    writeln!(
+                        output,
+                        "    params.set('{}', String({}));",
+                        param.name, param.name
+                    )
+                    .unwrap();
+                } else {
+                    writeln!(
+                        output,
+                        "    if ({} !== undefined) params.set('{}', String({}));",
+                        param.name, param.name, param.name
+                    )
+                    .unwrap();
+                }
+            }
+            writeln!(output, "    const query = params.toString();").unwrap();
+            writeln!(
+                output,
+                "    return this.get(`{}${{query ? '?' + query : ''}}`);",
+                path
+            )
+            .unwrap();
+        }
+
+        writeln!(output, "  }}\n").unwrap();
+    }
+}
+
+fn endpoint_to_method_name(endpoint: &Endpoint) -> String {
+    to_camel_case(&endpoint.operation_name())
+}
+
+fn build_method_params(endpoint: &Endpoint) -> String {
+    let mut params = Vec::new();
+    for param in &endpoint.path_params {
+        params.push(param.name.clone());
+    }
+    for param in &endpoint.query_params {
+        params.push(param.name.clone());
+    }
+    params.join(", ")
+}
+
+fn build_path_template(path: &str, path_params: &[super::Parameter]) -> String {
+    let mut result = path.to_string();
+    for param in path_params {
+        let placeholder = format!("{{{}}}", param.name);
+        let interpolation = format!("${{{}}}", param.name);
+        result = result.replace(&placeholder, &interpolation);
+    }
+    result
+}

crates/brk_binder/src/js.rs

@@ -0,0 +1,240 @@
+use std::{
+    collections::{BTreeMap, HashMap},
+    fs, io,
+    path::Path,
+};
+
+use brk_query::Query;
+use brk_types::{Index, pools};
+
+use crate::VERSION;
+
+const AUTO_GENERATED_DISCLAIMER: &str = "//
+// File auto-generated, any modifications will be overwritten
+//";
+
+pub fn generate_js_files(query: &Query, modules_path: &Path) -> io::Result<()> {
+    let path = modules_path.join("brk-client");
+
+    if !fs::exists(&path)? {
+        return Ok(());
+    }
+
+    let path = path.join("generated");
+    fs::create_dir_all(&path)?;
+
+    generate_version_file(&path)?;
+    generate_metrics_file(query, &path)?;
+    generate_pools_file(&path)
+}
+
+fn generate_version_file(parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("version.js"));
+
+    let contents = format!(
+        "{AUTO_GENERATED_DISCLAIMER}
+
+export const VERSION = \"v{VERSION}\";
+"
+    );
+
+    fs::write(path, contents)
+}
+
+fn generate_pools_file(parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("pools.js"));
+
+    let pools = pools();
+
+    let mut contents = format!("{AUTO_GENERATED_DISCLAIMER}\n");
+
+    contents += "
+/**
+ * @typedef {typeof POOL_ID_TO_POOL_NAME} PoolIdToPoolName
+ * @typedef {keyof PoolIdToPoolName} PoolId
+ */
+
+export const POOL_ID_TO_POOL_NAME = /** @type {const} */ ({
+";
+
+    let mut sorted_pools: Vec<_> = pools.iter().collect();
+    sorted_pools.sort_by(|a, b| a.name.to_lowercase().cmp(&b.name.to_lowercase()));
+
+    contents += &sorted_pools
+        .iter()
+        .map(|pool| {
+            let slug = pool.slug();
+            format!("  {slug}: \"{}\",", pool.name)
+        })
+        .collect::<Vec<_>>()
+        .join("\n");
+
+    contents += "\n});\n";
+
+    fs::write(path, contents)
+}
+
+fn generate_metrics_file(query: &Query, parent: &Path) -> io::Result<()> {
+    let path = parent.join(Path::new("metrics.js"));
+
+    let indexes = Index::all();
+
+    let mut contents = format!(
+        "{AUTO_GENERATED_DISCLAIMER}
+
+export const INDEXES = /** @type {{const}} */ ([
+{}
+]);
+
+/**
+ * @typedef {{typeof INDEXES[number]}} IndexName
+ */
+",
+        indexes
+            .iter()
+            .map(|i| format!("  \"{}\"", i.serialize_long()))
+            .collect::<Vec<_>>()
+            .join(",\n")
+    );
+
+    // contents += &indexes
+    //     .iter()
+    //     .map(|i| format!(" * @typedef {{\"{}\"}} {i}", i.serialize_long()))
+    //     .collect::<Vec<_>>()
+    //     .join("\n");
+
+    //     contents += &format!(
+    //         "
+    //  * @typedef {{{}}} Index
+    //  */
+    // ",
+    //         indexes
+    //             .iter()
+    //             .map(|i| i.to_string())
+    //             .collect::<Vec<_>>()
+    //             .join(" | ")
+    //     );
+
+    let mut unique_index_groups = BTreeMap::new();
+
+    let mut word_to_freq: BTreeMap<_, usize> = BTreeMap::new();
+    query.metric_to_index_to_vec().keys().for_each(|metric| {
+        metric.split("_").for_each(|word| {
+            *word_to_freq.entry(word).or_default() += 1;
+        });
+    });
+    let mut word_to_freq = word_to_freq.into_iter().collect::<Vec<_>>();
+    word_to_freq.sort_unstable_by(|a, b| b.1.cmp(&a.1).then(a.0.cmp(b.0)));
+    let words = word_to_freq
+        .into_iter()
+        .map(|(str, _)| str)
+        .collect::<Vec<_>>();
+
+    contents += &format!(
+        "
+export const INDEX_TO_WORD = [
+  {}
+];
+
+",
+        words
+            .iter()
+            .enumerate()
+            .map(|(index, word)| format!("\"{word}\", // {}", index_to_letters(index)))
+            .collect::<Vec<_>>()
+            .join("\n  ")
+    );
+
+    let word_to_base62 = words
+        .into_iter()
+        .enumerate()
+        .map(|(i, w)| (w, index_to_letters(i)))
+        .collect::<HashMap<_, _>>();
+
+    let mut ser_metric_to_indexes = "
+/** @type {Record<string, IndexName[]>} */
+export const COMPRESSED_METRIC_TO_INDEXES = {
+"
+    .to_string();
+
+    query
+        .metric_to_index_to_vec()
+        .iter()
+        .for_each(|(metric, index_to_vec)| {
+            let indexes = index_to_vec
+                .keys()
+                .map(|i| format!("\"{}\"", i.serialize_long()))
+                .collect::<Vec<_>>()
+                .join(", ");
+            let indexes = format!("[{indexes}]");
+            let unique = unique_index_groups.len();
+            let index = index_to_letters(*unique_index_groups.entry(indexes).or_insert(unique));
+
+            let compressed_metric = metric.split('_').fold(String::new(), |mut acc, w| {
+                if !acc.is_empty() {
+                    acc.push('_');
+                }
+                acc.push_str(&word_to_base62[w]);
+                acc
+            });
+
+            ser_metric_to_indexes += &format!("  {compressed_metric}: {index},\n");
+        });
+
+    ser_metric_to_indexes += "};
+";
+
+    let mut sorted_groups: Vec<_> = unique_index_groups.into_iter().collect();
+    sorted_groups.sort_by_key(|(_, index)| *index);
+    sorted_groups.into_iter().for_each(|(group, index)| {
+        let index = index_to_letters(index);
+        contents += &format!("/** @type {{IndexName[]}} */\nconst {index} = {group};\n");
+    });
+
+    contents += &ser_metric_to_indexes;
+
+    fs::write(path, contents)
+}
+
+fn index_to_letters(mut index: usize) -> String {
+    if index < 52 {
+        return (index_to_char(index) as char).to_string();
+    }
+    let mut result = [0u8; 8];
+    let mut pos = 8;
+    loop {
+        pos -= 1;
+        result[pos] = index_to_char(index % 52);
+        index /= 52;
+        if index == 0 {
+            break;
+        }
+        index -= 1;
+    }
+    unsafe { String::from_utf8_unchecked(result[pos..].to_vec()) }
+}
+
+fn index_to_char(index: usize) -> u8 {
+    match index {
+        0..=25 => b'A' + index as u8,
+        26..=51 => b'a' + (index - 26) as u8,
+        _ => unreachable!(),
+    }
+}
+
+// fn letters_to_index(s: &str) -> usize {
+//     let mut result = 0;
+//     for byte in s.bytes() {
+//         let value = char_to_index(byte) as usize;
+//         result = result * 52 + value + 1;
+//     }
+//     result - 1
+// }
+
+// fn char_to_index(byte: u8) -> u8 {
+//     match byte {
+//         b'A'..=b'Z' => byte - b'A',
+//         b'a'..=b'z' => byte - b'a' + 26,
+//         _ => 255, // Invalid
+//     }
+// }

crates/brk_binder/src/lib.rs

@@ -0,0 +1,155 @@
+//! Client library generator for BRK.
+//!
+//! This crate generates typed client libraries in multiple languages (Rust, JavaScript, Python)
+//! from the BRK metric catalog and OpenAPI specification.
+//!
+//! # Usage
+//!
+//! ```ignore
+//! use brk_binder::generate_clients;
+//! use brk_query::Vecs;
+//! use std::path::Path;
+//!
+//! let vecs = Vecs::load("path/to/data")?;
+//! let openapi_json = std::fs::read_to_string("openapi.json")?;
+//! generate_clients(&vecs, &openapi_json, Path::new("output"))?;
+//! ```
+//!
+//! # Architecture
+//!
+//! The generator works in several phases:
+//!
+//! 1. **Metadata extraction** - Analyzes the metric catalog tree to detect:
+//!    - Structural patterns (repeated tree shapes)
+//!    - Index set patterns (common index combinations)
+//!    - Generic patterns (structures that differ only in value type)
+//!
+//! 2. **Schema collection** - Merges OpenAPI schemas with schemars-generated type schemas
+//!
+//! 3. **Code generation** - Produces language-specific clients:
+//!    - Rust: Uses `brk_types` directly, generates structs with lifetimes
+//!    - JavaScript: Generates JSDoc-typed ES modules with factory functions
+//!    - Python: Generates typed classes with TypedDict and Generic support
+
+use std::{collections::btree_map::Entry, fs::create_dir_all, io, path::Path};
+
+use brk_query::Vecs;
+
+mod javascript;
+mod js;
+mod openapi;
+mod python;
+mod rust;
+mod types;
+
+pub use javascript::*;
+pub use js::*;
+pub use openapi::*;
+pub use python::*;
+pub use rust::*;
+pub use types::*;
+
+pub const VERSION: &str = env!("CARGO_PKG_VERSION");
+
+/// Generate all client libraries from the query vecs and OpenAPI JSON
+pub fn generate_clients(vecs: &Vecs, openapi_json: &str, output_dir: &Path) -> io::Result<()> {
+    let metadata = ClientMetadata::from_vecs(vecs);
+
+    // Parse OpenAPI spec
+    let spec = parse_openapi_json(openapi_json)?;
+    let endpoints = extract_endpoints(&spec);
+    let mut schemas = extract_schemas(openapi_json);
+
+    // Collect leaf type schemas from the catalog and merge into schemas
+    collect_leaf_type_schemas(&metadata.catalog, &mut schemas);
+
+    // Also collect definitions from all schemas (including OpenAPI schemas)
+    // We need to do this after collecting leaf schemas so we process everything
+    let schema_values: Vec<_> = schemas.values().cloned().collect();
+    for schema in &schema_values {
+        collect_schema_definitions(schema, &mut schemas);
+    }
+
+    // Generate Rust client (uses real brk_types, no schema conversion needed)
+    let rust_path = output_dir.join("rust");
+    create_dir_all(&rust_path)?;
+    generate_rust_client(&metadata, &endpoints, &rust_path)?;
+
+    // Generate JavaScript client (needs schemas for type definitions)
+    let js_path = output_dir.join("javascript");
+    create_dir_all(&js_path)?;
+    generate_javascript_client(&metadata, &endpoints, &schemas, &js_path)?;
+
+    // Generate Python client (needs schemas for type definitions)
+    let python_path = output_dir.join("python");
+    create_dir_all(&python_path)?;
+    generate_python_client(&metadata, &endpoints, &schemas, &python_path)?;
+
+    Ok(())
+}
+
+use brk_types::TreeNode;
+use serde_json::Value;
+
+/// Recursively collect leaf type schemas from the tree and add to schemas map.
+/// Only adds schemas that aren't already present (OpenAPI schemas take precedence).
+/// Collects definitions from schemars-generated schemas (for referenced types).
+fn collect_leaf_type_schemas(node: &TreeNode, schemas: &mut TypeSchemas) {
+    match node {
+        TreeNode::Leaf(leaf) => {
+            // Collect definitions from the schema (schemars puts type schemas here)
+            // This includes the inner types like `Bitcoin` from `Close<Bitcoin>`
+            collect_schema_definitions(&leaf.schema, schemas);
+
+            // Get the type name for this leaf
+            let type_name = extract_inner_type(leaf.value_type());
+
+            if let Entry::Vacant(e) = schemas.entry(type_name) {
+                // Unwrap single-element allOf
+                let schema = unwrap_allof(&leaf.schema);
+
+                // Add the schema if it's usable:
+                // - Simple type (has "type")
+                // - Object type with properties (complex types like OHLCCents, EmptyAddressData)
+                // - Enum type (has "enum" or "oneOf")
+                // - Or a $ref to another type
+                let has_type = schema.get("type").is_some();
+                let has_properties = schema.get("properties").is_some();
+                let has_enum = schema.get("enum").is_some() || schema.get("oneOf").is_some();
+                let is_ref = schema.get("$ref").is_some();
+
+                if has_type || has_properties || has_enum || is_ref {
+                    e.insert(schema.clone());
+                }
+            }
+        }
+        TreeNode::Branch(children) => {
+            for child in children.values() {
+                collect_leaf_type_schemas(child, schemas);
+            }
+        }
+    }
+}
+
+/// Collect type definitions from schemars-generated schema's definitions section.
+/// Schemars uses `definitions` or `$defs` to store referenced types.
+fn collect_schema_definitions(schema: &Value, schemas: &mut TypeSchemas) {
+    // Check for definitions (JSON Schema draft-07 style)
+    if let Some(defs) = schema.get("definitions").and_then(|d| d.as_object()) {
+        for (name, def_schema) in defs {
+            // Use the definition name as-is (schemars names match $ref paths)
+            if !schemas.contains_key(name) {
+                schemas.insert(name.clone(), def_schema.clone());
+            }
+        }
+    }
+
+    // Check for $defs (JSON Schema draft 2019-09+ style)
+    if let Some(defs) = schema.get("$defs").and_then(|d| d.as_object()) {
+        for (name, def_schema) in defs {
+            if !schemas.contains_key(name) {
+                schemas.insert(name.clone(), def_schema.clone());
+            }
+        }
+    }
+}

crates/brk_binder/src/openapi.rs

@@ -0,0 +1,273 @@
+use std::collections::BTreeMap;
+use std::io;
+
+use oas3::Spec;
+use oas3::spec::{ObjectOrReference, Operation, ParameterIn, PathItem, Schema, SchemaTypeSet};
+use serde_json::Value;
+
+/// Type schema extracted from OpenAPI components
+pub type TypeSchemas = BTreeMap<String, Value>;
+
+/// Endpoint information extracted from OpenAPI spec
+#[derive(Debug, Clone)]
+pub struct Endpoint {
+    /// HTTP method (GET, POST, etc.)
+    pub method: String,
+    /// Path template (e.g., "/blocks/{hash}")
+    pub path: String,
+    /// Operation ID (e.g., "getBlockByHash")
+    pub operation_id: Option<String>,
+    /// Summary/description
+    pub summary: Option<String>,
+    /// Tags for grouping
+    pub tags: Vec<String>,
+    /// Path parameters
+    pub path_params: Vec<Parameter>,
+    /// Query parameters
+    pub query_params: Vec<Parameter>,
+    /// Response type (simplified)
+    pub response_type: Option<String>,
+    /// Whether this endpoint is deprecated
+    pub deprecated: bool,
+}
+
+impl Endpoint {
+    /// Returns true if this endpoint should be included in client generation.
+    /// Only non-deprecated GET endpoints are included.
+    pub fn should_generate(&self) -> bool {
+        self.method == "GET" && !self.deprecated
+    }
+
+    /// Returns the operation ID or generates one from the path.
+    /// The returned string uses the raw case from the spec (typically camelCase).
+    pub fn operation_name(&self) -> String {
+        if let Some(op_id) = &self.operation_id {
+            return op_id.clone();
+        }
+        // Generate from path: /api/blocks/{hash} -> "get_api_blocks_by_hash"
+        let parts: Vec<String> = self
+            .path
+            .split('/')
+            .filter(|s| !s.is_empty())
+            .map(|segment| {
+                if let Some(param) = segment.strip_prefix('{').and_then(|s| s.strip_suffix('}')) {
+                    format!("by_{}", param)
+                } else {
+                    segment.to_string()
+                }
+            })
+            .collect();
+        format!("get_{}", parts.join("_"))
+    }
+}
+
+/// Parameter information
+#[derive(Debug, Clone)]
+pub struct Parameter {
+    pub name: String,
+    pub required: bool,
+    pub param_type: String,
+    pub description: Option<String>,
+}
+
+/// Parse OpenAPI spec from JSON string
+///
+/// Pre-processes the JSON to handle oas3 limitations:
+/// - Removes unsupported siblings from `$ref` objects (oas3 only supports `summary` and `description`)
+pub fn parse_openapi_json(json: &str) -> io::Result<Spec> {
+    let mut value: Value =
+        serde_json::from_str(json).map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;
+
+    // Clean up for oas3 compatibility
+    clean_for_oas3(&mut value);
+
+    let cleaned_json =
+        serde_json::to_string(&value).map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))?;
+
+    oas3::from_json(&cleaned_json).map_err(|e| io::Error::new(io::ErrorKind::InvalidData, e))
+}
+
+/// Extract type schemas from OpenAPI JSON
+pub fn extract_schemas(json: &str) -> TypeSchemas {
+    let Ok(value) = serde_json::from_str::<Value>(json) else {
+        return BTreeMap::new();
+    };
+
+    value
+        .get("components")
+        .and_then(|c| c.get("schemas"))
+        .and_then(|s| s.as_object())
+        .map(|schemas| {
+            schemas
+                .iter()
+                .map(|(name, schema)| (name.clone(), schema.clone()))
+                .collect()
+        })
+        .unwrap_or_default()
+}
+
+/// Clean up OpenAPI spec for oas3 compatibility.
+/// - Removes unsupported siblings from $ref objects (oas3 only supports summary and description)
+/// - Converts boolean schemas to object schemas (oas3 doesn't handle `"schema": true`)
+fn clean_for_oas3(value: &mut Value) {
+    match value {
+        Value::Object(map) => {
+            // Handle $ref with unsupported siblings
+            if map.contains_key("$ref") {
+                map.retain(|k, _| k == "$ref" || k == "summary" || k == "description");
+            } else {
+                // Convert boolean schemas to empty object schemas
+                if let Some(schema) = map.get_mut("schema")
+                    && schema.is_boolean()
+                {
+                    *schema = Value::Object(serde_json::Map::new());
+                }
+                for v in map.values_mut() {
+                    clean_for_oas3(v);
+                }
+            }
+        }
+        Value::Array(arr) => {
+            for v in arr {
+                clean_for_oas3(v);
+            }
+        }
+        _ => {}
+    }
+}
+
+/// Extract all endpoints from OpenAPI spec
+pub fn extract_endpoints(spec: &Spec) -> Vec<Endpoint> {
+    let mut endpoints = Vec::new();
+
+    let Some(paths) = &spec.paths else {
+        return endpoints;
+    };
+
+    for (path, path_item) in paths {
+        for (method, operation) in get_operations(path_item) {
+            if let Some(endpoint) = extract_endpoint(path, &method, operation) {
+                endpoints.push(endpoint);
+            }
+        }
+    }
+
+    endpoints
+}
+
+fn get_operations(path_item: &PathItem) -> Vec<(String, &Operation)> {
+    let mut ops = Vec::new();
+    if let Some(op) = &path_item.get {
+        ops.push(("GET".to_string(), op));
+    }
+    if let Some(op) = &path_item.post {
+        ops.push(("POST".to_string(), op));
+    }
+    if let Some(op) = &path_item.put {
+        ops.push(("PUT".to_string(), op));
+    }
+    if let Some(op) = &path_item.delete {
+        ops.push(("DELETE".to_string(), op));
+    }
+    if let Some(op) = &path_item.patch {
+        ops.push(("PATCH".to_string(), op));
+    }
+    ops
+}
+
+fn extract_endpoint(path: &str, method: &str, operation: &Operation) -> Option<Endpoint> {
+    let path_params = extract_parameters(operation, ParameterIn::Path);
+    let query_params = extract_parameters(operation, ParameterIn::Query);
+
+    let response_type = extract_response_type(operation);
+
+    Some(Endpoint {
+        method: method.to_string(),
+        path: path.to_string(),
+        operation_id: operation.operation_id.clone(),
+        summary: operation
+            .summary
+            .clone()
+            .or_else(|| operation.description.clone()),
+        tags: operation.tags.clone(),
+        path_params,
+        query_params,
+        response_type,
+        deprecated: operation.deprecated.unwrap_or(false),
+    })
+}
+
+fn extract_parameters(operation: &Operation, location: ParameterIn) -> Vec<Parameter> {
+    operation
+        .parameters
+        .iter()
+        .filter_map(|p| match p {
+            ObjectOrReference::Object(param) if param.location == location => Some(Parameter {
+                name: param.name.clone(),
+                required: param.required.unwrap_or(false),
+                param_type: "string".to_string(), // Simplified
+                description: param.description.clone(),
+            }),
+            _ => None,
+        })
+        .collect()
+}
+
+fn extract_response_type(operation: &Operation) -> Option<String> {
+    let responses = operation.responses.as_ref()?;
+
+    // Look for 200 OK response
+    let response = responses.get("200")?;
+
+    match response {
+        ObjectOrReference::Object(response) => {
+            // Look for JSON content
+            let content = response.content.get("application/json")?;
+
+            match &content.schema {
+                Some(ObjectOrReference::Ref { ref_path, .. }) => {
+                    // Extract type name from reference like "#/components/schemas/Block"
+                    Some(ref_path.rsplit('/').next()?.to_string())
+                }
+                Some(ObjectOrReference::Object(schema)) => schema_to_type_name(schema),
+                None => None,
+            }
+        }
+        ObjectOrReference::Ref { .. } => None,
+    }
+}
+
+fn schema_type_from_schema(schema: &Schema) -> Option<String> {
+    match schema {
+        Schema::Boolean(_) => Some("boolean".to_string()),
+        Schema::Object(obj_or_ref) => match obj_or_ref.as_ref() {
+            ObjectOrReference::Object(obj_schema) => schema_to_type_name(obj_schema),
+            ObjectOrReference::Ref { ref_path, .. } => {
+                ref_path.rsplit('/').next().map(|s| s.to_string())
+            }
+        },
+    }
+}
+
+fn schema_to_type_name(schema: &oas3::spec::ObjectSchema) -> Option<String> {
+    let schema_type = schema.schema_type.as_ref()?;
+
+    match schema_type {
+        SchemaTypeSet::Single(t) => match t {
+            oas3::spec::SchemaType::String => Some("string".to_string()),
+            oas3::spec::SchemaType::Number => Some("number".to_string()),
+            oas3::spec::SchemaType::Integer => Some("number".to_string()),
+            oas3::spec::SchemaType::Boolean => Some("boolean".to_string()),
+            oas3::spec::SchemaType::Array => {
+                let inner = match &schema.items {
+                    Some(boxed_schema) => schema_type_from_schema(boxed_schema),
+                    None => Some("*".to_string()),
+                };
+                inner.map(|t| format!("{}[]", t))
+            }
+            oas3::spec::SchemaType::Object => Some("Object".to_string()),
+            oas3::spec::SchemaType::Null => Some("null".to_string()),
+        },
+        SchemaTypeSet::Multiple(_) => Some("*".to_string()),
+    }
+}

crates/brk_binder/src/python.rs

@@ -0,0 +1,962 @@
+use std::collections::HashSet;
+use std::fmt::Write as FmtWrite;
+use std::fs;
+use std::io;
+use std::path::Path;
+
+use brk_types::{Index, TreeNode};
+use serde_json::Value;
+
+use crate::{
+    ClientMetadata, Endpoint, FieldNamePosition, IndexSetPattern, PatternField, StructuralPattern,
+    TypeSchemas, extract_inner_type, get_fields_with_child_info, get_node_fields,
+    get_pattern_instance_base, is_enum_schema, to_pascal_case, to_snake_case,
+};
+
+/// Generate Python client from metadata and OpenAPI endpoints
+pub fn generate_python_client(
+    metadata: &ClientMetadata,
+    endpoints: &[Endpoint],
+    schemas: &TypeSchemas,
+    output_dir: &Path,
+) -> io::Result<()> {
+    let mut output = String::new();
+
+    // Header
+    writeln!(output, "# Auto-generated BRK Python client").unwrap();
+    writeln!(output, "# Do not edit manually\n").unwrap();
+    writeln!(output, "from __future__ import annotations").unwrap();
+    writeln!(
+        output,
+        "from typing import TypeVar, Generic, Any, Optional, List, Literal, TypedDict"
+    )
+    .unwrap();
+    writeln!(output, "import httpx\n").unwrap();
+
+    // Type variable for generic MetricNode
+    writeln!(output, "T = TypeVar('T')\n").unwrap();
+
+    // Generate type definitions from OpenAPI schemas (now includes leaf types from catalog)
+    generate_type_definitions(&mut output, schemas);
+
+    // Generate base client class
+    generate_base_client(&mut output);
+
+    // Generate MetricNode class
+    generate_metric_node(&mut output);
+
+    // Generate index accessor classes
+    generate_index_accessors(&mut output, &metadata.index_set_patterns);
+
+    // Generate structural pattern classes
+    generate_structural_patterns(&mut output, &metadata.structural_patterns, metadata);
+
+    // Generate tree classes
+    generate_tree_classes(&mut output, &metadata.catalog, metadata);
+
+    // Generate main client with tree and API methods
+    generate_main_client(&mut output, endpoints);
+
+    fs::write(output_dir.join("client.py"), output)?;
+
+    Ok(())
+}
+
+/// Generate Python type definitions from OpenAPI schemas
+fn generate_type_definitions(output: &mut String, schemas: &TypeSchemas) {
+    if schemas.is_empty() {
+        return;
+    }
+
+    writeln!(output, "# Type definitions\n").unwrap();
+
+    // Sort types by dependencies (types that reference other types must come after)
+    let sorted_names = topological_sort_schemas(schemas);
+
+    for name in sorted_names {
+        let Some(schema) = schemas.get(&name) else {
+            continue;
+        };
+        if let Some(props) = schema.get("properties").and_then(|p| p.as_object()) {
+            // Object type -> TypedDict
+            writeln!(output, "class {}(TypedDict):", name).unwrap();
+            for (prop_name, prop_schema) in props {
+                let prop_type = schema_to_python_type_ctx(prop_schema, Some(&name));
+                let safe_name = escape_python_keyword(prop_name);
+                writeln!(output, "    {}: {}", safe_name, prop_type).unwrap();
+            }
+            writeln!(output).unwrap();
+        } else if is_enum_schema(schema) {
+            // Enum type -> Literal union
+            let py_type = schema_to_python_type_ctx(schema, Some(&name));
+            writeln!(output, "{} = {}", name, py_type).unwrap();
+        } else {
+            // Primitive type alias
+            let py_type = schema_to_python_type_ctx(schema, Some(&name));
+            writeln!(output, "{} = {}", name, py_type).unwrap();
+        }
+    }
+    writeln!(output).unwrap();
+}
+
+/// Topologically sort schema names so dependencies come before dependents.
+/// Types that reference other types (via $ref) must be defined after their dependencies.
+fn topological_sort_schemas(schemas: &TypeSchemas) -> Vec<String> {
+    use std::collections::{HashMap, HashSet};
+
+    // Build dependency graph
+    let mut deps: HashMap<String, HashSet<String>> = HashMap::new();
+    for (name, schema) in schemas {
+        let mut type_deps = HashSet::new();
+        collect_schema_refs(schema, &mut type_deps);
+        // Only keep deps that are in our schemas
+        type_deps.retain(|d| schemas.contains_key(d));
+        deps.insert(name.clone(), type_deps);
+    }
+
+    // Kahn's algorithm for topological sort
+    let mut in_degree: HashMap<String, usize> = HashMap::new();
+    for name in schemas.keys() {
+        in_degree.insert(name.clone(), 0);
+    }
+    for type_deps in deps.values() {
+        for dep in type_deps {
+            *in_degree.entry(dep.clone()).or_insert(0) += 1;
+        }
+    }
+
+    // Start with types that have no dependents (are not referenced by others)
+    let mut queue: Vec<String> = in_degree
+        .iter()
+        .filter(|(_, count)| **count == 0)
+        .map(|(name, _)| name.clone())
+        .collect();
+    queue.sort(); // Deterministic order
+
+    let mut result = Vec::new();
+    while let Some(name) = queue.pop() {
+        result.push(name.clone());
+        if let Some(type_deps) = deps.get(&name) {
+            for dep in type_deps {
+                if let Some(count) = in_degree.get_mut(dep) {
+                    *count = count.saturating_sub(1);
+                    if *count == 0 {
+                        queue.push(dep.clone());
+                        queue.sort(); // Keep sorted for determinism
+                    }
+                }
+            }
+        }
+    }
+
+    // Reverse so dependencies come first
+    result.reverse();
+
+    // Add any types that weren't processed (e.g., due to circular refs or other edge cases)
+    let result_set: HashSet<_> = result.iter().cloned().collect();
+    let mut missing: Vec<_> = schemas
+        .keys()
+        .filter(|k| !result_set.contains(*k))
+        .cloned()
+        .collect();
+    missing.sort();
+    result.extend(missing);
+
+    result
+}
+
+/// Collect all type references ($ref) from a schema
+fn collect_schema_refs(schema: &Value, refs: &mut std::collections::HashSet<String>) {
+    match schema {
+        Value::Object(map) => {
+            if let Some(ref_path) = map.get("$ref").and_then(|r| r.as_str())
+                && let Some(type_name) = ref_path.rsplit('/').next()
+            {
+                refs.insert(type_name.to_string());
+            }
+            for value in map.values() {
+                collect_schema_refs(value, refs);
+            }
+        }
+        Value::Array(arr) => {
+            for item in arr {
+                collect_schema_refs(item, refs);
+            }
+        }
+        _ => {}
+    }
+}
+
+/// Convert a single JSON type string to Python type
+fn json_type_to_python(ty: &str, schema: &Value, current_type: Option<&str>) -> String {
+    match ty {
+        "integer" => "int".to_string(),
+        "number" => "float".to_string(),
+        "boolean" => "bool".to_string(),
+        "string" => "str".to_string(),
+        "null" => "None".to_string(),
+        "array" => {
+            let item_type = schema
+                .get("items")
+                .map(|s| schema_to_python_type_ctx(s, current_type))
+                .unwrap_or_else(|| "Any".to_string());
+            format!("List[{}]", item_type)
+        }
+        "object" => {
+            // Check if it has additionalProperties (dict-like)
+            if let Some(add_props) = schema.get("additionalProperties") {
+                let value_type = schema_to_python_type_ctx(add_props, current_type);
+                return format!("dict[str, {}]", value_type);
+            }
+            "dict".to_string()
+        }
+        _ => "Any".to_string(),
+    }
+}
+
+/// Convert JSON Schema to Python type with context for detecting self-references
+fn schema_to_python_type_ctx(schema: &Value, current_type: Option<&str>) -> String {
+    // Handle allOf (try each element until we find a resolvable type)
+    if let Some(all_of) = schema.get("allOf").and_then(|v| v.as_array()) {
+        for item in all_of {
+            let resolved = schema_to_python_type_ctx(item, current_type);
+            if resolved != "Any" {
+                return resolved;
+            }
+        }
+    }
+
+    // Handle $ref
+    if let Some(ref_path) = schema.get("$ref").and_then(|r| r.as_str()) {
+        let type_name = ref_path.rsplit('/').next().unwrap_or("Any");
+        // Quote self-references to handle recursive types
+        if current_type == Some(type_name) {
+            return format!("\"{}\"", type_name);
+        }
+        return type_name.to_string();
+    }
+
+    // Handle enum (array of string values)
+    if let Some(enum_values) = schema.get("enum").and_then(|e| e.as_array()) {
+        let literals: Vec<String> = enum_values
+            .iter()
+            .filter_map(|v| v.as_str())
+            .map(|s| format!("\"{}\"", s))
+            .collect();
+        if !literals.is_empty() {
+            return format!("Literal[{}]", literals.join(", "));
+        }
+    }
+
+    // Handle type field (can be string or array of strings)
+    if let Some(ty) = schema.get("type") {
+        // Handle array of types like ["string", "null"] for Optional
+        if let Some(type_array) = ty.as_array() {
+            let types: Vec<String> = type_array
+                .iter()
+                .filter_map(|t| t.as_str())
+                .filter(|t| *t != "null") // Filter out null for cleaner Optional handling
+                .map(|t| json_type_to_python(t, schema, current_type))
+                .collect();
+            let has_null = type_array.iter().any(|t| t.as_str() == Some("null"));
+
+            if types.len() == 1 {
+                let base_type = &types[0];
+                return if has_null {
+                    format!("Optional[{}]", base_type)
+                } else {
+                    base_type.clone()
+                };
+            } else if !types.is_empty() {
+                let union = types.join(" | ");
+                return if has_null {
+                    format!("Optional[{}]", union)
+                } else {
+                    union
+                };
+            }
+        }
+
+        // Handle single type string
+        if let Some(ty_str) = ty.as_str() {
+            return json_type_to_python(ty_str, schema, current_type);
+        }
+    }
+
+    // Handle anyOf/oneOf
+    if let Some(variants) = schema
+        .get("anyOf")
+        .or_else(|| schema.get("oneOf"))
+        .and_then(|v| v.as_array())
+    {
+        let types: Vec<String> = variants
+            .iter()
+            .map(|v| schema_to_python_type_ctx(v, current_type))
+            .collect();
+        // Filter out Any and null for cleaner unions
+        let filtered: Vec<_> = types.iter().filter(|t| *t != "Any").collect();
+        if !filtered.is_empty() {
+            return filtered
+                .iter()
+                .map(|s| s.as_str())
+                .collect::<Vec<_>>()
+                .join(" | ");
+        }
+        return types.join(" | ");
+    }
+
+    // Check for format hint without type (common in OpenAPI)
+    if let Some(format) = schema.get("format").and_then(|f| f.as_str()) {
+        return match format {
+            "int32" | "int64" => "int".to_string(),
+            "float" | "double" => "float".to_string(),
+            "date" | "date-time" => "str".to_string(),
+            _ => "Any".to_string(),
+        };
+    }
+
+    "Any".to_string()
+}
+
+/// Make a name safe for Python: escape keywords and prefix digit-starting names
+fn escape_python_keyword(name: &str) -> String {
+    const PYTHON_KEYWORDS: &[&str] = &[
+        "False", "None", "True", "and", "as", "assert", "async", "await", "break", "class",
+        "continue", "def", "del", "elif", "else", "except", "finally", "for", "from", "global",
+        "if", "import", "in", "is", "lambda", "nonlocal", "not", "or", "pass", "raise", "return",
+        "try", "while", "with", "yield",
+    ];
+    // Names starting with digit need underscore prefix
+    let name = if name
+        .chars()
+        .next()
+        .map(|c| c.is_ascii_digit())
+        .unwrap_or(false)
+    {
+        format!("_{}", name)
+    } else {
+        name.to_string()
+    };
+    // Reserved keywords get underscore suffix
+    if PYTHON_KEYWORDS.contains(&name.as_str()) {
+        format!("{}_", name)
+    } else {
+        name
+    }
+}
+
+/// Generate the base BrkClient class with HTTP functionality
+fn generate_base_client(output: &mut String) {
+    writeln!(
+        output,
+        r#"class BrkError(Exception):
+    """Custom error class for BRK client errors."""
+
+    def __init__(self, message: str, status: Optional[int] = None):
+        super().__init__(message)
+        self.status = status
+
+
+class BrkClientBase:
+    """Base HTTP client for making requests."""
+
+    def __init__(self, base_url: str, timeout: float = 30.0):
+        self.base_url = base_url
+        self.timeout = timeout
+        self._client = httpx.Client(timeout=timeout)
+
+    def get(self, path: str) -> Any:
+        """Make a GET request."""
+        try:
+            response = self._client.get(f"{{self.base_url}}{{path}}")
+            response.raise_for_status()
+            return response.json()
+        except httpx.HTTPStatusError as e:
+            raise BrkError(f"HTTP error: {{e.response.status_code}}", e.response.status_code)
+        except httpx.RequestError as e:
+            raise BrkError(str(e))
+
+    def close(self):
+        """Close the HTTP client."""
+        self._client.close()
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.close()
+
+"#
+    )
+    .unwrap();
+}
+
+/// Generate the MetricNode class
+fn generate_metric_node(output: &mut String) {
+    writeln!(
+        output,
+        r#"class MetricNode(Generic[T]):
+    """A metric node that can fetch data for different indexes."""
+
+    def __init__(self, client: BrkClientBase, path: str):
+        self._client = client
+        self._path = path
+
+    def get(self) -> List[T]:
+        """Fetch all data points for this metric."""
+        return self._client.get(self._path)
+
+    def get_range(self, from_date: str, to_date: str) -> List[T]:
+        """Fetch data points within a date range."""
+        return self._client.get(f"{{self._path}}?from={{from_date}}&to={{to_date}}")
+
+"#
+    )
+    .unwrap();
+}
+
+/// Generate index accessor classes
+fn generate_index_accessors(output: &mut String, patterns: &[IndexSetPattern]) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "# Index accessor classes\n").unwrap();
+
+    for pattern in patterns {
+        writeln!(output, "class {}(Generic[T]):", pattern.name).unwrap();
+        writeln!(
+            output,
+            "    \"\"\"Index accessor for metrics with {} indexes.\"\"\"",
+            pattern.indexes.len()
+        )
+        .unwrap();
+        writeln!(output, "    ").unwrap();
+        writeln!(
+            output,
+            "    def __init__(self, client: BrkClientBase, base_path: str):"
+        )
+        .unwrap();
+
+        for index in &pattern.indexes {
+            let field_name = index_to_snake_case(index);
+            let path_segment = index.serialize_long();
+            writeln!(
+                output,
+                "        self.{}: MetricNode[T] = MetricNode(client, f'{{base_path}}/{}')",
+                field_name, path_segment
+            )
+            .unwrap();
+        }
+
+        writeln!(output).unwrap();
+    }
+}
+
+/// Convert an Index to a snake_case field name (e.g., DateIndex -> by_date_index)
+fn index_to_snake_case(index: &Index) -> String {
+    format!("by_{}", to_snake_case(index.serialize_long()))
+}
+
+/// Generate structural pattern classes
+fn generate_structural_patterns(
+    output: &mut String,
+    patterns: &[StructuralPattern],
+    metadata: &ClientMetadata,
+) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "# Reusable structural pattern classes\n").unwrap();
+
+    for pattern in patterns {
+        let is_parameterizable = pattern.is_parameterizable();
+
+        // For generic patterns, inherit from Generic[T]
+        if pattern.is_generic {
+            writeln!(output, "class {}(Generic[T]):", pattern.name).unwrap();
+        } else {
+            writeln!(output, "class {}:", pattern.name).unwrap();
+        }
+        writeln!(
+            output,
+            "    \"\"\"Pattern struct for repeated tree structure.\"\"\""
+        )
+        .unwrap();
+        writeln!(output, "    ").unwrap();
+
+        if is_parameterizable {
+            writeln!(
+                output,
+                "    def __init__(self, client: BrkClientBase, acc: str):"
+            )
+            .unwrap();
+            writeln!(
+                output,
+                "        \"\"\"Create pattern node with accumulated metric name.\"\"\""
+            )
+            .unwrap();
+        } else {
+            writeln!(
+                output,
+                "    def __init__(self, client: BrkClientBase, base_path: str):"
+            )
+            .unwrap();
+        }
+
+        for field in &pattern.fields {
+            if is_parameterizable {
+                generate_parameterized_python_field(output, field, pattern, metadata);
+            } else {
+                generate_tree_path_python_field(output, field, metadata);
+            }
+        }
+
+        writeln!(output).unwrap();
+    }
+}
+
+/// Generate a field using parameterized (prepend/append) metric name construction
+fn generate_parameterized_python_field(
+    output: &mut String,
+    field: &PatternField,
+    pattern: &StructuralPattern,
+    metadata: &ClientMetadata,
+) {
+    let field_name = to_snake_case(&field.name);
+    let py_type = field_to_python_type_generic(field, metadata, pattern.is_generic);
+
+    // For branch fields, pass the accumulated name to nested pattern
+    if metadata.is_pattern_type(&field.rust_type) {
+        let child_acc = if let Some(pos) = pattern.get_field_position(&field.name) {
+            match pos {
+                FieldNamePosition::Append(suffix) => format!("f'{{acc}}{}'", suffix),
+                FieldNamePosition::Prepend(prefix) => format!("f'{}{{acc}}'", prefix),
+                FieldNamePosition::Identity => "acc".to_string(),
+                FieldNamePosition::SetBase(base) => format!("'{}'", base),
+            }
+        } else {
+            format!("f'{{acc}}_{}'", field.name)
+        };
+
+        writeln!(
+            output,
+            "        self.{}: {} = {}(client, {})",
+            field_name, py_type, field.rust_type, child_acc
+        )
+        .unwrap();
+        return;
+    }
+
+    // For leaf fields, construct the metric path based on position
+    let metric_expr = if let Some(pos) = pattern.get_field_position(&field.name) {
+        match pos {
+            FieldNamePosition::Append(suffix) => format!("f'/{{acc}}{}'", suffix),
+            FieldNamePosition::Prepend(prefix) => format!("f'/{}{{acc}}'", prefix),
+            FieldNamePosition::Identity => "f'/{acc}'".to_string(),
+            FieldNamePosition::SetBase(base) => format!("'/{}'", base),
+        }
+    } else {
+        format!("f'/{{acc}}_{}'", field.name)
+    };
+
+    if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "        self.{}: {} = {}(client, {})",
+            field_name, py_type, accessor.name, metric_expr
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "        self.{}: {} = MetricNode(client, {})",
+            field_name, py_type, metric_expr
+        )
+        .unwrap();
+    }
+}
+
+/// Generate a field using tree path construction (fallback for non-parameterizable patterns)
+fn generate_tree_path_python_field(
+    output: &mut String,
+    field: &PatternField,
+    metadata: &ClientMetadata,
+) {
+    let field_name = to_snake_case(&field.name);
+    let py_type = field_to_python_type(field, metadata);
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        writeln!(
+            output,
+            "        self.{}: {} = {}(client, f'{{base_path}}/{}')",
+            field_name, py_type, field.rust_type, field.name
+        )
+        .unwrap();
+    } else if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "        self.{}: {} = {}(client, f'{{base_path}}/{}')",
+            field_name, py_type, accessor.name, field.name
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "        self.{}: {} = MetricNode(client, f'{{base_path}}/{}')",
+            field_name, py_type, field.name
+        )
+        .unwrap();
+    }
+}
+
+/// Convert pattern field to Python type annotation
+fn field_to_python_type(field: &PatternField, metadata: &ClientMetadata) -> String {
+    field_to_python_type_generic(field, metadata, false)
+}
+
+/// Convert pattern field to Python type annotation, with optional generic support
+fn field_to_python_type_generic(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+) -> String {
+    field_to_python_type_with_generic_value(field, metadata, is_generic, None)
+}
+
+/// Convert pattern field to Python type annotation.
+/// - `is_generic`: If true and field.rust_type is "T", use T in the output
+/// - `generic_value_type`: For branch fields that reference a generic pattern, this is the concrete type to substitute
+fn field_to_python_type_with_generic_value(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+    generic_value_type: Option<&str>,
+) -> String {
+    // For generic patterns, use T instead of concrete value type
+    // Also extract inner type from wrappers like Close<Dollars> -> Dollars
+    let value_type = if is_generic && field.rust_type == "T" {
+        "T".to_string()
+    } else {
+        extract_inner_type(&field.rust_type)
+    };
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        // Check if this pattern is generic and we have a value type
+        if metadata.is_pattern_generic(&field.rust_type)
+            && let Some(vt) = generic_value_type
+        {
+            return format!("{}[{}]", field.rust_type, vt);
+        }
+        field.rust_type.clone()
+    } else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
+        // Leaf with accessor - use value_type as the generic
+        format!("{}[{}]", accessor.name, value_type)
+    } else {
+        // Leaf - use value_type as the generic
+        format!("MetricNode[{}]", value_type)
+    }
+}
+
+
+/// Generate tree classes
+fn generate_tree_classes(output: &mut String, catalog: &TreeNode, metadata: &ClientMetadata) {
+    writeln!(output, "# Catalog tree classes\n").unwrap();
+
+    let pattern_lookup = metadata.pattern_lookup();
+    let mut generated = HashSet::new();
+    generate_tree_class(
+        output,
+        "CatalogTree",
+        catalog,
+        &pattern_lookup,
+        metadata,
+        &mut generated,
+    );
+}
+
+/// Recursively generate tree classes
+fn generate_tree_class(
+    output: &mut String,
+    name: &str,
+    node: &TreeNode,
+    pattern_lookup: &std::collections::HashMap<Vec<PatternField>, String>,
+    metadata: &ClientMetadata,
+    generated: &mut HashSet<String>,
+) {
+    let TreeNode::Branch(children) = node else {
+        return;
+    };
+
+    let fields_with_child_info = get_fields_with_child_info(children, name, pattern_lookup);
+    let fields: Vec<PatternField> = fields_with_child_info
+        .iter()
+        .map(|(f, _)| f.clone())
+        .collect();
+
+    // Skip if this matches a pattern (already generated)
+    if pattern_lookup.contains_key(&fields) && pattern_lookup.get(&fields) != Some(&name.to_string())
+    {
+        return;
+    }
+
+    if generated.contains(name) {
+        return;
+    }
+    generated.insert(name.to_string());
+
+    writeln!(output, "class {}:", name).unwrap();
+    writeln!(output, "    \"\"\"Catalog tree node.\"\"\"").unwrap();
+    writeln!(output, "    ").unwrap();
+    writeln!(
+        output,
+        "    def __init__(self, client: BrkClientBase, base_path: str = ''):"
+    )
+    .unwrap();
+
+    for ((field, child_fields_opt), (child_name, child_node)) in
+        fields_with_child_info.iter().zip(children.iter())
+    {
+        let generic_value_type = child_fields_opt
+            .as_ref()
+            .and_then(|cf| metadata.get_generic_value_type(&field.rust_type, cf));
+        let py_type = field_to_python_type_with_generic_value(
+            field,
+            metadata,
+            false,
+            generic_value_type.as_deref(),
+        );
+        let field_name_py = to_snake_case(&field.name);
+
+        if metadata.is_pattern_type(&field.rust_type) {
+            let pattern = metadata.find_pattern(&field.rust_type);
+            let is_parameterizable = pattern.is_some_and(|p| p.is_parameterizable());
+
+            if is_parameterizable {
+                let metric_base = get_pattern_instance_base(child_node, child_name);
+                writeln!(
+                    output,
+                    "        self.{}: {} = {}(client, '{}')",
+                    field_name_py, py_type, field.rust_type, metric_base
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "        self.{}: {} = {}(client, f'{{base_path}}/{}')",
+                    field_name_py, py_type, field.rust_type, field.name
+                )
+                .unwrap();
+            }
+        } else if metadata.field_uses_accessor(field) {
+            let metric_path = if let TreeNode::Leaf(leaf) = child_node {
+                format!("/{}", leaf.name())
+            } else {
+                format!("{{base_path}}/{}", field.name)
+            };
+            let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+            if metric_path.contains("{base_path}") {
+                writeln!(
+                    output,
+                    "        self.{}: {} = {}(client, f'{}')",
+                    field_name_py, py_type, accessor.name, metric_path
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "        self.{}: {} = {}(client, '{}')",
+                    field_name_py, py_type, accessor.name, metric_path
+                )
+                .unwrap();
+            }
+        } else {
+            let metric_path = if let TreeNode::Leaf(leaf) = child_node {
+                format!("/{}", leaf.name())
+            } else {
+                format!("{{base_path}}/{}", field.name)
+            };
+            if metric_path.contains("{base_path}") {
+                writeln!(
+                    output,
+                    "        self.{}: {} = MetricNode(client, f'{}')",
+                    field_name_py, py_type, metric_path
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "        self.{}: {} = MetricNode(client, '{}')",
+                    field_name_py, py_type, metric_path
+                )
+                .unwrap();
+            }
+        }
+    }
+
+    writeln!(output).unwrap();
+
+    // Generate child classes
+    for (child_name, child_node) in children {
+        if let TreeNode::Branch(grandchildren) = child_node {
+            let child_fields = get_node_fields(grandchildren, pattern_lookup);
+            if !pattern_lookup.contains_key(&child_fields) {
+                let child_class_name = format!("{}_{}", name, to_pascal_case(child_name));
+                generate_tree_class(
+                    output,
+                    &child_class_name,
+                    child_node,
+                    pattern_lookup,
+                    metadata,
+                    generated,
+                );
+            }
+        }
+    }
+}
+
+/// Generate the main client class
+fn generate_main_client(output: &mut String, endpoints: &[Endpoint]) {
+    writeln!(output, "class BrkClient(BrkClientBase):").unwrap();
+    writeln!(
+        output,
+        "    \"\"\"Main BRK client with catalog tree and API methods.\"\"\""
+    )
+    .unwrap();
+    writeln!(output, "    ").unwrap();
+    writeln!(
+        output,
+        "    def __init__(self, base_url: str = 'http://localhost:3000', timeout: float = 30.0):"
+    )
+    .unwrap();
+    writeln!(output, "        super().__init__(base_url, timeout)").unwrap();
+    writeln!(output, "        self.tree = CatalogTree(self)").unwrap();
+    writeln!(output).unwrap();
+
+    // Generate API methods
+    generate_api_methods(output, endpoints);
+}
+
+/// Generate API methods from OpenAPI endpoints
+fn generate_api_methods(output: &mut String, endpoints: &[Endpoint]) {
+    for endpoint in endpoints {
+        if !endpoint.should_generate() {
+            continue;
+        }
+
+        let method_name = endpoint_to_method_name(endpoint);
+        let return_type = endpoint
+            .response_type
+            .as_deref()
+            .map(js_type_to_python)
+            .unwrap_or_else(|| "Any".to_string());
+
+        // Build method signature
+        let params = build_method_params(endpoint);
+        writeln!(
+            output,
+            "    def {}(self{}) -> {}:",
+            method_name, params, return_type
+        )
+        .unwrap();
+
+        // Docstring
+        if let Some(summary) = &endpoint.summary {
+            writeln!(output, "        \"\"\"{}\"\"\"", summary).unwrap();
+        }
+
+        // Build path
+        let path = build_path_template(&endpoint.path, &endpoint.path_params);
+
+        if endpoint.query_params.is_empty() {
+            if endpoint.path_params.is_empty() {
+                writeln!(output, "        return self.get('{}')", path).unwrap();
+            } else {
+                writeln!(output, "        return self.get(f'{}')", path).unwrap();
+            }
+        } else {
+            writeln!(output, "        params = []").unwrap();
+            for param in &endpoint.query_params {
+                // Use safe name for Python variable, original name for API query parameter
+                let safe_name = escape_python_keyword(&param.name);
+                if param.required {
+                    writeln!(
+                        output,
+                        "        params.append(f'{}={{{}}}')",
+                        param.name, safe_name
+                    )
+                    .unwrap();
+                } else {
+                    writeln!(
+                        output,
+                        "        if {} is not None: params.append(f'{}={{{}}}')",
+                        safe_name, param.name, safe_name
+                    )
+                    .unwrap();
+                }
+            }
+            writeln!(output, "        query = '&'.join(params)").unwrap();
+            writeln!(
+                output,
+                "        return self.get(f'{}{{\"?\" + query if query else \"\"}}')",
+                path
+            )
+            .unwrap();
+        }
+
+        writeln!(output).unwrap();
+    }
+}
+
+fn endpoint_to_method_name(endpoint: &Endpoint) -> String {
+    to_snake_case(&endpoint.operation_name())
+}
+
+/// Convert JS-style type to Python type (e.g., "Txid[]" -> "List[Txid]", "number" -> "int")
+fn js_type_to_python(js_type: &str) -> String {
+    if let Some(inner) = js_type.strip_suffix("[]") {
+        format!("List[{}]", js_type_to_python(inner))
+    } else {
+        match js_type {
+            "number" => "int".to_string(),
+            "boolean" => "bool".to_string(),
+            "string" => "str".to_string(),
+            "null" => "None".to_string(),
+            "Object" | "object" => "dict".to_string(),
+            "*" => "Any".to_string(),
+            _ => js_type.to_string(),
+        }
+    }
+}
+
+fn build_method_params(endpoint: &Endpoint) -> String {
+    let mut params = Vec::new();
+    for param in &endpoint.path_params {
+        let safe_name = escape_python_keyword(&param.name);
+        params.push(format!(", {}: str", safe_name));
+    }
+    for param in &endpoint.query_params {
+        let safe_name = escape_python_keyword(&param.name);
+        if param.required {
+            params.push(format!(", {}: str", safe_name));
+        } else {
+            params.push(format!(", {}: Optional[str] = None", safe_name));
+        }
+    }
+    params.join("")
+}
+
+fn build_path_template(path: &str, path_params: &[super::Parameter]) -> String {
+    let mut result = path.to_string();
+    for param in path_params {
+        let placeholder = format!("{{{}}}", param.name);
+        // Use escaped name for Python variable interpolation in f-string
+        let safe_name = escape_python_keyword(&param.name);
+        let interpolation = format!("{{{}}}", safe_name);
+        result = result.replace(&placeholder, &interpolation);
+    }
+    result
+}

crates/brk_binder/src/rust.rs

@@ -0,0 +1,770 @@
+use std::collections::HashSet;
+use std::fmt::Write as FmtWrite;
+use std::fs;
+use std::io;
+use std::path::Path;
+
+use brk_types::{Index, TreeNode};
+
+use crate::{
+    ClientMetadata, Endpoint, FieldNamePosition, IndexSetPattern, PatternField, StructuralPattern,
+    extract_inner_type, get_fields_with_child_info, get_node_fields, get_pattern_instance_base,
+    to_pascal_case, to_snake_case,
+};
+
+/// Generate Rust client from metadata and OpenAPI endpoints
+pub fn generate_rust_client(
+    metadata: &ClientMetadata,
+    endpoints: &[Endpoint],
+    output_dir: &Path,
+) -> io::Result<()> {
+    let mut output = String::new();
+
+    // Header
+    writeln!(output, "// Auto-generated BRK Rust client").unwrap();
+    writeln!(output, "// Do not edit manually\n").unwrap();
+    writeln!(output, "#![allow(non_camel_case_types)]").unwrap();
+    writeln!(output, "#![allow(dead_code)]\n").unwrap();
+
+    // Imports
+    generate_imports(&mut output);
+
+    // Generate base client
+    generate_base_client(&mut output);
+
+    // Generate MetricNode
+    generate_metric_node(&mut output);
+
+    // Generate index accessor structs (for each unique set of indexes)
+    generate_index_accessors(&mut output, &metadata.index_set_patterns);
+
+    // Generate pattern structs (reusable, appearing 2+ times)
+    generate_pattern_structs(&mut output, &metadata.structural_patterns, metadata);
+
+    // Generate tree - each node uses its pattern or is generated inline
+    generate_tree(&mut output, &metadata.catalog, metadata);
+
+    // Generate main client with API methods
+    generate_main_client(&mut output, endpoints);
+
+    fs::write(output_dir.join("client.rs"), output)?;
+
+    Ok(())
+}
+
+fn generate_imports(output: &mut String) {
+    writeln!(
+        output,
+        r#"use std::marker::PhantomData;
+use serde::de::DeserializeOwned;
+use brk_types::*;
+
+"#
+    )
+    .unwrap();
+}
+
+fn generate_base_client(output: &mut String) {
+    writeln!(
+        output,
+        r#"/// Error type for BRK client operations.
+#[derive(Debug)]
+pub struct BrkError {{
+    pub message: String,
+}}
+
+impl std::fmt::Display for BrkError {{
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {{
+        write!(f, "{{}}", self.message)
+    }}
+}}
+
+impl std::error::Error for BrkError {{}}
+
+/// Result type for BRK client operations.
+pub type Result<T> = std::result::Result<T, BrkError>;
+
+/// Options for configuring the BRK client.
+#[derive(Debug, Clone)]
+pub struct BrkClientOptions {{
+    pub base_url: String,
+    pub timeout_ms: u64,
+}}
+
+impl Default for BrkClientOptions {{
+    fn default() -> Self {{
+        Self {{
+            base_url: "http://localhost:3000".to_string(),
+            timeout_ms: 30000,
+        }}
+    }}
+}}
+
+/// Base HTTP client for making requests.
+#[derive(Debug, Clone)]
+pub struct BrkClientBase {{
+    base_url: String,
+    client: reqwest::blocking::Client,
+}}
+
+impl BrkClientBase {{
+    /// Create a new client with the given base URL.
+    pub fn new(base_url: impl Into<String>) -> Result<Self> {{
+        let base_url = base_url.into();
+        let client = reqwest::blocking::Client::new();
+        Ok(Self {{ base_url, client }})
+    }}
+
+    /// Create a new client with options.
+    pub fn with_options(options: BrkClientOptions) -> Result<Self> {{
+        let client = reqwest::blocking::Client::builder()
+            .timeout(std::time::Duration::from_millis(options.timeout_ms))
+            .build()
+            .map_err(|e| BrkError {{ message: e.to_string() }})?;
+        Ok(Self {{
+            base_url: options.base_url,
+            client,
+        }})
+    }}
+
+    /// Make a GET request.
+    pub fn get<T: DeserializeOwned>(&self, path: &str) -> Result<T> {{
+        let url = format!("{{}}{{}}", self.base_url, path);
+        self.client
+            .get(&url)
+            .send()
+            .map_err(|e| BrkError {{ message: e.to_string() }})?
+            .json()
+            .map_err(|e| BrkError {{ message: e.to_string() }})
+    }}
+}}
+
+"#
+    )
+    .unwrap();
+}
+
+fn generate_metric_node(output: &mut String) {
+    writeln!(
+        output,
+        r#"/// A metric node that can fetch data for different indexes.
+pub struct MetricNode<'a, T> {{
+    client: &'a BrkClientBase,
+    path: String,
+    _marker: PhantomData<T>,
+}}
+
+impl<'a, T: DeserializeOwned> MetricNode<'a, T> {{
+    pub fn new(client: &'a BrkClientBase, path: String) -> Self {{
+        Self {{
+            client,
+            path,
+            _marker: PhantomData,
+        }}
+    }}
+
+    /// Fetch all data points for this metric.
+    pub fn get(&self) -> Result<Vec<T>> {{
+        self.client.get(&self.path)
+    }}
+
+    /// Fetch data points within a date range.
+    pub fn get_range(&self, from: &str, to: &str) -> Result<Vec<T>> {{
+        let path = format!("{{}}?from={{}}&to={{}}", self.path, from, to);
+        self.client.get(&path)
+    }}
+}}
+
+"#
+    )
+    .unwrap();
+}
+
+/// Generate index accessor structs for each unique set of indexes
+fn generate_index_accessors(output: &mut String, patterns: &[IndexSetPattern]) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "// Index accessor structs\n").unwrap();
+
+    for pattern in patterns {
+        writeln!(
+            output,
+            "/// Index accessor for metrics with {} indexes.",
+            pattern.indexes.len()
+        )
+        .unwrap();
+        writeln!(output, "pub struct {}<'a, T> {{", pattern.name).unwrap();
+
+        for index in &pattern.indexes {
+            let field_name = index_to_field_name(index);
+            writeln!(output, "    pub {}: MetricNode<'a, T>,", field_name).unwrap();
+        }
+
+        writeln!(output, "    _marker: PhantomData<T>,").unwrap();
+        writeln!(output, "}}\n").unwrap();
+
+        // Generate impl block with constructor
+        writeln!(
+            output,
+            "impl<'a, T: DeserializeOwned> {}<'a, T> {{",
+            pattern.name
+        )
+        .unwrap();
+        writeln!(
+            output,
+            "    pub fn new(client: &'a BrkClientBase, base_path: &str) -> Self {{"
+        )
+        .unwrap();
+        writeln!(output, "        Self {{").unwrap();
+
+        for index in &pattern.indexes {
+            let field_name = index_to_field_name(index);
+            let path_segment = index.serialize_long();
+            writeln!(
+                output,
+                "            {}: MetricNode::new(client, format!(\"{{base_path}}/{}\")),",
+                field_name, path_segment
+            )
+            .unwrap();
+        }
+
+        writeln!(output, "            _marker: PhantomData,").unwrap();
+        writeln!(output, "        }}").unwrap();
+        writeln!(output, "    }}").unwrap();
+        writeln!(output, "}}\n").unwrap();
+    }
+}
+
+/// Convert an Index to a snake_case field name (e.g., DateIndex -> by_date_index)
+fn index_to_field_name(index: &Index) -> String {
+    format!("by_{}", to_snake_case(index.serialize_long()))
+}
+
+/// Generate pattern structs (those appearing 2+ times)
+fn generate_pattern_structs(
+    output: &mut String,
+    patterns: &[StructuralPattern],
+    metadata: &ClientMetadata,
+) {
+    if patterns.is_empty() {
+        return;
+    }
+
+    writeln!(output, "// Reusable pattern structs\n").unwrap();
+
+    for pattern in patterns {
+        let is_parameterizable = pattern.is_parameterizable();
+        let generic_params = if pattern.is_generic {
+            "<'a, T>"
+        } else {
+            "<'a>"
+        };
+
+        writeln!(output, "/// Pattern struct for repeated tree structure.").unwrap();
+        writeln!(output, "pub struct {}{} {{", pattern.name, generic_params).unwrap();
+
+        for field in &pattern.fields {
+            let field_name = to_snake_case(&field.name);
+            let type_annotation =
+                field_to_type_annotation_generic(field, metadata, pattern.is_generic);
+            writeln!(output, "    pub {}: {},", field_name, type_annotation).unwrap();
+        }
+
+        writeln!(output, "}}\n").unwrap();
+
+        // Generate impl block with constructor
+        writeln!(
+            output,
+            "impl{} {}{} {{",
+            generic_params, pattern.name, generic_params
+        )
+        .unwrap();
+
+        if is_parameterizable {
+            writeln!(
+                output,
+                "    /// Create a new pattern node with accumulated metric name."
+            )
+            .unwrap();
+            writeln!(
+                output,
+                "    pub fn new(client: &'a BrkClientBase, acc: &str) -> Self {{"
+            )
+            .unwrap();
+        } else {
+            writeln!(
+                output,
+                "    pub fn new(client: &'a BrkClientBase, base_path: &str) -> Self {{"
+            )
+            .unwrap();
+        }
+        writeln!(output, "        Self {{").unwrap();
+
+        for field in &pattern.fields {
+            if is_parameterizable {
+                generate_parameterized_rust_field(output, field, pattern, metadata);
+            } else {
+                generate_tree_path_rust_field(output, field, metadata);
+            }
+        }
+
+        writeln!(output, "        }}").unwrap();
+        writeln!(output, "    }}").unwrap();
+        writeln!(output, "}}\n").unwrap();
+    }
+}
+
+/// Generate a field using parameterized (prepend/append) metric name construction
+fn generate_parameterized_rust_field(
+    output: &mut String,
+    field: &PatternField,
+    pattern: &StructuralPattern,
+    metadata: &ClientMetadata,
+) {
+    let field_name = to_snake_case(&field.name);
+
+    // For branch fields, pass the accumulated name to nested pattern
+    if metadata.is_pattern_type(&field.rust_type) {
+        let child_acc = if let Some(pos) = pattern.get_field_position(&field.name) {
+            match pos {
+                FieldNamePosition::Append(suffix) => format!("&format!(\"{{acc}}{}\")", suffix),
+                FieldNamePosition::Prepend(prefix) => format!("&format!(\"{}{{acc}}\")", prefix),
+                FieldNamePosition::Identity => "acc".to_string(),
+                FieldNamePosition::SetBase(base) => format!("\"{}\"", base),
+            }
+        } else {
+            format!("&format!(\"{{acc}}_{}\")", field.name)
+        };
+
+        writeln!(
+            output,
+            "            {}: {}::new(client, {}),",
+            field_name, field.rust_type, child_acc
+        )
+        .unwrap();
+        return;
+    }
+
+    // For leaf fields, construct the metric path based on position
+    let metric_expr = if let Some(pos) = pattern.get_field_position(&field.name) {
+        match pos {
+            FieldNamePosition::Append(suffix) => format!("format!(\"/{{acc}}{}\")", suffix),
+            FieldNamePosition::Prepend(prefix) => format!("format!(\"/{}{{acc}}\")", prefix),
+            FieldNamePosition::Identity => "format!(\"/{acc}\")".to_string(),
+            FieldNamePosition::SetBase(base) => format!("\"/{}\".to_string()", base),
+        }
+    } else {
+        format!("format!(\"/{{acc}}_{}\")", field.name)
+    };
+
+    if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "            {}: {}::new(client, &{}),",
+            field_name, accessor.name, metric_expr
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "            {}: MetricNode::new(client, {}),",
+            field_name, metric_expr
+        )
+        .unwrap();
+    }
+}
+
+/// Generate a field using tree path construction (fallback for non-parameterizable patterns)
+fn generate_tree_path_rust_field(
+    output: &mut String,
+    field: &PatternField,
+    metadata: &ClientMetadata,
+) {
+    let field_name = to_snake_case(&field.name);
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        writeln!(
+            output,
+            "            {}: {}::new(client, &format!(\"{{base_path}}/{}\")),",
+            field_name, field.rust_type, field.name
+        )
+        .unwrap();
+    } else if metadata.field_uses_accessor(field) {
+        let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+        writeln!(
+            output,
+            "            {}: {}::new(client, &format!(\"{{base_path}}/{}\")),",
+            field_name, accessor.name, field.name
+        )
+        .unwrap();
+    } else {
+        writeln!(
+            output,
+            "            {}: MetricNode::new(client, format!(\"{{base_path}}/{}\")),",
+            field_name, field.name
+        )
+        .unwrap();
+    }
+}
+
+/// Convert a PatternField to the full type annotation, with optional generic support
+fn field_to_type_annotation_generic(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+) -> String {
+    field_to_type_annotation_with_generic(field, metadata, is_generic, None)
+}
+
+/// Convert a PatternField to the full type annotation.
+/// - `is_generic`: If true and field.rust_type is "T", use T in the output
+/// - `generic_value_type`: For branch fields that reference a generic pattern, this is the concrete type to substitute
+fn field_to_type_annotation_with_generic(
+    field: &PatternField,
+    metadata: &ClientMetadata,
+    is_generic: bool,
+    generic_value_type: Option<&str>,
+) -> String {
+    // For generic patterns, use T instead of concrete value type
+    // Also extract inner type from wrappers like Close<Dollars> -> Dollars
+    let value_type = if is_generic && field.rust_type == "T" {
+        "T".to_string()
+    } else {
+        extract_inner_type(&field.rust_type)
+    };
+
+    if metadata.is_pattern_type(&field.rust_type) {
+        // Check if this pattern is generic and we have a value type
+        if metadata.is_pattern_generic(&field.rust_type)
+            && let Some(vt) = generic_value_type
+        {
+            return format!("{}<'a, {}>", field.rust_type, vt);
+        }
+        format!("{}<'a>", field.rust_type)
+    } else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
+        // Leaf with a reusable accessor pattern
+        format!("{}<'a, {}>", accessor.name, value_type)
+    } else {
+        // Leaf with unique index set - use MetricNode directly
+        format!("MetricNode<'a, {}>", value_type)
+    }
+}
+
+/// Generate the catalog tree structure
+fn generate_tree(output: &mut String, catalog: &TreeNode, metadata: &ClientMetadata) {
+    writeln!(output, "// Catalog tree\n").unwrap();
+
+    let pattern_lookup = metadata.pattern_lookup();
+    let mut generated = HashSet::new();
+    generate_tree_node(
+        output,
+        "CatalogTree",
+        catalog,
+        &pattern_lookup,
+        metadata,
+        &mut generated,
+    );
+}
+
+/// Recursively generate tree nodes
+fn generate_tree_node(
+    output: &mut String,
+    name: &str,
+    node: &TreeNode,
+    pattern_lookup: &std::collections::HashMap<Vec<PatternField>, String>,
+    metadata: &ClientMetadata,
+    generated: &mut HashSet<String>,
+) {
+    let TreeNode::Branch(children) = node else {
+        return;
+    };
+
+    let fields_with_child_info = get_fields_with_child_info(children, name, pattern_lookup);
+    let fields: Vec<PatternField> = fields_with_child_info
+        .iter()
+        .map(|(f, _)| f.clone())
+        .collect();
+
+    // Skip if this matches a pattern (already generated separately)
+    if let Some(pattern_name) = pattern_lookup.get(&fields)
+        && pattern_name != name
+    {
+        return;
+    }
+
+    if generated.contains(name) {
+        return;
+    }
+    generated.insert(name.to_string());
+
+    writeln!(output, "/// Catalog tree node.").unwrap();
+    writeln!(output, "pub struct {}<'a> {{", name).unwrap();
+
+    for (field, child_fields) in &fields_with_child_info {
+        let field_name = to_snake_case(&field.name);
+        let generic_value_type = child_fields
+            .as_ref()
+            .and_then(|cf| metadata.get_generic_value_type(&field.rust_type, cf));
+        let type_annotation = field_to_type_annotation_with_generic(
+            field,
+            metadata,
+            false,
+            generic_value_type.as_deref(),
+        );
+        writeln!(output, "    pub {}: {},", field_name, type_annotation).unwrap();
+    }
+
+    writeln!(output, "}}\n").unwrap();
+
+    // Generate impl block
+    writeln!(output, "impl<'a> {}<'a> {{", name).unwrap();
+    writeln!(
+        output,
+        "    pub fn new(client: &'a BrkClientBase, base_path: &str) -> Self {{"
+    )
+    .unwrap();
+    writeln!(output, "        Self {{").unwrap();
+
+    for (field, (child_name, child_node)) in fields.iter().zip(children.iter()) {
+        let field_name = to_snake_case(&field.name);
+        if metadata.is_pattern_type(&field.rust_type) {
+            let pattern = metadata.find_pattern(&field.rust_type);
+            let is_parameterizable = pattern.is_some_and(|p| p.is_parameterizable());
+
+            if is_parameterizable {
+                let metric_base = get_pattern_instance_base(child_node, child_name);
+                writeln!(
+                    output,
+                    "            {}: {}::new(client, \"{}\"),",
+                    field_name, field.rust_type, metric_base
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "            {}: {}::new(client, &format!(\"{{base_path}}/{}\")),",
+                    field_name, field.rust_type, field.name
+                )
+                .unwrap();
+            }
+        } else if metadata.field_uses_accessor(field) {
+            let metric_path = if let TreeNode::Leaf(leaf) = child_node {
+                format!("/{}", leaf.name())
+            } else {
+                format!("{{base_path}}/{}", field.name)
+            };
+            let accessor = metadata.find_index_set_pattern(&field.indexes).unwrap();
+            if metric_path.contains("{base_path}") {
+                writeln!(
+                    output,
+                    "            {}: {}::new(client, &format!(\"{}\")),",
+                    field_name, accessor.name, metric_path
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "            {}: {}::new(client, \"{}\"),",
+                    field_name, accessor.name, metric_path
+                )
+                .unwrap();
+            }
+        } else {
+            let metric_path = if let TreeNode::Leaf(leaf) = child_node {
+                format!("/{}", leaf.name())
+            } else {
+                format!("{{base_path}}/{}", field.name)
+            };
+            if metric_path.contains("{base_path}") {
+                writeln!(
+                    output,
+                    "            {}: MetricNode::new(client, format!(\"{}\")),",
+                    field_name, metric_path
+                )
+                .unwrap();
+            } else {
+                writeln!(
+                    output,
+                    "            {}: MetricNode::new(client, \"{}\".to_string()),",
+                    field_name, metric_path
+                )
+                .unwrap();
+            }
+        }
+    }
+
+    writeln!(output, "        }}").unwrap();
+    writeln!(output, "    }}").unwrap();
+    writeln!(output, "}}\n").unwrap();
+
+    // Recursively generate child nodes that aren't patterns
+    for (child_name, child_node) in children {
+        if let TreeNode::Branch(grandchildren) = child_node {
+            let child_fields = get_node_fields(grandchildren, pattern_lookup);
+            if !pattern_lookup.contains_key(&child_fields) {
+                let child_struct_name = format!("{}_{}", name, to_pascal_case(child_name));
+                generate_tree_node(
+                    output,
+                    &child_struct_name,
+                    child_node,
+                    pattern_lookup,
+                    metadata,
+                    generated,
+                );
+            }
+        }
+    }
+}
+
+/// Generate the main client struct
+fn generate_main_client(output: &mut String, endpoints: &[Endpoint]) {
+    writeln!(
+        output,
+        r#"/// Main BRK client with catalog tree and API methods.
+pub struct BrkClient {{
+    base: BrkClientBase,
+}}
+
+impl BrkClient {{
+    /// Create a new client with the given base URL.
+    pub fn new(base_url: impl Into<String>) -> Result<Self> {{
+        Ok(Self {{
+            base: BrkClientBase::new(base_url)?,
+        }})
+    }}
+
+    /// Create a new client with options.
+    pub fn with_options(options: BrkClientOptions) -> Result<Self> {{
+        Ok(Self {{
+            base: BrkClientBase::with_options(options)?,
+        }})
+    }}
+
+    /// Get the catalog tree for navigating metrics.
+    pub fn tree(&self) -> CatalogTree<'_> {{
+        CatalogTree::new(&self.base, "")
+    }}
+"#
+    )
+    .unwrap();
+
+    // Generate API methods
+    generate_api_methods(output, endpoints);
+
+    writeln!(output, "}}").unwrap();
+}
+
+/// Generate API methods from OpenAPI endpoints
+fn generate_api_methods(output: &mut String, endpoints: &[Endpoint]) {
+    for endpoint in endpoints {
+        if !endpoint.should_generate() {
+            continue;
+        }
+
+        let method_name = endpoint_to_method_name(endpoint);
+        let return_type = endpoint
+            .response_type
+            .as_deref()
+            .map(js_type_to_rust)
+            .unwrap_or_else(|| "serde_json::Value".to_string());
+
+        // Build doc comment
+        writeln!(
+            output,
+            "    /// {}",
+            endpoint.summary.as_deref().unwrap_or(&method_name)
+        )
+        .unwrap();
+
+        // Build method signature
+        let params = build_method_params(endpoint);
+        writeln!(
+            output,
+            "    pub fn {}(&self{}) -> Result<{}> {{",
+            method_name, params, return_type
+        )
+        .unwrap();
+
+        // Build path
+        let path = build_path_template(&endpoint.path, &endpoint.path_params);
+
+        if endpoint.query_params.is_empty() {
+            writeln!(output, "        self.base.get(&format!(\"{}\"))", path).unwrap();
+        } else {
+            writeln!(output, "        let mut query = Vec::new();").unwrap();
+            for param in &endpoint.query_params {
+                if param.required {
+                    writeln!(
+                        output,
+                        "        query.push(format!(\"{}={{}}\", {}));",
+                        param.name, param.name
+                    )
+                    .unwrap();
+                } else {
+                    writeln!(
+                        output,
+                        "        if let Some(v) = {} {{ query.push(format!(\"{}={{}}\", v)); }}",
+                        param.name, param.name
+                    )
+                    .unwrap();
+                }
+            }
+            writeln!(output, "        let query_str = if query.is_empty() {{ String::new() }} else {{ format!(\"?{{}}\", query.join(\"&\")) }};").unwrap();
+            writeln!(
+                output,
+                "        self.base.get(&format!(\"{}{{}}\", query_str))",
+                path
+            )
+            .unwrap();
+        }
+
+        writeln!(output, "    }}\n").unwrap();
+    }
+}
+
+fn endpoint_to_method_name(endpoint: &Endpoint) -> String {
+    to_snake_case(&endpoint.operation_name())
+}
+
+fn build_method_params(endpoint: &Endpoint) -> String {
+    let mut params = Vec::new();
+    for param in &endpoint.path_params {
+        params.push(format!(", {}: &str", param.name));
+    }
+    for param in &endpoint.query_params {
+        if param.required {
+            params.push(format!(", {}: &str", param.name));
+        } else {
+            params.push(format!(", {}: Option<&str>", param.name));
+        }
+    }
+    params.join("")
+}
+
+fn build_path_template(path: &str, path_params: &[super::Parameter]) -> String {
+    let mut result = path.to_string();
+    for param in path_params {
+        let placeholder = format!("{{{}}}", param.name);
+        let interpolation = format!("{{{}}}", param.name);
+        result = result.replace(&placeholder, &interpolation);
+    }
+    result
+}
+
+/// Convert JS-style type to Rust type (e.g., "Txid[]" -> "Vec<Txid>")
+fn js_type_to_rust(js_type: &str) -> String {
+    if let Some(inner) = js_type.strip_suffix("[]") {
+        format!("Vec<{}>", js_type_to_rust(inner))
+    } else {
+        match js_type {
+            "string" => "String".to_string(),
+            "number" => "f64".to_string(),
+            "boolean" => "bool".to_string(),
+            "*" => "serde_json::Value".to_string(),
+            other => other.to_string(),
+        }
+    }
+}

crates/brk_binder/src/types/case.rs

@@ -0,0 +1,54 @@
+//! Case conversion utilities for identifiers.
+
+/// Convert a string to PascalCase (e.g., "fee_rate" -> "FeeRate").
+pub fn to_pascal_case(s: &str) -> String {
+    s.replace('-', "_")
+        .split('_')
+        .map(|word| {
+            let mut chars = word.chars();
+            match chars.next() {
+                None => String::new(),
+                Some(first) => first.to_uppercase().collect::<String>() + chars.as_str(),
+            }
+        })
+        .collect()
+}
+
+/// Convert a string to snake_case, handling Rust keywords.
+pub fn to_snake_case(s: &str) -> String {
+    let sanitized = s.replace('-', "_");
+
+    // Prefix with _ if starts with digit
+    let sanitized = if sanitized.chars().next().is_some_and(|c| c.is_ascii_digit()) {
+        format!("_{}", sanitized)
+    } else {
+        sanitized
+    };
+
+    // Handle Rust keywords
+    match sanitized.as_str() {
+        "type" | "const" | "static" | "match" | "if" | "else" | "loop" | "while" | "for"
+        | "break" | "continue" | "return" | "fn" | "let" | "mut" | "ref" | "self" | "super"
+        | "mod" | "use" | "pub" | "crate" | "extern" | "impl" | "trait" | "struct" | "enum"
+        | "where" | "async" | "await" | "dyn" | "move" => format!("r#{}", sanitized),
+        _ => sanitized,
+    }
+}
+
+/// Convert a string to camelCase (e.g., "fee_rate" -> "feeRate").
+pub fn to_camel_case(s: &str) -> String {
+    let pascal = to_pascal_case(s);
+    let mut chars = pascal.chars();
+
+    let result = match chars.next() {
+        None => String::new(),
+        Some(first) => first.to_lowercase().collect::<String>() + chars.as_str(),
+    };
+
+    // Prefix with _ if starts with digit
+    if result.chars().next().is_some_and(|c| c.is_ascii_digit()) {
+        format!("_{}", result)
+    } else {
+        result
+    }
+}

crates/brk_binder/src/types/mod.rs

@@ -0,0 +1,198 @@
+//! Types and utilities for client generation.
+
+mod case;
+mod patterns;
+mod schema;
+mod tree;
+
+pub use case::*;
+pub use schema::*;
+pub use tree::*;
+
+use std::collections::{BTreeSet, HashMap};
+
+use brk_query::Vecs;
+use brk_types::Index;
+
+/// How a field modifies the accumulated metric name.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum FieldNamePosition {
+    /// Field prepends a prefix: leaf.name() = prefix + accumulated
+    Prepend(String),
+    /// Field appends a suffix: leaf.name() = accumulated + suffix
+    Append(String),
+    /// Field IS the accumulated name (no modification)
+    Identity,
+    /// Field sets a new base name (used at pattern entry points)
+    SetBase(String),
+}
+
+/// Metadata extracted from brk_query for client generation.
+#[derive(Debug)]
+pub struct ClientMetadata {
+    /// The catalog tree structure (with schemas in leaves)
+    pub catalog: brk_types::TreeNode,
+    /// Structural patterns - tree node shapes that repeat
+    pub structural_patterns: Vec<StructuralPattern>,
+    /// All indexes used across the catalog
+    pub used_indexes: BTreeSet<Index>,
+    /// Index set patterns - sets of indexes that appear together on metrics
+    pub index_set_patterns: Vec<IndexSetPattern>,
+    /// Maps concrete field signatures to pattern names
+    pub concrete_to_pattern: HashMap<Vec<PatternField>, String>,
+}
+
+impl ClientMetadata {
+    /// Extract metadata from brk_query::Vecs.
+    pub fn from_vecs(vecs: &Vecs) -> Self {
+        let catalog = vecs.catalog().clone();
+        let (structural_patterns, concrete_to_pattern) =
+            patterns::detect_structural_patterns(&catalog);
+        let (used_indexes, index_set_patterns) = tree::detect_index_patterns(&catalog);
+
+        ClientMetadata {
+            catalog,
+            structural_patterns,
+            used_indexes,
+            index_set_patterns,
+            concrete_to_pattern,
+        }
+    }
+
+    /// Find an index set pattern that matches the given indexes.
+    pub fn find_index_set_pattern(&self, indexes: &BTreeSet<Index>) -> Option<&IndexSetPattern> {
+        self.index_set_patterns
+            .iter()
+            .find(|p| &p.indexes == indexes)
+    }
+
+    /// Check if a type is a structural pattern name.
+    pub fn is_pattern_type(&self, type_name: &str) -> bool {
+        self.structural_patterns.iter().any(|p| p.name == type_name)
+    }
+
+    /// Find a pattern by name.
+    pub fn find_pattern(&self, name: &str) -> Option<&StructuralPattern> {
+        self.structural_patterns.iter().find(|p| p.name == name)
+    }
+
+    /// Check if a pattern is generic.
+    pub fn is_pattern_generic(&self, name: &str) -> bool {
+        self.find_pattern(name).is_some_and(|p| p.is_generic)
+    }
+
+    /// Extract the value type from concrete fields for a generic pattern.
+    pub fn get_generic_value_type(
+        &self,
+        pattern_name: &str,
+        fields: &[PatternField],
+    ) -> Option<String> {
+        if !self.is_pattern_generic(pattern_name) {
+            return None;
+        }
+        fields
+            .iter()
+            .find(|f| f.is_leaf())
+            .map(|f| extract_inner_type(&f.rust_type))
+    }
+
+    /// Build a lookup map from field signatures to pattern names.
+    pub fn pattern_lookup(&self) -> HashMap<Vec<PatternField>, String> {
+        let mut lookup = self.concrete_to_pattern.clone();
+        for p in &self.structural_patterns {
+            lookup.insert(p.fields.clone(), p.name.clone());
+        }
+        lookup
+    }
+
+    /// Check if a field should use a shared index accessor.
+    pub fn field_uses_accessor(&self, field: &PatternField) -> bool {
+        self.find_index_set_pattern(&field.indexes).is_some()
+    }
+}
+
+/// A pattern of indexes that appear together on multiple metrics.
+#[derive(Debug, Clone)]
+pub struct IndexSetPattern {
+    /// Pattern name (e.g., "DateHeightIndexes")
+    pub name: String,
+    /// The set of indexes
+    pub indexes: BTreeSet<Index>,
+}
+
+/// A structural pattern - a branch structure that appears multiple times.
+#[derive(Debug, Clone)]
+pub struct StructuralPattern {
+    /// Pattern name
+    pub name: String,
+    /// Ordered list of child fields
+    pub fields: Vec<PatternField>,
+    /// How each field modifies the accumulated name
+    pub field_positions: HashMap<String, FieldNamePosition>,
+    /// If true, all leaf fields use a type parameter T
+    pub is_generic: bool,
+}
+
+impl StructuralPattern {
+    /// Returns true if this pattern contains any leaf fields.
+    pub fn contains_leaves(&self) -> bool {
+        self.fields.iter().any(|f| f.is_leaf())
+    }
+
+    /// Returns true if all leaf fields have consistent name transformations.
+    pub fn is_parameterizable(&self) -> bool {
+        !self.field_positions.is_empty()
+            && self
+                .fields
+                .iter()
+                .all(|f| f.is_branch() || self.field_positions.contains_key(&f.name))
+    }
+
+    /// Get the field position for a given field name.
+    pub fn get_field_position(&self, field_name: &str) -> Option<&FieldNamePosition> {
+        self.field_positions.get(field_name)
+    }
+}
+
+/// A field in a structural pattern.
+#[derive(Debug, Clone, PartialOrd, Ord)]
+pub struct PatternField {
+    /// Field name
+    pub name: String,
+    /// Rust type for leaves or pattern name for branches
+    pub rust_type: String,
+    /// JSON type from schema
+    pub json_type: String,
+    /// For leaves: the set of supported indexes. Empty for branches.
+    pub indexes: BTreeSet<Index>,
+}
+
+impl PatternField {
+    /// Returns true if this is a leaf field (has indexes).
+    pub fn is_leaf(&self) -> bool {
+        !self.indexes.is_empty()
+    }
+
+    /// Returns true if this is a branch field (no indexes).
+    pub fn is_branch(&self) -> bool {
+        self.indexes.is_empty()
+    }
+}
+
+impl std::hash::Hash for PatternField {
+    fn hash<H: std::hash::Hasher>(&self, state: &mut H) {
+        self.name.hash(state);
+        self.rust_type.hash(state);
+        self.json_type.hash(state);
+    }
+}
+
+impl PartialEq for PatternField {
+    fn eq(&self, other: &Self) -> bool {
+        self.name == other.name
+            && self.rust_type == other.rust_type
+            && self.json_type == other.json_type
+    }
+}
+
+impl Eq for PatternField {}

crates/brk_binder/src/types/patterns.rs

@@ -0,0 +1,427 @@
+//! Pattern detection for structural patterns in the metric tree.
+
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use brk_types::TreeNode;
+
+use super::{
+    case::to_pascal_case, schema::schema_to_json_type, FieldNamePosition, PatternField,
+    StructuralPattern,
+};
+
+/// Detect structural patterns in the tree using a bottom-up approach.
+/// Returns (patterns, concrete_to_pattern_mapping).
+pub fn detect_structural_patterns(
+    tree: &TreeNode,
+) -> (Vec<StructuralPattern>, HashMap<Vec<PatternField>, String>) {
+    let mut signature_to_pattern: HashMap<Vec<PatternField>, String> = HashMap::new();
+    let mut signature_counts: HashMap<Vec<PatternField>, usize> = HashMap::new();
+    let mut normalized_to_name: HashMap<Vec<PatternField>, String> = HashMap::new();
+    let mut name_counts: HashMap<String, usize> = HashMap::new();
+
+    // Process tree bottom-up to resolve all branch types
+    resolve_branch_patterns(
+        tree,
+        "root",
+        &mut signature_to_pattern,
+        &mut signature_counts,
+        &mut normalized_to_name,
+        &mut name_counts,
+    );
+
+    // Identify generic patterns
+    let (generic_patterns, generic_mappings) = detect_generic_patterns(&signature_to_pattern);
+
+    // Build non-generic patterns: signatures appearing 2+ times that weren't merged into generics
+    let mut patterns: Vec<StructuralPattern> = signature_to_pattern
+        .iter()
+        .filter(|(sig, _)| {
+            signature_counts.get(*sig).copied().unwrap_or(0) >= 2
+                && !generic_mappings.contains_key(*sig)
+        })
+        .map(|(fields, name)| StructuralPattern {
+            name: name.clone(),
+            fields: fields.clone(),
+            field_positions: HashMap::new(),
+            is_generic: false,
+        })
+        .collect();
+
+    patterns.extend(generic_patterns);
+
+    // Build lookup for field position analysis
+    let mut pattern_lookup: HashMap<Vec<PatternField>, String> = HashMap::new();
+    for (sig, name) in &signature_to_pattern {
+        if signature_counts.get(sig).copied().unwrap_or(0) >= 2 {
+            pattern_lookup.insert(sig.clone(), name.clone());
+        }
+    }
+    pattern_lookup.extend(generic_mappings.clone());
+
+    let concrete_to_pattern = pattern_lookup.clone();
+
+    // Second pass: analyze field positions
+    analyze_pattern_field_positions(tree, &mut patterns, &pattern_lookup);
+
+    patterns.sort_by(|a, b| b.fields.len().cmp(&a.fields.len()));
+    (patterns, concrete_to_pattern)
+}
+
+/// Detect generic patterns by grouping signatures by their normalized form.
+fn detect_generic_patterns(
+    signature_to_pattern: &HashMap<Vec<PatternField>, String>,
+) -> (Vec<StructuralPattern>, HashMap<Vec<PatternField>, String>) {
+    let mut normalized_groups: HashMap<Vec<PatternField>, Vec<(Vec<PatternField>, String)>> =
+        HashMap::new();
+
+    for (fields, name) in signature_to_pattern {
+        if let Some(normalized) = normalize_fields_for_generic(fields) {
+            normalized_groups
+                .entry(normalized)
+                .or_default()
+                .push((fields.clone(), name.clone()));
+        }
+    }
+
+    let mut patterns = Vec::new();
+    let mut mappings: HashMap<Vec<PatternField>, String> = HashMap::new();
+
+    for (normalized_fields, group) in normalized_groups {
+        if group.len() >= 2 {
+            let generic_name = group[0].1.clone();
+            for (concrete_fields, _) in &group {
+                mappings.insert(concrete_fields.clone(), generic_name.clone());
+            }
+            patterns.push(StructuralPattern {
+                name: generic_name,
+                fields: normalized_fields,
+                field_positions: HashMap::new(),
+                is_generic: true,
+            });
+        }
+    }
+
+    (patterns, mappings)
+}
+
+/// Normalize fields by replacing concrete value types with "T".
+fn normalize_fields_for_generic(fields: &[PatternField]) -> Option<Vec<PatternField>> {
+    let leaf_types: Vec<&str> = fields
+        .iter()
+        .filter(|f| f.is_leaf())
+        .map(|f| f.rust_type.as_str())
+        .collect();
+
+    if leaf_types.is_empty() {
+        return None;
+    }
+
+    let first_type = leaf_types[0];
+    if !leaf_types.iter().all(|t| *t == first_type) {
+        return None;
+    }
+
+    let normalized = fields
+        .iter()
+        .map(|f| {
+            if f.is_branch() {
+                f.clone()
+            } else {
+                PatternField {
+                    name: f.name.clone(),
+                    rust_type: "T".to_string(),
+                    json_type: "T".to_string(),
+                    indexes: f.indexes.clone(),
+                }
+            }
+        })
+        .collect();
+
+    Some(normalized)
+}
+
+/// Recursively resolve branch patterns bottom-up.
+fn resolve_branch_patterns(
+    node: &TreeNode,
+    field_name: &str,
+    signature_to_pattern: &mut HashMap<Vec<PatternField>, String>,
+    signature_counts: &mut HashMap<Vec<PatternField>, usize>,
+    normalized_to_name: &mut HashMap<Vec<PatternField>, String>,
+    name_counts: &mut HashMap<String, usize>,
+) -> Option<String> {
+    let TreeNode::Branch(children) = node else {
+        return None;
+    };
+
+    let mut fields: Vec<PatternField> = Vec::new();
+    for (child_name, child_node) in children {
+        let (rust_type, json_type, indexes) = match child_node {
+            TreeNode::Leaf(leaf) => (
+                leaf.value_type().to_string(),
+                schema_to_json_type(&leaf.schema),
+                leaf.indexes().clone(),
+            ),
+            TreeNode::Branch(_) => {
+                let pattern_name = resolve_branch_patterns(
+                    child_node,
+                    child_name,
+                    signature_to_pattern,
+                    signature_counts,
+                    normalized_to_name,
+                    name_counts,
+                )
+                .unwrap_or_else(|| "Unknown".to_string());
+                (pattern_name.clone(), pattern_name, BTreeSet::new())
+            }
+        };
+        fields.push(PatternField {
+            name: child_name.clone(),
+            rust_type,
+            json_type,
+            indexes,
+        });
+    }
+
+    fields.sort_by(|a, b| a.name.cmp(&b.name));
+    *signature_counts.entry(fields.clone()).or_insert(0) += 1;
+
+    let pattern_name = if let Some(existing) = signature_to_pattern.get(&fields) {
+        existing.clone()
+    } else {
+        let normalized = normalize_fields_for_naming(&fields);
+        let name = normalized_to_name
+            .entry(normalized)
+            .or_insert_with(|| generate_pattern_name(field_name, name_counts))
+            .clone();
+        signature_to_pattern.insert(fields, name.clone());
+        name
+    };
+
+    Some(pattern_name)
+}
+
+/// Normalize fields for naming (same structure = same name).
+fn normalize_fields_for_naming(fields: &[PatternField]) -> Vec<PatternField> {
+    fields
+        .iter()
+        .map(|f| {
+            if f.is_branch() {
+                f.clone()
+            } else {
+                PatternField {
+                    name: f.name.clone(),
+                    rust_type: "_".to_string(),
+                    json_type: "_".to_string(),
+                    indexes: f.indexes.clone(),
+                }
+            }
+        })
+        .collect()
+}
+
+/// Generate a unique pattern name.
+fn generate_pattern_name(field_name: &str, name_counts: &mut HashMap<String, usize>) -> String {
+    let pascal = to_pascal_case(field_name);
+    let sanitized = if pascal.chars().next().is_some_and(|c| c.is_ascii_digit()) {
+        format!("_{}", pascal)
+    } else {
+        pascal
+    };
+
+    let base_name = format!("{}Pattern", sanitized);
+    let count = name_counts.entry(base_name.clone()).or_insert(0);
+    *count += 1;
+
+    if *count == 1 {
+        base_name
+    } else {
+        format!("{}{}", base_name, count)
+    }
+}
+
+// Field position analysis
+
+fn analyze_pattern_field_positions(
+    tree: &TreeNode,
+    patterns: &mut [StructuralPattern],
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) {
+    let mut instances: HashMap<String, Vec<(String, String, String)>> = HashMap::new();
+    collect_pattern_instances(tree, "", &mut instances, pattern_lookup);
+
+    for pattern in patterns.iter_mut() {
+        if let Some(pattern_instances) = instances.get(&pattern.name) {
+            pattern.field_positions = analyze_field_positions_from_instances(pattern_instances);
+        }
+    }
+}
+
+fn collect_pattern_instances(
+    node: &TreeNode,
+    accumulated_name: &str,
+    instances: &mut HashMap<String, Vec<(String, String, String)>>,
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) {
+    let TreeNode::Branch(children) = node else {
+        return;
+    };
+
+    let fields = get_node_fields_for_analysis(children, pattern_lookup);
+    if let Some(pattern_name) = pattern_lookup.get(&fields) {
+        for (field_name, child_node) in children {
+            if let TreeNode::Leaf(leaf) = child_node {
+                instances.entry(pattern_name.clone()).or_default().push((
+                    accumulated_name.to_string(),
+                    field_name.clone(),
+                    leaf.name().to_string(),
+                ));
+            }
+        }
+    }
+
+    for (field_name, child_node) in children {
+        let child_accumulated = match child_node {
+            TreeNode::Leaf(leaf) => leaf.name().to_string(),
+            TreeNode::Branch(_) => {
+                if let Some(desc_leaf_name) = get_descendant_leaf_name(child_node) {
+                    infer_accumulated_name(accumulated_name, field_name, &desc_leaf_name)
+                } else if accumulated_name.is_empty() {
+                    field_name.clone()
+                } else {
+                    format!("{}_{}", accumulated_name, field_name)
+                }
+            }
+        };
+        collect_pattern_instances(child_node, &child_accumulated, instances, pattern_lookup);
+    }
+}
+
+fn get_descendant_leaf_name(node: &TreeNode) -> Option<String> {
+    match node {
+        TreeNode::Leaf(leaf) => Some(leaf.name().to_string()),
+        TreeNode::Branch(children) => children.values().find_map(get_descendant_leaf_name),
+    }
+}
+
+fn infer_accumulated_name(parent_acc: &str, field_name: &str, descendant_leaf: &str) -> String {
+    if let Some(pos) = descendant_leaf.find(field_name) {
+        if pos == 0 {
+            return field_name.to_string();
+        }
+        if pos > 0 && descendant_leaf.chars().nth(pos - 1) == Some('_') {
+            return if parent_acc.is_empty() {
+                field_name.to_string()
+            } else {
+                format!("{}_{}", parent_acc, field_name)
+            };
+        }
+    }
+
+    if parent_acc.is_empty() {
+        field_name.to_string()
+    } else {
+        format!("{}_{}", parent_acc, field_name)
+    }
+}
+
+fn get_node_fields_for_analysis(
+    children: &BTreeMap<String, TreeNode>,
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) -> Vec<PatternField> {
+    let mut fields: Vec<PatternField> = children
+        .iter()
+        .map(|(name, node)| {
+            let (rust_type, json_type, indexes) = match node {
+                TreeNode::Leaf(leaf) => (
+                    leaf.value_type().to_string(),
+                    schema_to_json_type(&leaf.schema),
+                    leaf.indexes().clone(),
+                ),
+                TreeNode::Branch(grandchildren) => {
+                    let child_fields = get_node_fields_for_analysis(grandchildren, pattern_lookup);
+                    let pattern_name = pattern_lookup
+                        .get(&child_fields)
+                        .cloned()
+                        .unwrap_or_else(|| "Unknown".to_string());
+                    (pattern_name.clone(), pattern_name, BTreeSet::new())
+                }
+            };
+            PatternField {
+                name: name.clone(),
+                rust_type,
+                json_type,
+                indexes,
+            }
+        })
+        .collect();
+    fields.sort_by(|a, b| a.name.cmp(&b.name));
+    fields
+}
+
+fn analyze_field_positions_from_instances(
+    instances: &[(String, String, String)],
+) -> HashMap<String, FieldNamePosition> {
+    let mut field_instances: HashMap<String, Vec<(String, String)>> = HashMap::new();
+    for (acc, field, leaf) in instances {
+        field_instances
+            .entry(field.clone())
+            .or_default()
+            .push((acc.clone(), leaf.clone()));
+    }
+
+    let mut positions = HashMap::new();
+    for (field_name, field_data) in field_instances {
+        if let Some(position) = detect_field_position(&field_data) {
+            positions.insert(field_name, position);
+        }
+    }
+    positions
+}
+
+fn detect_field_position(data: &[(String, String)]) -> Option<FieldNamePosition> {
+    if data.is_empty() {
+        return None;
+    }
+
+    let (first_acc, first_leaf) = &data[0];
+
+    // Identity
+    if first_acc == first_leaf {
+        return Some(FieldNamePosition::Identity);
+    }
+
+    // Append
+    if let Some(suffix) = first_leaf.strip_prefix(first_acc.as_str()) {
+        let suffix = suffix.to_string();
+        if data.iter().all(|(acc, leaf)| {
+            if acc.is_empty() {
+                leaf == suffix.trim_start_matches('_')
+            } else {
+                leaf.strip_prefix(acc.as_str()) == Some(&suffix)
+            }
+        }) {
+            return Some(FieldNamePosition::Append(suffix));
+        }
+    }
+
+    // Prepend
+    if let Some(prefix) = first_leaf.strip_suffix(first_acc.as_str()) {
+        let prefix = prefix.to_string();
+        if data.iter().all(|(acc, leaf)| {
+            if acc.is_empty() {
+                leaf == prefix.trim_end_matches('_')
+            } else {
+                leaf.strip_suffix(acc.as_str()) == Some(&prefix)
+            }
+        }) {
+            return Some(FieldNamePosition::Prepend(prefix));
+        }
+    }
+
+    // SetBase
+    if first_acc.is_empty() {
+        return Some(FieldNamePosition::SetBase(first_leaf.clone()));
+    }
+
+    None
+}

crates/brk_binder/src/types/schema.rs

@@ -0,0 +1,47 @@
+//! JSON Schema utilities.
+
+use serde_json::Value;
+
+/// Unwrap allOf with a single element, returning the inner schema.
+/// Schemars uses allOf for composition, but often with just one $ref.
+pub fn unwrap_allof(schema: &Value) -> &Value {
+    if let Some(all_of) = schema.get("allOf").and_then(|v| v.as_array())
+        && all_of.len() == 1
+    {
+        return &all_of[0];
+    }
+    schema
+}
+
+/// Check if a schema represents an enum type.
+/// Enums have either an "enum" array or "oneOf" without properties.
+pub fn is_enum_schema(schema: &Value) -> bool {
+    schema.get("enum").is_some()
+        || (schema.get("oneOf").is_some() && schema.get("properties").is_none())
+}
+
+/// Extract inner type from a wrapper generic like `Close<Dollars>` -> `Dollars`.
+/// Also handles malformed types like `Dollars>` (from vecdb's short_type_name).
+pub fn extract_inner_type(type_str: &str) -> String {
+    // Handle proper generic wrappers like `Close<Dollars>` -> `Dollars`
+    if let Some(start) = type_str.find('<')
+        && let Some(end) = type_str.rfind('>')
+        && start < end
+    {
+        return type_str[start + 1..end].to_string();
+    }
+    // Handle malformed types like `Dollars>` (trailing > without <)
+    if type_str.ends_with('>') && !type_str.contains('<') {
+        return type_str.trim_end_matches('>').to_string();
+    }
+    type_str.to_string()
+}
+
+/// Extract JSON type from a schema ("integer", "number", "string", etc).
+pub fn schema_to_json_type(schema: &Value) -> String {
+    schema
+        .get("type")
+        .and_then(|v| v.as_str())
+        .unwrap_or("object")
+        .to_string()
+}

crates/brk_binder/src/types/tree.rs

@@ -0,0 +1,194 @@
+//! Tree traversal utilities.
+
+use std::collections::{BTreeMap, BTreeSet, HashMap};
+
+use brk_types::{Index, TreeNode};
+
+use super::{PatternField, case::to_pascal_case, schema::schema_to_json_type};
+
+/// Get the first leaf name from a tree node.
+pub fn get_first_leaf_name(node: &TreeNode) -> Option<String> {
+    match node {
+        TreeNode::Leaf(leaf) => Some(leaf.name().to_string()),
+        TreeNode::Branch(children) => children.values().find_map(get_first_leaf_name),
+    }
+}
+
+/// Get the metric base for a pattern instance by analyzing the first leaf descendant.
+pub fn get_pattern_instance_base(node: &TreeNode, field_name: &str) -> String {
+    if let Some(leaf_name) = get_first_leaf_name(node)
+        && leaf_name.contains(field_name)
+    {
+        return field_name.to_string();
+    }
+    field_name.to_string()
+}
+
+/// Get the field signature for a branch node's children.
+pub fn get_node_fields(
+    children: &BTreeMap<String, TreeNode>,
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) -> Vec<PatternField> {
+    let mut fields: Vec<PatternField> = children
+        .iter()
+        .map(|(name, node)| {
+            let (rust_type, json_type, indexes) = match node {
+                TreeNode::Leaf(leaf) => (
+                    leaf.value_type().to_string(),
+                    schema_to_json_type(&leaf.schema),
+                    leaf.indexes().clone(),
+                ),
+                TreeNode::Branch(grandchildren) => {
+                    let child_fields = get_node_fields(grandchildren, pattern_lookup);
+                    let pattern_name = pattern_lookup
+                        .get(&child_fields)
+                        .cloned()
+                        .unwrap_or_else(|| "Unknown".to_string());
+                    (pattern_name.clone(), pattern_name, BTreeSet::new())
+                }
+            };
+            PatternField {
+                name: name.clone(),
+                rust_type,
+                json_type,
+                indexes,
+            }
+        })
+        .collect();
+    fields.sort_by(|a, b| a.name.cmp(&b.name));
+    fields
+}
+
+/// Like get_node_fields but takes a parent name for generating child pattern names.
+pub fn get_node_fields_with_parent(
+    children: &BTreeMap<String, TreeNode>,
+    parent_name: &str,
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) -> Vec<PatternField> {
+    let mut fields: Vec<PatternField> = children
+        .iter()
+        .map(|(name, node)| {
+            let (rust_type, json_type, indexes) = match node {
+                TreeNode::Leaf(leaf) => (
+                    leaf.value_type().to_string(),
+                    schema_to_json_type(&leaf.schema),
+                    leaf.indexes().clone(),
+                ),
+                TreeNode::Branch(grandchildren) => {
+                    let child_fields = get_node_fields(grandchildren, pattern_lookup);
+                    let pattern_name = pattern_lookup
+                        .get(&child_fields)
+                        .cloned()
+                        .unwrap_or_else(|| format!("{}_{}", parent_name, to_pascal_case(name)));
+                    (pattern_name.clone(), pattern_name, BTreeSet::new())
+                }
+            };
+            PatternField {
+                name: name.clone(),
+                rust_type,
+                json_type,
+                indexes,
+            }
+        })
+        .collect();
+    fields.sort_by(|a, b| a.name.cmp(&b.name));
+    fields
+}
+
+/// Get fields with child field information for generic pattern lookup.
+/// Returns (field, child_fields) pairs where child_fields is Some for branches.
+pub fn get_fields_with_child_info(
+    children: &BTreeMap<String, TreeNode>,
+    parent_name: &str,
+    pattern_lookup: &HashMap<Vec<PatternField>, String>,
+) -> Vec<(PatternField, Option<Vec<PatternField>>)> {
+    children
+        .iter()
+        .map(|(name, node)| {
+            let (rust_type, json_type, indexes, child_fields) = match node {
+                TreeNode::Leaf(leaf) => (
+                    leaf.value_type().to_string(),
+                    schema_to_json_type(&leaf.schema),
+                    leaf.indexes().clone(),
+                    None,
+                ),
+                TreeNode::Branch(grandchildren) => {
+                    let child_fields = get_node_fields(grandchildren, pattern_lookup);
+                    let pattern_name = pattern_lookup
+                        .get(&child_fields)
+                        .cloned()
+                        .unwrap_or_else(|| format!("{}_{}", parent_name, to_pascal_case(name)));
+                    (
+                        pattern_name.clone(),
+                        pattern_name,
+                        BTreeSet::new(),
+                        Some(child_fields),
+                    )
+                }
+            };
+            (
+                PatternField {
+                    name: name.clone(),
+                    rust_type,
+                    json_type,
+                    indexes,
+                },
+                child_fields,
+            )
+        })
+        .collect()
+}
+
+/// Detect index patterns (sets of indexes that appear together on multiple metrics).
+pub fn detect_index_patterns(tree: &TreeNode) -> (BTreeSet<Index>, Vec<super::IndexSetPattern>) {
+    let mut used_indexes: BTreeSet<Index> = BTreeSet::new();
+    let mut index_sets: Vec<BTreeSet<Index>> = Vec::new();
+
+    collect_indexes_from_tree(tree, &mut used_indexes, &mut index_sets);
+
+    // Count occurrences of each unique index set
+    let mut index_set_counts: Vec<(BTreeSet<Index>, usize)> = Vec::new();
+    for index_set in index_sets {
+        if let Some(entry) = index_set_counts.iter_mut().find(|(s, _)| s == &index_set) {
+            entry.1 += 1;
+        } else {
+            index_set_counts.push((index_set, 1));
+        }
+    }
+
+    // Build patterns for index sets appearing 2+ times
+    let mut patterns: Vec<super::IndexSetPattern> = index_set_counts
+        .into_iter()
+        .filter(|(indexes, count)| *count >= 2 && !indexes.is_empty())
+        .enumerate()
+        .map(|(i, (indexes, _))| super::IndexSetPattern {
+            name: if i == 0 {
+                "Indexes".to_string()
+            } else {
+                format!("Indexes{}", i + 1)
+            },
+            indexes,
+        })
+        .collect();
+
+    patterns.sort_by(|a, b| b.indexes.len().cmp(&a.indexes.len()));
+    (used_indexes, patterns)
+}
+
+fn collect_indexes_from_tree(
+    node: &TreeNode,
+    used_indexes: &mut BTreeSet<Index>,
+    index_sets: &mut Vec<BTreeSet<Index>>,
+) {
+    match node {
+        TreeNode::Leaf(leaf) => {
+            used_indexes.extend(leaf.indexes().iter().cloned());
+            index_sets.push(leaf.indexes().clone());
+        }
+        TreeNode::Branch(children) => {
+            for child in children.values() {
+                collect_indexes_from_tree(child, used_indexes, index_sets);
+            }
+        }
+    }
+}

crates/brk_bundler/Cargo.toml

@@ -0,0 +1,17 @@
+[package]
+name = "brk_bundler"
+description = "A thin wrapper around rolldown"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+homepage.workspace = true
+repository.workspace = true
+build = "build.rs"
+
+[dependencies]
+log = { workspace = true }
+notify = "8.2.0"
+# rolldown = { path = "../../../rolldown/crates/rolldown", package = "brk_rolldown" }
+rolldown = { version = "0.5.1", package = "brk_rolldown" }
+sugar_path = "1.2.1"
+tokio = { workspace = true }

crates/brk_bundler/README.md

@@ -0,0 +1,32 @@
+# brk_bundler
+
+JavaScript bundling with watch mode for BRK web interfaces.
+
+## What It Enables
+
+Bundle and minify JavaScript modules using Rolldown, with file watching for development. Handles module copying, source map generation, and cache-busting via hashed filenames.
+
+## Key Features
+
+- **Rolldown integration**: Fast Rust-based bundler with tree-shaking and minification
+- **Watch mode**: Rebuilds on file changes with live module syncing
+- **Source maps**: Full debugging support in production builds
+- **Cache busting**: Hashes main bundle filename, updates HTML references automatically
+- **Service worker versioning**: Injects package version into service worker files
+
+## Core API
+
+```rust,ignore
+// One-shot build
+let dist = bundle(modules_path, websites_path, "src", false).await?;
+
+// Watch mode for development
+bundle(modules_path, websites_path, "src", true).await?;
+```
+
+## Build Pipeline
+
+1. Copy shared modules to source scripts directory
+2. Bundle with Rolldown (minified, with source maps)
+3. Update `index.html` with hashed script references
+4. Inject version into service worker

crates/brk_bundler/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_bundler/src/lib.rs

@@ -0,0 +1,193 @@
+#![doc = include_str!("../README.md")]
+
+use std::{
+    fs, io,
+    path::{Path, PathBuf},
+    sync::Arc,
+};
+
+use log::error;
+use notify::{EventKind, RecursiveMode, Watcher};
+use rolldown::{
+    Bundler, BundlerOptions, InlineConstConfig, InlineConstMode, InlineConstOption,
+    OptimizationOption, RawMinifyOptions, SourceMapType,
+};
+use sugar_path::SugarPath;
+use tokio::sync::Mutex;
+
+const VERSION: &str = env!("CARGO_PKG_VERSION");
+
+pub async fn bundle(
+    modules_path: &Path,
+    websites_path: &Path,
+    source_folder: &str,
+    watch: bool,
+) -> io::Result<PathBuf> {
+    let relative_modules_path = modules_path;
+    let relative_source_path = websites_path.join(source_folder);
+    let relative_dist_path = websites_path.join("dist");
+
+    let absolute_modules_path = relative_modules_path.absolutize();
+    let absolute_modules_path_clone = absolute_modules_path.clone();
+    let absolute_websites_path = websites_path.absolutize();
+    let absolute_websites_path_clone = absolute_websites_path.clone();
+
+    let absolute_source_path = relative_source_path.absolutize();
+    let absolute_source_index_path = absolute_source_path.join("index.html");
+    let absolute_source_index_path_clone = absolute_source_index_path.clone();
+    let absolute_source_scripts_path = absolute_source_path.join("scripts");
+    let absolute_source_scripts_modules_path = absolute_source_scripts_path.join("modules");
+    let absolute_source_sw_path = absolute_source_path.join("service-worker.js");
+    let absolute_source_sw_path_clone = absolute_source_sw_path.clone();
+
+    let absolute_dist_path = relative_dist_path.absolutize();
+    let absolute_dist_scripts_path = absolute_dist_path.join("scripts");
+    let absolute_dist_scripts_entry_path = absolute_dist_scripts_path.join("entry.js");
+    let absolute_dist_scripts_entry_path_clone = absolute_dist_scripts_entry_path.clone();
+    let absolute_dist_index_path = absolute_dist_path.join("index.html");
+    let absolute_dist_sw_path = absolute_dist_path.join("service-worker.js");
+
+    let _ = fs::remove_dir_all(&absolute_dist_path);
+    let _ = fs::remove_dir_all(&absolute_source_scripts_modules_path);
+    copy_dir_all(
+        &absolute_modules_path,
+        &absolute_source_scripts_modules_path,
+    )?;
+    copy_dir_all(&absolute_source_path, &absolute_dist_path)?;
+    fs::remove_dir_all(&absolute_dist_scripts_path)?;
+    fs::create_dir(&absolute_dist_scripts_path)?;
+
+    // dbg!(BundlerOptions::default());
+
+    let mut bundler = Bundler::new(BundlerOptions {
+        input: Some(vec![format!("./{source_folder}/scripts/entry.js").into()]),
+        dir: Some("./dist/scripts".to_string()),
+        cwd: Some(absolute_websites_path),
+        minify: Some(RawMinifyOptions::Bool(true)),
+        sourcemap: Some(SourceMapType::File),
+        // advanced_chunks: Some(AdvancedChunksOptions {
+        //     // min_size: Some(1000.0),
+        //     min_share_count: Some(20),
+        //     // min_module_size: S
+        //     // include_dependencies_recursively: Some(true),
+        //     ..Default::default()
+        // }),
+        //
+        // inline_dynamic_imports
+        // experimental: Some(ExperimentalOptions {
+        //     strict_execution_order: Some(true),
+        //     ..Default::default()
+        // }),
+        optimization: Some(OptimizationOption {
+            inline_const: Some(InlineConstOption::Config(InlineConstConfig {
+                mode: Some(InlineConstMode::All),
+                ..Default::default()
+            })),
+            // Needs benchmarks
+            // pife_for_module_wrappers: Some(true),
+            ..Default::default()
+        }),
+        ..Default::default()
+    })
+    .unwrap();
+
+    if let Err(error) = bundler.write().await {
+        error!("{error:?}");
+    }
+
+    let update_dist_index = move || {
+        let mut contents = fs::read_to_string(&absolute_source_index_path).unwrap();
+
+        if let Ok(entry) = fs::read_to_string(&absolute_dist_scripts_entry_path_clone)
+            && let Some(start) = entry.find("main")
+            && let Some(end) = entry.find(".js")
+        {
+            let main_hashed = &entry[start..end];
+            contents = contents.replace("/scripts/main.js", &format!("/scripts/{main_hashed}.js"));
+        }
+
+        let _ = fs::write(&absolute_dist_index_path, contents);
+    };
+
+    let update_source_sw = move || {
+        let contents = fs::read_to_string(&absolute_source_sw_path)
+            .unwrap()
+            .replace("__VERSION__", &format!("v{VERSION}"));
+        let _ = fs::write(&absolute_dist_sw_path, contents);
+    };
+
+    update_dist_index();
+    update_source_sw();
+
+    if !watch {
+        return Ok(relative_dist_path);
+    }
+
+    tokio::spawn(async move {
+        let mut event_watcher = notify::recommended_watcher(
+            move |res: Result<notify::Event, notify::Error>| match res {
+                Ok(event) => match event.kind {
+                    EventKind::Create(_) => event.paths,
+                    EventKind::Modify(_) => event.paths,
+                    _ => vec![],
+                }
+                .into_iter()
+                .for_each(|path| {
+                    let path = path.absolutize();
+
+                    if path == absolute_dist_scripts_entry_path
+                        || path == absolute_source_index_path_clone
+                    {
+                        update_dist_index();
+                    } else if path == absolute_source_sw_path_clone {
+                        update_source_sw();
+                    } else if let Ok(suffix) = path.strip_prefix(&absolute_modules_path) {
+                        let source_modules_path = absolute_source_scripts_modules_path.join(suffix);
+                        if path.is_file() {
+                            let _ = fs::create_dir_all(path.parent().unwrap());
+                            let _ = fs::copy(&path, &source_modules_path);
+                        }
+                    } else if let Ok(suffix) = path.strip_prefix(&absolute_source_path)
+                        // scripts are handled by rolldown
+                        && !path.starts_with(&absolute_source_scripts_path)
+                    {
+                        let dist_path = absolute_dist_path.join(suffix);
+                        if path.is_file() {
+                            let _ = fs::create_dir_all(path.parent().unwrap());
+                            let _ = fs::copy(&path, &dist_path);
+                        }
+                    }
+                }),
+                Err(e) => error!("watch error: {e:?}"),
+            },
+        )
+        .unwrap();
+
+        event_watcher
+            .watch(&absolute_websites_path_clone, RecursiveMode::Recursive)
+            .unwrap();
+        event_watcher
+            .watch(&absolute_modules_path_clone, RecursiveMode::Recursive)
+            .unwrap();
+
+        let watcher = rolldown::Watcher::new(vec![Arc::new(Mutex::new(bundler))], None).unwrap();
+
+        watcher.start().await;
+    });
+
+    Ok(relative_dist_path)
+}
+
+fn copy_dir_all(src: impl AsRef<Path>, dst: impl AsRef<Path>) -> io::Result<()> {
+    fs::create_dir_all(&dst)?;
+    for entry in fs::read_dir(src)? {
+        let entry = entry?;
+        let ty = entry.file_type()?;
+        if ty.is_dir() {
+            copy_dir_all(entry.path(), dst.as_ref().join(entry.file_name()))?;
+        } else {
+            fs::copy(entry.path(), dst.as_ref().join(entry.file_name()))?;
+        }
+    }
+    Ok(())
+}

crates/brk_cli/Cargo.toml

@@ -1,36 +1,41 @@
 [package]
 name = "brk_cli"
-description = "A command line interface to interact with the full Bitcoin Research Kit"
+description = "A command line interface to run a BRK instance"
 version.workspace = true
 edition.workspace = true
 license.workspace = true
 homepage.workspace = true
 repository.workspace = true
+build = "build.rs"
 
 [dependencies]
-bitcoincore-rpc = { workspace = true }
+brk_binder = { workspace = true }
+brk_bundler = { workspace = true }
 brk_computer = { workspace = true }
-brk_core = { workspace = true }
-brk_exit = { workspace = true }
+brk_error = { workspace = true }
 brk_fetcher = { workspace = true }
 brk_indexer = { workspace = true }
+brk_iterator = { workspace = true }
 brk_logger = { workspace = true }
-brk_parser = { workspace = true }
+brk_mempool = { workspace = true }
 brk_query = { workspace = true }
+brk_reader = { workspace = true }
+brk_rpc = { workspace = true }
 brk_server = { workspace = true }
-brk_vec = { workspace = true }
-clap = { workspace = true }
-clap_derive = { workspace = true }
+clap = { version = "4.5.53", features = ["derive", "string"] }
 color-eyre = { workspace = true }
 log = { workspace = true }
+mimalloc = { workspace = true }
+minreq = { workspace = true }
 serde = { workspace = true }
-tabled = { workspace = true }
 tokio = { workspace = true }
-toml = "0.8.23"
+toml = "0.9.10"
+vecdb = { workspace = true }
+zip = { version = "7.0.0", default-features = false, features = ["deflate"] }
 
 [[bin]]
 name = "brk"
 path = "src/main.rs"
 
 [package.metadata.dist]
-dist = false
+dist = true

crates/brk_cli/README.md

@@ -1,91 +1,54 @@
-# BRK Cli
+# brk_cli
 
-<p align="left">
-  <a href="https://github.com/bitcoinresearchkit/brk">
-    <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
-  </a>
-  <a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
-    <img src="https://img.shields.io/crates/l/brk" alt="License" />
-  </a>
-  <a href="https://crates.io/crates/brk_cli">
-    <img src="https://img.shields.io/crates/v/brk_cli" alt="Version" />
-  </a>
-  <a href="https://docs.rs/brk_cli">
-    <img src="https://img.shields.io/docsrs/brk_cli" alt="Documentation" />
-  </a>
-  <img src="https://img.shields.io/crates/size/brk_cli" alt="Size" />
-  <a href="https://deps.rs/crate/brk_cli">
-    <img src="https://deps.rs/crate/brk_cli/latest/status.svg" alt="Dependency status">
-  </a>
-  <a href="https://discord.gg/HaR3wpH3nr">
-    <img src="https://img.shields.io/discord/1350431684562124850?label=discord" alt="Discord" />
-  </a>
-  <a href="https://primal.net/p/nprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6">
-    <img src="https://img.shields.io/badge/nostr-purple?link=https%3A%2F%2Fprimal.net%2Fp%2Fnprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6" alt="Nostr" />
-  </a>
-  <a href="https://bsky.app/profile/bitcoinresearchkit.org">
-    <img src="https://img.shields.io/badge/bluesky-blue?link=https%3A%2F%2Fbsky.app%2Fprofile%2Fbitcoinresearchkit.org" alt="Bluesky" />
-  </a>
-  <a href="https://x.com/brkdotorg">
-    <img src="https://img.shields.io/badge/x.com-black" alt="X" />
-  </a>
-</p>
+Command-line interface for running the Bitcoin Research Kit.
 
-A command line interface to interact with the full Bitcoin Research Kit. It's built on top of every other create and gives the possility to use BRK using the terminal instead of Rust.
+## What It Enables
 
-It has 2 commandes for now (other than `help` and `version`) which are `run` and `query`. The former is used to run the processing (indexer + computer) and/or the server. The latter uses `brk_query` as its backend just like to server to be able to get datasets via the terminal instead of the API. Both commands are very costumizable by having all the parameters of their Rust counterparts ([`run`](https://github.com/bitcoinresearchkit/brk/blob/main/crates/brk_cli/src/run.rs#L91-L147), [`query`](https://github.com/bitcoinresearchkit/brk/blob/main/crates/brk_query/src/params.rs)).
+Run a full BRK instance: index the blockchain, compute metrics, serve the API, and optionally host a web interface. Continuously syncs with new blocks.
 
-## Requirements
+## Key Features
 
-### Hardware
-
-#### Recommended
-
-- [Latest base model Mac mini](https://www.apple.com/mac-mini/)
-- [Thunderbolt 4 SSD enclosure](https://satechi.net/products/usb4-nvme-ssd-pro-enclosure/Z2lkOi8vc2hvcGlmeS9Qcm9kdWN0VmFyaWFudC80MDE4ODQ3MDA2NzI4OA==?queryID=7961465089021ee203a60db7e62e90d2)
-- [2 TB NVMe SSD](https://shop.sandisk.com/products/ssd/internal-ssd/wd-black-sn850x-nvme-ssd?sku=WDS200T2X0E-00BCA0)
-
-#### Minimum
-
-To be determined
-
-### Software
-
-- [Bitcoin](https://bitcoin.org/en/full-node)
-- [Rust](https://www.rust-lang.org/tools/install)
-- Unix based operating system (Mac OS or Linux)
-  - Ubuntu users need to install `open-ssl` via `sudo apt install libssl-dev pkg-config`
-
-## Download
-
-### Binaries
-
-You can find a pre-built binary for your operating system on the releases page ([link](https://github.com/bitcoinresearchkit/brk/releases/latest)).
-
-### Cargo
-
-```bash
-# Install
-cargo install brk # or `cargo install brk_cli`, the result is the same
-
-# Update
-cargo install brk # or `cargo install-update -a` if you have `cargo-update` installed
-```
-
-### Source
-
-```bash
-git clone https://github.com/bitcoinresearchkit/brk.git
-cd brk/crates/brk
-cargo run -r
-```
+- **All-in-one**: Single binary runs indexer, computer, mempool monitor, and server
+- **Auto-sync**: Waits for new blocks and processes them automatically
+- **Web interface**: Downloads and bundles frontend from GitHub releases
+- **Configurable**: TOML config for RPC, paths, and features
+- **Collision checking**: Optional TXID collision validation mode
+- **Memory optimized**: Uses mimalloc allocator, 512MB stack for deep recursion
 
 ## Usage
 
-Run `brk -h` to view each available command and their respective description.
+```bash
+# See all options
+brk --help
 
-`-h` works also for commands, which mean that `brk run -h` will explain all the parameters of `brk run` for example.
+# The CLI will:
+# 1. Index new blocks
+# 2. Compute derived metrics
+# 3. Start mempool monitor
+# 4. Launch API server (port 3110)
+# 5. Wait for new blocks and repeat
+```
 
-Every parameter set for `brk run` will be saved at `~/.brk/config.toml`, which will allow you to simply run `brk run` next time.
+## Components
 
-Then the easiest to let others access your server is to use `cloudflared` which will also cache requests. For more information go to: https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/
+1. **Indexer**: Processes blocks into queryable indexes
+2. **Computer**: Derives 1000+ on-chain metrics
+3. **Mempool**: Real-time fee estimation
+4. **Server**: REST API + MCP endpoint
+5. **Bundler**: JS bundling for web interface (if enabled)
+
+## Performance
+
+| Machine | Time | Disk | Peak Disk | Memory | Peak Memory |
+|---------|------|------|-----------|--------|-------------|
+| MBP M3 Pro (36GB, internal SSD) | 5.2h | 341 GB | 415 GB | 6.4 GB | 12 GB |
+
+Full benchmark data: [`https://github.com/bitcoinresearchkit/benches/tree/main/brk`](/benches/brk)
+
+## Built On
+
+- `brk_indexer` for blockchain indexing
+- `brk_computer` for metric computation
+- `brk_mempool` for mempool monitoring
+- `brk_server` for HTTP API
+- `brk_bundler` for web interface bundling

crates/brk_cli/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_cli/src/config.rs

@@ -1,184 +1,86 @@
 use std::{
     fs,
     path::{Path, PathBuf},
-    thread::{self, sleep},
-    time::Duration,
 };
 
-use bitcoincore_rpc::{self, Auth, Client, RpcApi};
-use brk_computer::Computer;
-use brk_core::{default_bitcoin_path, default_brk_path, dot_brk_path};
-use brk_exit::Exit;
+use brk_error::{Error, Result};
 use brk_fetcher::Fetcher;
-use brk_indexer::Indexer;
-use brk_server::{Server, Website};
-use brk_vec::{Computation, Format};
-use clap_derive::{Parser, ValueEnum};
-use color_eyre::eyre::eyre;
-use log::info;
-use serde::{Deserialize, Serialize};
+use brk_rpc::{Auth, Client};
+use clap::Parser;
+use serde::{Deserialize, Deserializer, Serialize};
 
-pub fn run(config: RunConfig) -> color_eyre::Result<()> {
-    let config = RunConfig::import(Some(config))?;
+use crate::{default_brk_path, dot_brk_path, website::Website};
 
-    let rpc = config.rpc()?;
-
-    let exit = Exit::new();
-
-    let parser = brk_parser::Parser::new(config.blocksdir(), rpc);
-
-    let format = config.format();
-
-    let mut indexer = Indexer::new(&config.outputsdir(), format, config.check_collisions())?;
-    indexer.import_stores()?;
-    indexer.import_vecs()?;
-
-    let wait_for_synced_node = || -> color_eyre::Result<()> {
-        let is_synced = || -> color_eyre::Result<bool> {
-            let info = rpc.get_blockchain_info()?;
-            Ok(info.headers == info.blocks)
-        };
-
-        if !is_synced()? {
-            info!("Waiting for node to be synced...");
-            while !is_synced()? {
-                sleep(Duration::from_secs(1))
-            }
-        }
-
-        Ok(())
-    };
-
-    let f = move || -> color_eyre::Result<()> {
-        let mut computer = Computer::new(&config.outputsdir(), config.fetcher(), format);
-        computer.import_stores(&indexer)?;
-        computer.import_vecs(&indexer, config.computation())?;
-
-        tokio::runtime::Builder::new_multi_thread()
-            .enable_all()
-            .build()?
-            .block_on(async {
-                let server = if config.serve() {
-                    let served_indexer = indexer.clone();
-                    let served_computer = computer.clone();
-
-                    let server = Server::new(served_indexer, served_computer, config.website())?;
-
-                    let opt = Some(tokio::spawn(async move {
-                        server.serve().await.unwrap();
-                    }));
-
-                    sleep(Duration::from_secs(1));
-
-                    opt
-                } else {
-                    None
-                };
-
-                if config.process() {
-                    loop {
-                        wait_for_synced_node()?;
-
-                        let block_count = rpc.get_block_count()?;
-
-                        info!("{} blocks found.", block_count + 1);
-
-                        let starting_indexes = indexer.index(&parser, rpc, &exit)?;
-
-                        computer.compute(&mut indexer, starting_indexes, &exit)?;
-
-                        if let Some(delay) = config.delay() {
-                            sleep(Duration::from_secs(delay))
-                        }
-
-                        info!("Waiting for new blocks...");
-
-                        while block_count == rpc.get_block_count()? {
-                            sleep(Duration::from_secs(1))
-                        }
-                    }
-                }
-
-                if let Some(handle) = server {
-                    handle.await.unwrap();
-                }
-
-                Ok(())
-            })
-    };
-
-    thread::Builder::new()
-        .stack_size(128 * 1024 * 1024)
-        .spawn(f)?
-        .join()
-        .unwrap()
-}
+const DOWNLOADS: &str = "downloads";
 
 #[derive(Parser, Debug, Default, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize)]
-pub struct RunConfig {
+#[command(version, about)]
+pub struct Config {
     /// Bitcoin main directory path, defaults: ~/.bitcoin, ~/Library/Application\ Support/Bitcoin, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PATH")]
     bitcoindir: Option<String>,
 
     /// Bitcoin blocks directory path, default: --bitcoindir/blocks, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PATH")]
     blocksdir: Option<String>,
 
     /// Bitcoin Research Kit outputs directory path, default: ~/.brk, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PATH")]
     brkdir: Option<String>,
 
-    /// Executed by the runner, default: all, saved
-    #[arg(short, long)]
-    mode: Option<Mode>,
-
-    /// Computation mode for compatible datasets, `lazy` computes data whenever requested without saving it, `eager` computes the data once and saves it to disk, default: Lazy, saved
-    #[arg(short, long)]
-    computation: Option<Computation>,
-
-    /// Activate compression of datasets, set to true to save disk space or false if prioritize speed, default: true, saved
-    #[arg(short, long, value_name = "FORMAT")]
-    format: Option<Format>,
-
-    /// Activate fetching prices from exchanges APIs and the computation of all related datasets, default: true, saved
+    /// Activate fetching prices from BRK's API and the computation of all price related datasets, default: true, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(short = 'F', long, value_name = "BOOL")]
     fetch: Option<bool>,
 
-    /// Website served by the server (if active), default: default, saved
+    /// Activate fetching prices from exchanges APIs if `fetch` is also set to `true`, default: true, saved
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(long, value_name = "BOOL")]
+    exchanges: Option<bool>,
+
+    /// Website served by the server, default: default, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(short, long)]
     website: Option<Website>,
 
     /// Bitcoin RPC ip, default: localhost, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "IP")]
     rpcconnect: Option<String>,
 
     /// Bitcoin RPC port, default: 8332, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PORT")]
     rpcport: Option<u16>,
 
     /// Bitcoin RPC cookie file, default: --bitcoindir/.cookie, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PATH")]
     rpccookiefile: Option<String>,
 
     /// Bitcoin RPC username, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "USERNAME")]
     rpcuser: Option<String>,
 
     /// Bitcoin RPC password, saved
+    #[serde(default, deserialize_with = "default_on_error")]
     #[arg(long, value_name = "PASSWORD")]
     rpcpassword: Option<String>,
 
-    /// Delay between runs, default: 0, saved
-    #[arg(long, value_name = "SECONDS")]
-    delay: Option<u64>,
-
     /// DEV: Activate checking address hashes for collisions when indexing, default: false, saved
-    #[arg(long, value_name = "BOOL")]
+    #[serde(default, deserialize_with = "default_on_error")]
+    #[arg(skip)]
     check_collisions: Option<bool>,
 }
 
-impl RunConfig {
-    pub fn import(config_args: Option<RunConfig>) -> color_eyre::Result<Self> {
+impl Config {
+    pub fn import() -> Result<Self> {
+        let config_args = Some(Config::parse());
+
         let path = dot_brk_path();
 
         let _ = fs::create_dir_all(&path);
@@ -200,20 +102,12 @@ impl RunConfig {
                 config_saved.brkdir = Some(brkdir);
             }
 
-            if let Some(mode) = config_args.mode.take() {
-                config_saved.mode = Some(mode);
-            }
-
-            if let Some(computation) = config_args.computation.take() {
-                config_saved.computation = Some(computation);
-            }
-
             if let Some(fetch) = config_args.fetch.take() {
                 config_saved.fetch = Some(fetch);
             }
 
-            if let Some(format) = config_args.format.take() {
-                config_saved.format = Some(format);
+            if let Some(exchanges) = config_args.exchanges.take() {
+                config_saved.exchanges = Some(exchanges);
             }
 
             if let Some(website) = config_args.website.take() {
@@ -240,15 +134,11 @@ impl RunConfig {
                 config_saved.rpcpassword = Some(rpcpassword);
             }
 
-            if let Some(delay) = config_args.delay.take() {
-                config_saved.delay = Some(delay);
-            }
-
             if let Some(check_collisions) = config_args.check_collisions.take() {
                 config_saved.check_collisions = Some(check_collisions);
             }
 
-            if config_args != RunConfig::default() {
+            if config_args != Config::default() {
                 dbg!(config_args);
                 panic!("Didn't consume the full config")
             }
@@ -260,19 +150,6 @@ impl RunConfig {
 
         config.write(&path)?;
 
-        // info!("Configuration {{");
-        // info!("  bitcoindir: {:?}", config.bitcoindir);
-        // info!("  brkdir: {:?}", config.brkdir);
-        // info!("  mode: {:?}", config.mode);
-        // info!("  website: {:?}", config.website);
-        // info!("  rpcconnect: {:?}", config.rpcconnect);
-        // info!("  rpcport: {:?}", config.rpcport);
-        // info!("  rpccookiefile: {:?}", config.rpccookiefile);
-        // info!("  rpcuser: {:?}", config.rpcuser);
-        // info!("  rpcpassword: {:?}", config.rpcpassword);
-        // info!("  delay: {:?}", config.delay);
-        // info!("}}");
-
         Ok(config)
     }
 
@@ -300,7 +177,9 @@ impl RunConfig {
 
         if self.rpc_auth().is_err() {
             println!(
-                "No way found to authenticate the RPC client, please either set --rpccookiefile or --rpcuser and --rpcpassword.\nRun the program with '-h' for help."
+                "Unsuccessful authentication with the RPC client.
+First make sure that `bitcoind` is running. If it is then please either set --rpccookiefile or --rpcuser and --rpcpassword as the default values seemed to have failed.
+Finally, you can run the program with '-h' for help."
             );
             std::process::exit(1);
         }
@@ -308,7 +187,7 @@ impl RunConfig {
 
     fn read(path: &Path) -> Self {
         fs::read_to_string(path).map_or_else(
-            |_| RunConfig::default(),
+            |_| Config::default(),
             |contents| toml::from_str(&contents).unwrap_or_default(),
         )
     }
@@ -317,18 +196,18 @@ impl RunConfig {
         fs::write(path, toml::to_string(self).unwrap())
     }
 
-    pub fn rpc(&self) -> color_eyre::Result<&'static Client> {
-        Ok(Box::leak(Box::new(Client::new(
+    pub fn rpc(&self) -> Result<Client> {
+        Client::new(
             &format!(
                 "http://{}:{}",
                 self.rpcconnect().unwrap_or(&"localhost".to_string()),
                 self.rpcport().unwrap_or(8332)
             ),
-            self.rpc_auth().unwrap(),
-        )?)))
+            self.rpc_auth()?,
+        )
     }
 
-    fn rpc_auth(&self) -> color_eyre::Result<Auth> {
+    fn rpc_auth(&self) -> Result<Auth> {
         let cookie = self.path_cookiefile();
 
         if cookie.is_file() {
@@ -339,7 +218,7 @@ impl RunConfig {
                 self.rpcpassword.clone().unwrap(),
             ))
         } else {
-            Err(eyre!("Failed to find correct auth"))
+            Err(Error::AuthFailed)
         }
     }
 
@@ -351,14 +230,12 @@ impl RunConfig {
         self.rpcport
     }
 
-    pub fn delay(&self) -> Option<u64> {
-        self.delay
-    }
-
     pub fn bitcoindir(&self) -> PathBuf {
         self.bitcoindir
             .as_ref()
-            .map_or_else(default_bitcoin_path, |s| Self::fix_user_path(s.as_ref()))
+            .map_or_else(Client::default_bitcoin_path, |s| {
+                Self::fix_user_path(s.as_ref())
+            })
     }
 
     pub fn blocksdir(&self) -> PathBuf {
@@ -374,22 +251,12 @@ impl RunConfig {
             .map_or_else(default_brk_path, |s| Self::fix_user_path(s.as_ref()))
     }
 
-    pub fn outputsdir(&self) -> PathBuf {
-        self.brkdir().join("outputs")
-    }
-
     pub fn harsdir(&self) -> PathBuf {
-        self.outputsdir().join("hars")
+        self.brkdir().join("hars")
     }
 
-    pub fn process(&self) -> bool {
-        self.mode
-            .is_none_or(|m| m == Mode::All || m == Mode::Processor)
-    }
-
-    pub fn serve(&self) -> bool {
-        self.mode
-            .is_none_or(|m| m == Mode::All || m == Mode::Server)
+    pub fn downloads_dir(&self) -> PathBuf {
+        dot_brk_path().join(DOWNLOADS)
     }
 
     fn path_cookiefile(&self) -> PathBuf {
@@ -418,24 +285,20 @@ impl RunConfig {
     }
 
     pub fn website(&self) -> Website {
-        self.website.unwrap_or(Website::Default)
+        self.website.unwrap_or(Website::Bitview)
     }
 
     pub fn fetch(&self) -> bool {
         self.fetch.is_none_or(|b| b)
     }
 
+    pub fn exchanges(&self) -> bool {
+        self.exchanges.is_none_or(|b| b)
+    }
+
     pub fn fetcher(&self) -> Option<Fetcher> {
         self.fetch()
-            .then(|| Fetcher::import(Some(self.harsdir().as_path())).unwrap())
-    }
-
-    pub fn computation(&self) -> Computation {
-        self.computation.unwrap_or_default()
-    }
-
-    pub fn format(&self) -> Format {
-        self.format.unwrap_or_default()
+            .then(|| Fetcher::import(self.exchanges(), Some(self.harsdir().as_path())).unwrap())
     }
 
     pub fn check_collisions(&self) -> bool {
@@ -443,23 +306,13 @@ impl RunConfig {
     }
 }
 
-#[derive(
-    Default,
-    Debug,
-    Clone,
-    Copy,
-    Parser,
-    ValueEnum,
-    Serialize,
-    Deserialize,
-    PartialEq,
-    Eq,
-    PartialOrd,
-    Ord,
-)]
-pub enum Mode {
-    #[default]
-    All,
-    Processor,
-    Server,
+fn default_on_error<'de, D, T>(deserializer: D) -> Result<T, D::Error>
+where
+    D: Deserializer<'de>,
+    T: Deserialize<'de> + Default,
+{
+    match T::deserialize(deserializer) {
+        Ok(v) => Ok(v),
+        Err(_) => Ok(T::default()),
+    }
 }

crates/brk_cli/src/lib.rs

@@ -1,42 +0,0 @@
-use std::fs;
-
-use brk_core::{dot_brk_log_path, dot_brk_path};
-use brk_query::Params as QueryArgs;
-use clap::Parser;
-use clap_derive::{Parser, Subcommand};
-use query::query;
-use run::{RunConfig, run};
-
-mod query;
-mod run;
-
-#[derive(Parser)]
-#[command(version, about)]
-#[command(propagate_version = true)]
-struct Cli {
-    #[command(subcommand)]
-    command: Commands,
-}
-
-#[derive(Subcommand, Debug)]
-enum Commands {
-    /// Run the indexer, computer and server
-    Run(RunConfig),
-    /// Query generated datasets via the `run` command in a similar fashion as the server's API
-    Query(QueryArgs),
-}
-
-pub fn main() -> color_eyre::Result<()> {
-    color_eyre::install()?;
-
-    fs::create_dir_all(dot_brk_path())?;
-
-    brk_logger::init(Some(&dot_brk_log_path()));
-
-    let cli = Cli::parse();
-
-    match cli.command {
-        Commands::Run(args) => run(args),
-        Commands::Query(args) => query(args),
-    }
-}

crates/brk_cli/src/main.rs

@@ -1 +1,166 @@
-use brk_cli::main;
+#![doc = include_str!("../README.md")]
+
+use std::{
+    fs,
+    io::Cursor,
+    path::Path,
+    thread::{self, sleep},
+    time::Duration,
+};
+
+use brk_binder::generate_js_files;
+use brk_bundler::bundle;
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_mempool::Mempool;
+use brk_query::AsyncQuery;
+use brk_reader::Reader;
+use brk_server::{Server, VERSION};
+use log::info;
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+mod config;
+mod paths;
+mod website;
+
+use crate::{config::Config, paths::*};
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> color_eyre::Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+pub fn run() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    fs::create_dir_all(dot_brk_path())?;
+
+    brk_logger::init(Some(&dot_brk_log_path()))?;
+
+    let config = Config::import()?;
+
+    let client = config.rpc()?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let reader = Reader::new(config.blocksdir(), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&config.brkdir())?;
+
+    let mut computer = Computer::forced_import(&config.brkdir(), &indexer, config.fetcher())?;
+
+    let mempool = Mempool::new(&client);
+
+    let mempool_clone = mempool.clone();
+    thread::spawn(move || {
+        mempool_clone.start();
+    });
+
+    let query = AsyncQuery::build(&reader, &indexer, &computer, Some(mempool));
+
+    let website = config.website();
+
+    let downloads_path = config.downloads_dir();
+
+    let future = async move {
+        let bundle_path = if website.is_some() {
+            let websites_dev_path = Path::new("../../websites");
+            let modules_dev_path = Path::new("../../modules");
+
+            let websites_path;
+            let modules_path;
+
+            if fs::exists(websites_dev_path)? && fs::exists(modules_dev_path)? {
+                websites_path = websites_dev_path.to_path_buf();
+                modules_path = modules_dev_path.to_path_buf();
+            } else {
+                let downloaded_brk_path = downloads_path.join(format!("brk-{VERSION}"));
+
+                let downloaded_websites_path = downloaded_brk_path.join("websites");
+                let downloaded_modules_path = downloaded_brk_path.join("modules");
+
+                if !fs::exists(&downloaded_websites_path)? {
+                    info!("Downloading source from Github...");
+
+                    let url = format!(
+                        "https://github.com/bitcoinresearchkit/brk/archive/refs/tags/v{VERSION}.zip",
+                    );
+
+                    let response = minreq::get(url).send()?;
+                    let bytes = response.as_bytes();
+                    let cursor = Cursor::new(bytes);
+
+                    let mut zip = zip::ZipArchive::new(cursor).unwrap();
+
+                    zip.extract(downloads_path).unwrap();
+                }
+
+                websites_path = downloaded_websites_path;
+                modules_path = downloaded_modules_path;
+            }
+
+            generate_js_files(query.inner(), &modules_path)?;
+
+            Some(
+                bundle(
+                    &modules_path,
+                    &websites_path,
+                    website.to_folder_name(),
+                    true,
+                )
+                .await?,
+            )
+        } else {
+            None
+        };
+
+        let server = Server::new(&query, bundle_path);
+
+        tokio::spawn(async move {
+            server.serve(true).await.unwrap();
+        });
+
+        Ok(()) as Result<()>
+    };
+
+    let runtime = tokio::runtime::Builder::new_multi_thread()
+        .enable_all()
+        .build()?;
+
+    let _handle = runtime.spawn(future);
+
+    loop {
+        client.wait_for_synced_node()?;
+
+        let last_height = client.get_last_height()?;
+
+        info!("{} blocks found.", u32::from(last_height) + 1);
+
+        let starting_indexes = if config.check_collisions() {
+            indexer.checked_index(&blocks, &client, &exit)?
+        } else {
+            indexer.index(&blocks, &client, &exit)?
+        };
+
+        computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+
+        info!("Waiting for new blocks...");
+
+        while last_height == client.get_last_height()? {
+            sleep(Duration::from_secs(1))
+        }
+    }
+}

crates/brk_cli/src/paths.rs

@@ -0,0 +1,14 @@
+use std::path::{Path, PathBuf};
+
+pub fn dot_brk_path() -> PathBuf {
+    let home = std::env::var("HOME").unwrap();
+    Path::new(&home).join(".brk")
+}
+
+pub fn dot_brk_log_path() -> PathBuf {
+    dot_brk_path().join("log")
+}
+
+pub fn default_brk_path() -> PathBuf {
+    dot_brk_path()
+}

crates/brk_cli/src/query.rs

@@ -1,53 +0,0 @@
-use brk_computer::Computer;
-use brk_indexer::Indexer;
-use brk_query::{Index, Output, Params as QueryParams, Query, Tabled, Value};
-use tabled::settings::Style;
-
-use crate::run::RunConfig;
-
-pub fn query(params: QueryParams) -> color_eyre::Result<()> {
-    let config = RunConfig::import(None)?;
-
-    let format = config.format();
-
-    let mut indexer = Indexer::new(&config.outputsdir(), format, config.check_collisions())?;
-    indexer.import_vecs()?;
-
-    let mut computer = Computer::new(&config.outputsdir(), config.fetcher(), format);
-    computer.import_vecs(&indexer, config.computation())?;
-
-    let query = Query::build(&indexer, &computer);
-
-    let index = Index::try_from(params.index.as_str())?;
-
-    let ids = params.values.iter().map(|s| s.as_str()).collect::<Vec<_>>();
-
-    let res = query.search_and_format(index, &ids, params.from, params.to, params.format)?;
-
-    if params.format.is_some() {
-        println!("{}", res);
-    } else {
-        println!(
-            "{}",
-            match res {
-                Output::Json(v) => match v {
-                    Value::Single(v) => v.to_string().replace("\"", ""),
-                    v => {
-                        let v = match v {
-                            Value::Single(_) => unreachable!("Already processed"),
-                            Value::List(v) => vec![v],
-                            Value::Matrix(v) => v,
-                        };
-                        let mut table =
-                            v.to_table(ids.iter().map(|id| id.to_string()).collect::<Vec<_>>());
-                        table.with(Style::psql());
-                        table.to_string()
-                    }
-                },
-                _ => unreachable!(),
-            }
-        );
-    }
-
-    Ok(())
-}

crates/brk_cli/src/website.rs

@@ -1,10 +1,11 @@
-use clap_derive::ValueEnum;
+use clap::ValueEnum;
 use serde::{Deserialize, Serialize};
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Deserialize, Serialize, ValueEnum)]
+#[serde(rename_all = "lowercase")]
 pub enum Website {
     None,
-    Default,
+    Bitview,
     Custom,
 }
 
@@ -17,10 +18,10 @@ impl Website {
         !self.is_none()
     }
 
-    pub fn to_folder_name(&self) -> &str {
+    pub fn to_folder_name(self) -> &'static str {
         match self {
             Self::Custom => "custom",
-            Self::Default => "default",
+            Self::Bitview => "bitview",
             Self::None => unreachable!(),
         }
     }

crates/brk_computer/.gitignore

@@ -0,0 +1,2 @@
+bottlenecks.md
+BUG.md

crates/brk_computer/Cargo.toml

@@ -1,25 +1,37 @@
 [package]
 name = "brk_computer"
-description = "A Bitcoin dataset computer, built on top of brk_indexer"
+description = "A Bitcoin dataset computer built on top of brk_indexer"
 version.workspace = true
 edition.workspace = true
 license.workspace = true
 homepage.workspace = true
 repository.workspace = true
+build = "build.rs"
 
 [dependencies]
 bitcoin = { workspace = true }
-bitcoincore-rpc = { workspace = true }
-brk_core = { workspace = true }
-brk_exit = { workspace = true }
+brk_error = { workspace = true }
 brk_fetcher = { workspace = true }
+brk_grouper = { workspace = true }
 brk_indexer = { workspace = true }
+brk_iterator = { workspace = true }
 brk_logger = { workspace = true }
-brk_parser = { workspace = true }
-brk_state = { workspace = true }
-brk_vec = { workspace = true }
-color-eyre = { workspace = true }
-fjall = { workspace = true }
-jiff = { workspace = true }
+brk_reader = { workspace = true }
+brk_rpc = { workspace = true }
+brk_store = { workspace = true }
+brk_traversable = { workspace = true }
+brk_types = { workspace = true }
+derive_deref = { workspace = true }
 log = { workspace = true }
+pco = "0.4.7"
 rayon = { workspace = true }
+rustc-hash = { workspace = true }
+schemars = { workspace = true }
+serde = { workspace = true }
+smallvec = { workspace = true }
+vecdb = { workspace = true }
+
+[dev-dependencies]
+brk_bencher = { workspace = true }
+color-eyre = { workspace = true }
+mimalloc = { workspace = true }

crates/brk_computer/README.md

@@ -1,34 +1,73 @@
-# BRK Computer
+# brk_computer
 
-<p align="left">
-  <a href="https://github.com/bitcoinresearchkit/brk">
-    <img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/bitcoinresearchkit/brk?style=social">
-  </a>
-  <a href="https://github.com/bitcoinresearchkit/brk/blob/main/LICENSE.md">
-    <img src="https://img.shields.io/crates/l/brk" alt="License" />
-  </a>
-  <a href="https://crates.io/crates/brk_computer">
-    <img src="https://img.shields.io/crates/v/brk_computer" alt="Version" />
-  </a>
-  <a href="https://docs.rs/brk_computer">
-    <img src="https://img.shields.io/docsrs/brk_computer" alt="Documentation" />
-  </a>
-  <img src="https://img.shields.io/crates/size/brk_computer" alt="Size" />
-  <a href="https://deps.rs/crate/brk_computer">
-    <img src="https://deps.rs/crate/brk_computer/latest/status.svg" alt="Dependency status">
-  </a>
-  <a href="https://discord.gg/HaR3wpH3nr">
-    <img src="https://img.shields.io/discord/1350431684562124850?label=discord" alt="Discord" />
-  </a>
-  <a href="https://primal.net/p/nprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6">
-    <img src="https://img.shields.io/badge/nostr-purple?link=https%3A%2F%2Fprimal.net%2Fp%2Fnprofile1qqsfw5dacngjlahye34krvgz7u0yghhjgk7gxzl5ptm9v6n2y3sn03sqxu2e6" alt="Nostr" />
-  </a>
-  <a href="https://bsky.app/profile/bitcoinresearchkit.org">
-    <img src="https://img.shields.io/badge/bluesky-blue?link=https%3A%2F%2Fbsky.app%2Fprofile%2Fbitcoinresearchkit.org" alt="Bluesky" />
-  </a>
-  <a href="https://x.com/brkdotorg">
-    <img src="https://img.shields.io/badge/x.com-black" alt="X" />
-  </a>
-</p>
+Derived metrics computation engine for Bitcoin on-chain analytics.
 
-A dataset computer, built on top of `brk_indexer` and `brk_fetcher`. It computes any dataset you can think of and if it doesn't feel free to create an issue.
+## What It Enables
+
+Compute 1000+ on-chain metrics from indexed blockchain data: supply breakdowns, realized/unrealized P&L, SOPR, MVRV, cohort analysis (by age, amount, address type), cointime economics, mining pool attribution, and price-weighted valuations.
+
+## Key Features
+
+- **Cohort metrics**: Filter by UTXO age (STH/LTH, age bands), amount ranges, address types
+- **Stateful computation**: Track per-UTXO cost basis, realized/unrealized states
+- **Multi-index support**: Metrics available by height, date, week, month, year, decade
+- **Price integration**: USD-denominated metrics when price data available
+- **Mining pool attribution**: Tag blocks/rewards to known pools
+- **Cointime economics**: Liveliness, vaultedness, activity-weighted metrics
+- **Incremental updates**: Resume from checkpoints, compute only new blocks
+
+## Core API
+
+```rust,ignore
+let mut computer = Computer::forced_import(&outputs_path, &indexer, fetcher)?;
+
+// Compute all metrics for new blocks
+computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+
+// Access computed data
+let supply = computer.chain.height_to_supply.get(height)?;
+let realized_cap = computer.stateful.utxo.all.height_to_realized_cap.get(height)?;
+```
+
+## Metric Categories
+
+| Module | Examples |
+|--------|----------|
+| `chain` | Supply, subsidy, fees, transaction counts |
+| `stateful` | Realized cap, MVRV, SOPR, unrealized P&L |
+| `cointime` | Liveliness, vaultedness, true market mean |
+| `pools` | Per-pool block counts, rewards, fees |
+| `market` | Market cap, NVT, Puell multiple |
+| `price` | Height-to-price mapping from fetched data |
+
+## Cohort System
+
+UTXO and address cohorts support filtering by:
+- **Age**: STH (<150d), LTH (≥150d), age bands (1d, 1w, 1m, 3m, 6m, 1y, 2y, ...)
+- **Amount**: 0-0.001 BTC, 0.001-0.01, ..., 10k+ BTC
+- **Type**: P2PKH, P2SH, P2WPKH, P2WSH, P2TR
+- **Epoch**: By halving epoch
+
+## Performance
+
+### End-to-End
+
+Full pipeline benchmarks (indexer + computer):
+
+| Machine | Time | Disk | Peak Disk | Memory | Peak Memory |
+|---------|------|------|-----------|--------|-------------|
+| MBP M3 Pro (36GB, internal SSD) | 5.2h | 341 GB | 415 GB | 6.4 GB | 12 GB |
+
+Full benchmark data: [`https://github.com/bitcoinresearchkit/benches/tree/main/brk`](/benches/brk)
+
+## Recommended: mimalloc v3
+
+Use [mimalloc v3](https://crates.io/crates/mimalloc) as the global allocator to reduce memory usage.
+
+## Built On
+
+- `brk_indexer` for indexed blockchain data
+- `brk_fetcher` for price data
+- `brk_reader` for raw block access
+- `brk_grouper` for cohort filtering
+- `brk_traversable` for data export

crates/brk_computer/build.rs

@@ -0,0 +1,8 @@
+fn main() {
+    let profile = std::env::var("PROFILE").unwrap_or_default();
+
+    if profile == "release" {
+        println!("cargo:rustc-flag=-C");
+        println!("cargo:rustc-flag=target-cpu=native");
+    }
+}

crates/brk_computer/examples/computer.rs

@@ -0,0 +1,68 @@
+use std::{
+    env,
+    path::Path,
+    thread::{self, sleep},
+    time::{Duration, Instant},
+};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_reader::Reader;
+use brk_rpc::{Auth, Client};
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()?;
+
+    Ok(())
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let bitcoin_dir = Client::default_bitcoin_path();
+    // let bitcoin_dir = Path::new("/Volumes/WD_BLACK/bitcoin");
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let client = Client::new(
+        Client::default_url(),
+        Auth::CookieFile(bitcoin_dir.join(".cookie")),
+    )?;
+
+    let reader = Reader::new(bitcoin_dir.join("blocks"), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let mut computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+    loop {
+        let i = Instant::now();
+        let starting_indexes = indexer.checked_index(&blocks, &client, &exit)?;
+        computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+        dbg!(i.elapsed());
+        sleep(Duration::from_secs(10));
+    }
+}

crates/brk_computer/examples/computer_bench.rs

@@ -0,0 +1,77 @@
+use std::{env, path::Path, thread, time::Instant};
+
+use brk_bencher::Bencher;
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_reader::Reader;
+use brk_rpc::{Auth, Client};
+use log::{debug, info};
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(None)?;
+
+    let bitcoin_dir = Client::default_bitcoin_path();
+    // let bitcoin_dir = Path::new("/Volumes/WD_BLACK/bitcoin");
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    let outputs_benches_dir = outputs_dir.join("benches");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let client = Client::new(
+        Client::default_url(),
+        Auth::CookieFile(bitcoin_dir.join(".cookie")),
+    )?;
+
+    let reader = Reader::new(bitcoin_dir.join("blocks"), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let mut computer = Computer::forced_import(&outputs_benches_dir, &indexer, Some(fetcher))?;
+
+    let mut bencher =
+        Bencher::from_cargo_env(env!("CARGO_PKG_NAME"), &outputs_dir.join("computed"))?;
+    bencher.start()?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+    let bencher_clone = bencher.clone();
+    exit.register_cleanup(move || {
+        let _ = bencher_clone.stop();
+        debug!("Bench stopped.");
+    });
+
+    let i = Instant::now();
+    let starting_indexes = indexer.index(&blocks, &client, &exit)?;
+    info!("Done in {:?}", i.elapsed());
+
+    let i = Instant::now();
+    computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+    info!("Done in {:?}", i.elapsed());
+
+    // We want to benchmark the drop too
+    drop(computer);
+    drop(indexer);
+
+    Ok(())
+}

crates/brk_computer/examples/computer_read.rs

@@ -0,0 +1,96 @@
+use std::{env, path::Path, thread};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_types::TxIndex;
+use mimalloc::MiMalloc;
+use vecdb::{Exit, GenericStoredVec};
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> Result<()> {
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()
+}
+
+fn run() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    // let outputs_dir = Path::new("../../_outputs");
+
+    let indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    let computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+    let txindex = TxIndex::new(134217893);
+
+    dbg!(
+        indexer
+            .vecs
+            .tx
+            .txindex_to_txid
+            .read_once(txindex)
+            .unwrap()
+            .to_string()
+    );
+    let first_txinindex = indexer
+        .vecs
+        .tx
+        .txindex_to_first_txinindex
+        .read_once(txindex)?;
+    dbg!(first_txinindex);
+    let first_txoutindex = indexer
+        .vecs
+        .tx
+        .txindex_to_first_txoutindex
+        .read_once(txindex)?;
+    dbg!(first_txoutindex);
+    let input_count = *computer.indexes.txindex_to_input_count.read_once(txindex)?;
+    dbg!(input_count);
+    let output_count = *computer
+        .indexes
+        .txindex_to_output_count
+        .read_once(txindex)?;
+    dbg!(output_count);
+
+    let _ = dbg!(computer.chain.txinindex_to_value.read_once(first_txinindex));
+    let _ = dbg!(
+        computer
+            .chain
+            .txinindex_to_value
+            .read_once(first_txinindex + 1)
+    );
+    let _ = dbg!(
+        indexer
+            .vecs
+            .txout
+            .txoutindex_to_value
+            .read_once(first_txoutindex)
+    );
+    let _ = dbg!(
+        indexer
+            .vecs
+            .txout
+            .txoutindex_to_value
+            .read_once(first_txoutindex + 1)
+    );
+    let _ = dbg!(computer.chain.txindex_to_input_value.read_once(txindex));
+    let _ = dbg!(computer.chain.txindex_to_input_value.read_once(txindex));
+    let _ = dbg!(computer.chain.txindex_to_output_value.read_once(txindex));
+    // dbg!(computer.indexes.txindex_to_txindex.ge(txindex));
+
+    Ok(())
+}

crates/brk_computer/examples/debug_indexer.rs

@@ -0,0 +1,141 @@
+use std::{env, path::Path};
+
+use brk_indexer::Indexer;
+use brk_types::{Height, P2PKHAddressIndex, P2SHAddressIndex, TxOutIndex, TypeIndex};
+use mimalloc::MiMalloc;
+use vecdb::GenericStoredVec;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+fn main() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+
+    let indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let reader_outputtype = indexer.vecs.txout.txoutindex_to_outputtype.create_reader();
+    let reader_typeindex = indexer.vecs.txout.txoutindex_to_typeindex.create_reader();
+    let reader_txindex = indexer.vecs.txout.txoutindex_to_txindex.create_reader();
+    let reader_txid = indexer.vecs.tx.txindex_to_txid.create_reader();
+    let reader_height_to_first_txoutindex = indexer
+        .vecs
+        .txout
+        .height_to_first_txoutindex
+        .create_reader();
+    let reader_p2pkh = indexer
+        .vecs
+        .address
+        .p2pkhaddressindex_to_p2pkhbytes
+        .create_reader();
+    let reader_p2sh = indexer
+        .vecs
+        .address
+        .p2shaddressindex_to_p2shbytes
+        .create_reader();
+
+    // Check what's stored at typeindex 254909199 in both P2PKH and P2SH vecs
+    let typeindex = TypeIndex::from(254909199_usize);
+
+    let p2pkh_bytes = indexer
+        .vecs
+        .address
+        .p2pkhaddressindex_to_p2pkhbytes
+        .read(P2PKHAddressIndex::from(typeindex), &reader_p2pkh);
+    println!("P2PKH at typeindex 254909199: {:?}", p2pkh_bytes);
+
+    let p2sh_bytes = indexer
+        .vecs
+        .address
+        .p2shaddressindex_to_p2shbytes
+        .read(P2SHAddressIndex::from(typeindex), &reader_p2sh);
+    println!("P2SH at typeindex 254909199: {:?}", p2sh_bytes);
+
+    // Check first P2SH index at height 476152
+    let reader_first_p2sh = indexer
+        .vecs
+        .address
+        .height_to_first_p2shaddressindex
+        .create_reader();
+    let reader_first_p2pkh = indexer
+        .vecs
+        .address
+        .height_to_first_p2pkhaddressindex
+        .create_reader();
+    let first_p2sh_at_476152 = indexer
+        .vecs
+        .address
+        .height_to_first_p2shaddressindex
+        .read(Height::from(476152_usize), &reader_first_p2sh);
+    let first_p2pkh_at_476152 = indexer
+        .vecs
+        .address
+        .height_to_first_p2pkhaddressindex
+        .read(Height::from(476152_usize), &reader_first_p2pkh);
+    println!(
+        "First P2SH index at height 476152: {:?}",
+        first_p2sh_at_476152
+    );
+    println!(
+        "First P2PKH index at height 476152: {:?}",
+        first_p2pkh_at_476152
+    );
+
+    // Check the problematic txoutindexes found during debugging
+    for txoutindex_usize in [653399433_usize, 653399443_usize] {
+        let txoutindex = TxOutIndex::from(txoutindex_usize);
+        let outputtype = indexer
+            .vecs
+            .txout
+            .txoutindex_to_outputtype
+            .read(txoutindex, &reader_outputtype)
+            .unwrap();
+        let typeindex = indexer
+            .vecs
+            .txout
+            .txoutindex_to_typeindex
+            .read(txoutindex, &reader_typeindex)
+            .unwrap();
+        let txindex = indexer
+            .vecs
+            .txout
+            .txoutindex_to_txindex
+            .read(txoutindex, &reader_txindex)
+            .unwrap();
+        let txid = indexer
+            .vecs
+            .tx
+            .txindex_to_txid
+            .read(txindex, &reader_txid)
+            .unwrap();
+
+        // Find height by binary search
+        let mut height = Height::from(0_usize);
+        for h in 0..900_000_usize {
+            let first_txoutindex = indexer
+                .vecs
+                .txout
+                .height_to_first_txoutindex
+                .read(Height::from(h), &reader_height_to_first_txoutindex);
+            if let Ok(first) = first_txoutindex {
+                if usize::from(first) > txoutindex_usize {
+                    break;
+                }
+                height = Height::from(h);
+            }
+        }
+
+        println!(
+            "txoutindex={}, outputtype={:?}, typeindex={}, txindex={}, txid={}, height={}",
+            txoutindex_usize,
+            outputtype,
+            usize::from(typeindex),
+            usize::from(txindex),
+            txid,
+            usize::from(height)
+        );
+    }
+
+    Ok(())
+}

crates/brk_computer/examples/full_bench.rs

@@ -0,0 +1,83 @@
+use std::{
+    env, fs,
+    path::Path,
+    thread::{self, sleep},
+    time::{Duration, Instant},
+};
+
+use brk_bencher::Bencher;
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_iterator::Blocks;
+use brk_reader::Reader;
+use brk_rpc::{Auth, Client};
+use log::{debug, info};
+use mimalloc::MiMalloc;
+use vecdb::Exit;
+
+#[global_allocator]
+static GLOBAL: MiMalloc = MiMalloc;
+
+pub fn main() -> color_eyre::Result<()> {
+    color_eyre::install()?;
+
+    // Can't increase main thread's stack size, thus we need to use another thread
+    thread::Builder::new()
+        .stack_size(512 * 1024 * 1024)
+        .spawn(run)?
+        .join()
+        .unwrap()?;
+
+    Ok(())
+}
+
+fn run() -> Result<()> {
+    let bitcoin_dir = Client::default_bitcoin_path();
+    // let bitcoin_dir = Path::new("/Volumes/WD_BLACK1/bitcoin");
+
+    let outputs_dir = Path::new(&env::var("HOME").unwrap()).join(".brk");
+    // let outputs_dir = Path::new("/Volumes/WD_BLACK1/brk");
+    fs::create_dir_all(&outputs_dir)?;
+
+    brk_logger::init(Some(&outputs_dir.join("log")))?;
+
+    let client = Client::new(
+        Client::default_url(),
+        Auth::CookieFile(bitcoin_dir.join(".cookie")),
+    )?;
+
+    let reader = Reader::new(bitcoin_dir.join("blocks"), &client);
+
+    let blocks = Blocks::new(&client, &reader);
+
+    let mut indexer = Indexer::forced_import(&outputs_dir)?;
+
+    let fetcher = Fetcher::import(true, None)?;
+
+    let mut computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+    let mut bencher = Bencher::from_cargo_env("brk", &outputs_dir)?;
+    bencher.start()?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+    let bencher_clone = bencher.clone();
+    exit.register_cleanup(move || {
+        let _ = bencher_clone.stop();
+        debug!("Bench stopped.");
+    });
+
+    loop {
+        let i = Instant::now();
+        let starting_indexes = indexer.index(&blocks, &client, &exit)?;
+        info!("Done in {:?}", i.elapsed());
+
+        let i = Instant::now();
+        computer.compute(&indexer, starting_indexes, &reader, &exit)?;
+        info!("Done in {:?}", i.elapsed());
+
+        sleep(Duration::from_secs(60));
+    }
+}

crates/brk_computer/examples/main.rs

@@ -1,54 +0,0 @@
-use std::{path::Path, thread};
-
-use brk_computer::Computer;
-use brk_core::{default_bitcoin_path, default_brk_path};
-use brk_exit::Exit;
-use brk_fetcher::Fetcher;
-use brk_indexer::Indexer;
-use brk_parser::Parser;
-use brk_vec::{Computation, Format};
-
-pub fn main() -> color_eyre::Result<()> {
-    color_eyre::install()?;
-
-    brk_logger::init(Some(Path::new(".log")));
-
-    let bitcoin_dir = default_bitcoin_path();
-
-    let rpc = Box::leak(Box::new(bitcoincore_rpc::Client::new(
-        "http://localhost:8332",
-        bitcoincore_rpc::Auth::CookieFile(bitcoin_dir.join(".cookie")),
-    )?));
-    let exit = Exit::new();
-
-    // Can't increase main thread's stack programatically, thus we need to use another thread
-    thread::Builder::new()
-        .stack_size(32 * 1024 * 1024)
-        .spawn(move || -> color_eyre::Result<()> {
-            let parser = Parser::new(bitcoin_dir.join("blocks"), rpc);
-
-            let _outputs_dir = default_brk_path().join("outputs");
-            let outputs_dir = _outputs_dir.as_path();
-            // let outputs_dir = Path::new("../../_outputs");
-
-            let format = Format::Raw;
-
-            let mut indexer = Indexer::new(outputs_dir, format, true)?;
-            indexer.import_stores()?;
-            indexer.import_vecs()?;
-
-            let fetcher = Fetcher::import(None)?;
-
-            let mut computer = Computer::new(outputs_dir, Some(fetcher), format);
-            computer.import_stores(&indexer)?;
-            computer.import_vecs(&indexer, Computation::Lazy)?;
-
-            let starting_indexes = indexer.index(&parser, rpc, &exit)?;
-
-            computer.compute(&mut indexer, starting_indexes, &exit)?;
-
-            Ok(())
-        })?
-        .join()
-        .unwrap()
-}

crates/brk_computer/examples/pools.rs

@@ -0,0 +1,121 @@
+use std::{collections::BTreeMap, path::Path, thread};
+
+use brk_computer::Computer;
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_types::{Address, AddressBytes, OutputType, TxOutIndex, pools};
+use vecdb::{Exit, IterableVec, TypedVecIterator};
+
+fn main() -> Result<()> {
+    brk_logger::init(Some(Path::new(".log")))?;
+
+    let exit = Exit::new();
+    exit.set_ctrlc_handler();
+
+    thread::Builder::new()
+        .stack_size(256 * 1024 * 1024)
+        .spawn(move || -> Result<()> {
+            let outputs_dir = Path::new(&std::env::var("HOME").unwrap()).join(".brk");
+
+            let indexer = Indexer::forced_import(&outputs_dir)?;
+
+            let fetcher = Fetcher::import(true, None)?;
+
+            let computer = Computer::forced_import(&outputs_dir, &indexer, Some(fetcher))?;
+
+            let pools = pools();
+
+            let mut res: BTreeMap<&'static str, usize> = BTreeMap::default();
+
+            let vecs = indexer.vecs;
+            let stores = indexer.stores;
+
+            let mut height_to_first_txindex_iter = vecs.tx.height_to_first_txindex.iter()?;
+            let mut txindex_to_first_txoutindex_iter = vecs.tx.txindex_to_first_txoutindex.iter()?;
+            let mut txindex_to_output_count_iter = computer.indexes.txindex_to_output_count.iter();
+            let mut txoutindex_to_outputtype_iter = vecs.txout.txoutindex_to_outputtype.iter()?;
+            let mut txoutindex_to_typeindex_iter = vecs.txout.txoutindex_to_typeindex.iter()?;
+            let mut p2pk65addressindex_to_p2pk65bytes_iter =
+                vecs.address.p2pk65addressindex_to_p2pk65bytes.iter()?;
+            let mut p2pk33addressindex_to_p2pk33bytes_iter =
+                vecs.address.p2pk33addressindex_to_p2pk33bytes.iter()?;
+            let mut p2pkhaddressindex_to_p2pkhbytes_iter =
+                vecs.address.p2pkhaddressindex_to_p2pkhbytes.iter()?;
+            let mut p2shaddressindex_to_p2shbytes_iter =
+                vecs.address.p2shaddressindex_to_p2shbytes.iter()?;
+            let mut p2wpkhaddressindex_to_p2wpkhbytes_iter =
+                vecs.address.p2wpkhaddressindex_to_p2wpkhbytes.iter()?;
+            let mut p2wshaddressindex_to_p2wshbytes_iter =
+                vecs.address.p2wshaddressindex_to_p2wshbytes.iter()?;
+            let mut p2traddressindex_to_p2trbytes_iter =
+                vecs.address.p2traddressindex_to_p2trbytes.iter()?;
+            let mut p2aaddressindex_to_p2abytes_iter = vecs.address.p2aaddressindex_to_p2abytes.iter()?;
+
+            let unknown = pools.get_unknown();
+
+            stores
+                .height_to_coinbase_tag
+                .iter()
+                .for_each(|(height, coinbase_tag)| {
+                    let txindex = height_to_first_txindex_iter.get_unwrap(height);
+                    let txoutindex = txindex_to_first_txoutindex_iter.get_unwrap(txindex);
+                    let outputcount = txindex_to_output_count_iter.get_unwrap(txindex);
+
+                    let pool = (*txoutindex..(*txoutindex + *outputcount))
+                        .map(TxOutIndex::from)
+                        .find_map(|txoutindex| {
+                            let outputtype = txoutindex_to_outputtype_iter.get_unwrap(txoutindex);
+                            let typeindex = txoutindex_to_typeindex_iter.get_unwrap(txoutindex);
+
+                            match outputtype {
+                                OutputType::P2PK65 => Some(AddressBytes::from(
+                                    p2pk65addressindex_to_p2pk65bytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2PK33 => Some(AddressBytes::from(
+                                    p2pk33addressindex_to_p2pk33bytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2PKH => Some(AddressBytes::from(
+                                    p2pkhaddressindex_to_p2pkhbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2SH => Some(AddressBytes::from(
+                                    p2shaddressindex_to_p2shbytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2WPKH => Some(AddressBytes::from(
+                                    p2wpkhaddressindex_to_p2wpkhbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2WSH => Some(AddressBytes::from(
+                                    p2wshaddressindex_to_p2wshbytes_iter
+                                        .get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2TR => Some(AddressBytes::from(
+                                    p2traddressindex_to_p2trbytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                OutputType::P2A => Some(AddressBytes::from(
+                                    p2aaddressindex_to_p2abytes_iter.get_unwrap(typeindex.into()),
+                                )),
+                                _ => None,
+                            }
+                            .map(|bytes| Address::try_from(&bytes).unwrap())
+                            .and_then(|address| pools.find_from_address(&address))
+                        })
+                        .or_else(|| pools.find_from_coinbase_tag(&coinbase_tag))
+                        .unwrap_or(unknown);
+
+                    *res.entry(pool.name).or_default() += 1;
+                });
+
+            let mut v = res.into_iter().map(|(k, v)| (v, k)).collect::<Vec<_>>();
+            v.sort_unstable();
+            println!("{:#?}", v);
+            println!("{:#?}", v.len());
+
+            Ok(())
+        })?
+        .join()
+        .unwrap()
+}

crates/brk_computer/flamegraph.sh

@@ -0,0 +1,2 @@
+cargo build --profile profiling
+flamegraph -- ../../target/profiling/examples/main

crates/brk_computer/samply.sh

@@ -0,0 +1,2 @@
+cargo build --example main --profile profiling
+samply record ../../target/profiling/examples/main

crates/brk_computer/src/blks.rs

@@ -0,0 +1,119 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_reader::Reader;
+use brk_traversable::Traversable;
+use brk_types::{BlkPosition, Height, TxIndex, Version};
+use vecdb::{
+    AnyStoredVec, AnyVec, Database, Exit, GenericStoredVec, ImportableVec, PAGE_SIZE, PcoVec,
+    TypedVecIterator,
+};
+
+use super::Indexes;
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub height_to_position: PcoVec<Height, BlkPosition>,
+    pub txindex_to_position: PcoVec<TxIndex, BlkPosition>,
+}
+
+impl Vecs {
+    pub fn forced_import(parent_path: &Path, parent_version: Version) -> Result<Self> {
+        let db = Database::open(&parent_path.join("blks"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let this = Self {
+            height_to_position: PcoVec::forced_import(&db, "position", version + Version::TWO)?,
+            txindex_to_position: PcoVec::forced_import(&db, "position", version + Version::TWO)?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: &Indexes,
+        reader: &Reader,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexer, starting_indexes, reader, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: &Indexes,
+        parser: &Reader,
+        exit: &Exit,
+    ) -> Result<()> {
+        let min_txindex =
+            TxIndex::from(self.txindex_to_position.len()).min(starting_indexes.txindex);
+
+        let Some(min_height) = indexer
+            .vecs
+            .tx.txindex_to_height
+            .iter()?
+            .get(min_txindex)
+            .map(|h| h.min(starting_indexes.height))
+        else {
+            return Ok(());
+        };
+
+        let mut height_to_first_txindex_iter = indexer.vecs.tx.height_to_first_txindex.iter()?;
+
+        parser
+            .read(
+                Some(min_height),
+                Some((indexer.vecs.tx.height_to_first_txindex.len() - 1).into()),
+            )
+            .iter()
+            .try_for_each(|block| -> Result<()> {
+                let height = block.height();
+
+                self.height_to_position
+                    .truncate_push(height, block.metadata().position())?;
+
+                let txindex = height_to_first_txindex_iter.get_unwrap(height);
+
+                block.tx_metadata().iter().enumerate().try_for_each(
+                    |(index, metadata)| -> Result<()> {
+                        let txindex = txindex + index;
+                        self.txindex_to_position
+                            .truncate_push(txindex, metadata.position())?;
+                        Ok(())
+                    },
+                )?;
+
+                if *height % 1_000 == 0 {
+                    let _lock = exit.lock();
+                    self.height_to_position.flush()?;
+                    self.txindex_to_position.flush()?;
+                }
+                Ok(())
+            })?;
+
+        let _lock = exit.lock();
+        self.height_to_position.flush()?;
+        self.txindex_to_position.flush()?;
+
+        Ok(())
+    }
+}

crates/brk_computer/src/chain/import.rs

@@ -0,0 +1,475 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{StoredBool, StoredU64, TxIndex, Version, Weight};
+use vecdb::{
+    Database, EagerVec, ImportableVec, IterableCloneableVec, LazyVecFrom1, LazyVecFrom2, PAGE_SIZE,
+    VecIndex,
+};
+
+use crate::{
+    grouped::{
+        ComputedValueVecsFromHeight, ComputedValueVecsFromTxindex, ComputedVecsFromDateIndex,
+        ComputedVecsFromHeight, ComputedVecsFromTxindex, Source, VecBuilderOptions,
+    },
+    indexes, price,
+};
+
+use super::{
+    Vecs, TARGET_BLOCKS_PER_DAY, TARGET_BLOCKS_PER_DECADE, TARGET_BLOCKS_PER_MONTH,
+    TARGET_BLOCKS_PER_QUARTER, TARGET_BLOCKS_PER_SEMESTER, TARGET_BLOCKS_PER_WEEK,
+    TARGET_BLOCKS_PER_YEAR,
+};
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("chain"))?;
+        db.set_min_len(PAGE_SIZE * 50_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let compute_dollars = price.is_some();
+        let v0 = Version::ZERO;
+        let v2 = Version::TWO;
+        let v4 = Version::new(4);
+        let v5 = Version::new(5);
+
+        // Helper macros for common patterns
+        macro_rules! eager {
+            ($name:expr) => {
+                EagerVec::forced_import(&db, $name, version + v0)?
+            };
+            ($name:expr, $v:expr) => {
+                EagerVec::forced_import(&db, $name, version + $v)?
+            };
+        }
+        macro_rules! computed_h {
+            ($name:expr, $source:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $source:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_di {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    version + $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_tx {
+            ($name:expr, $source:expr, $opts:expr) => {
+                ComputedVecsFromTxindex::forced_import(
+                    &db,
+                    $name,
+                    $source,
+                    version + v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        let last = || VecBuilderOptions::default().add_last();
+        let sum = || VecBuilderOptions::default().add_sum();
+        let sum_cum = || VecBuilderOptions::default().add_sum().add_cumulative();
+        let stats = || {
+            VecBuilderOptions::default()
+                .add_average()
+                .add_minmax()
+                .add_percentiles()
+        };
+        let full_stats = || {
+            VecBuilderOptions::default()
+                .add_average()
+                .add_minmax()
+                .add_percentiles()
+                .add_sum()
+                .add_cumulative()
+        };
+
+        let txinindex_to_value = eager!("value");
+
+        let txindex_to_weight = LazyVecFrom2::init(
+            "weight",
+            version + Version::ZERO,
+            indexer.vecs.tx.txindex_to_base_size.boxed_clone(),
+            indexer.vecs.tx.txindex_to_total_size.boxed_clone(),
+            |index: TxIndex, txindex_to_base_size_iter, txindex_to_total_size_iter| {
+                let index = index.to_usize();
+                txindex_to_base_size_iter.get_at(index).map(|base_size| {
+                    let total_size = txindex_to_total_size_iter.get_at_unwrap(index);
+
+                    // This is the exact definition of a weight unit, as defined by BIP-141 (quote above).
+                    let wu = usize::from(base_size) * 3 + usize::from(total_size);
+
+                    Weight::from(bitcoin::Weight::from_wu_usize(wu))
+                })
+            },
+        );
+
+        let txindex_to_vsize = LazyVecFrom1::init(
+            "vsize",
+            version + Version::ZERO,
+            txindex_to_weight.boxed_clone(),
+            |index: TxIndex, iter| iter.get(index).map(brk_types::VSize::from),
+        );
+
+        let txindex_to_is_coinbase = LazyVecFrom2::init(
+            "is_coinbase",
+            version + Version::ZERO,
+            indexer.vecs.tx.txindex_to_height.boxed_clone(),
+            indexer.vecs.tx.height_to_first_txindex.boxed_clone(),
+            |index: TxIndex, txindex_to_height_iter, height_to_first_txindex_iter| {
+                txindex_to_height_iter.get(index).map(|height| {
+                    let txindex = height_to_first_txindex_iter.get_unwrap(height);
+                    StoredBool::from(index == txindex)
+                })
+            },
+        );
+
+        let txindex_to_input_value = eager!("input_value");
+        let txindex_to_output_value = eager!("output_value");
+        let txindex_to_fee = eager!("fee");
+        let txindex_to_fee_rate = eager!("fee_rate");
+
+        let dateindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.dateindex_to_dateindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_DAY)),
+        );
+        let weekindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.weekindex_to_weekindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_WEEK)),
+        );
+        let monthindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.monthindex_to_monthindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_MONTH)),
+        );
+        let quarterindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.quarterindex_to_quarterindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_QUARTER)),
+        );
+        let semesterindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.semesterindex_to_semesterindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_SEMESTER)),
+        );
+        let yearindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.yearindex_to_yearindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_YEAR)),
+        );
+        let decadeindex_to_block_count_target = LazyVecFrom1::init(
+            "block_count_target",
+            version + Version::ZERO,
+            indexes.decadeindex_to_decadeindex.boxed_clone(),
+            |_, _| Some(StoredU64::from(TARGET_BLOCKS_PER_DECADE)),
+        );
+
+        let this = Self {
+            dateindex_to_block_count_target,
+            weekindex_to_block_count_target,
+            monthindex_to_block_count_target,
+            quarterindex_to_block_count_target,
+            semesterindex_to_block_count_target,
+            yearindex_to_block_count_target,
+            decadeindex_to_block_count_target,
+            height_to_interval: eager!("interval"),
+            timeindexes_to_timestamp: computed_di!(
+                "timestamp",
+                VecBuilderOptions::default().add_first()
+            ),
+            indexes_to_block_interval: computed_h!("block_interval", Source::None, stats()),
+            indexes_to_block_count: computed_h!("block_count", Source::Compute, sum_cum()),
+            indexes_to_1w_block_count: computed_di!("1w_block_count", last()),
+            indexes_to_1m_block_count: computed_di!("1m_block_count", last()),
+            indexes_to_1y_block_count: computed_di!("1y_block_count", last()),
+            indexes_to_block_weight: computed_h!("block_weight", Source::None, full_stats()),
+            indexes_to_block_size: computed_h!("block_size", Source::None, full_stats()),
+            height_to_vbytes: eager!("vbytes"),
+            height_to_24h_block_count: eager!("24h_block_count"),
+            height_to_24h_coinbase_sum: eager!("24h_coinbase_sum"),
+            height_to_24h_coinbase_usd_sum: eager!("24h_coinbase_usd_sum"),
+            indexes_to_block_vbytes: computed_h!("block_vbytes", Source::None, full_stats()),
+            difficultyepoch_to_timestamp: eager!("timestamp"),
+            halvingepoch_to_timestamp: eager!("timestamp"),
+
+            dateindex_to_fee_dominance: eager!("fee_dominance"),
+            dateindex_to_subsidy_dominance: eager!("subsidy_dominance"),
+            indexes_to_difficulty: computed_h!("difficulty", Source::None, last()),
+            indexes_to_difficultyepoch: computed_di!("difficultyepoch", last()),
+            indexes_to_halvingepoch: computed_di!("halvingepoch", last()),
+            indexes_to_tx_count: computed_h!("tx_count", Source::Compute, full_stats()),
+            indexes_to_input_count: computed_tx!("input_count", Source::None, full_stats()),
+            indexes_to_output_count: computed_tx!("output_count", Source::None, full_stats()),
+            indexes_to_tx_v1: computed_h!("tx_v1", Source::Compute, sum_cum()),
+            indexes_to_tx_v2: computed_h!("tx_v2", Source::Compute, sum_cum()),
+            indexes_to_tx_v3: computed_h!("tx_v3", Source::Compute, sum_cum()),
+            indexes_to_sent_sum: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "sent_sum",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default().add_sum(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_fee: ComputedValueVecsFromTxindex::forced_import(
+                &db,
+                "fee",
+                indexer,
+                indexes,
+                Source::Vec(txindex_to_fee.boxed_clone()),
+                version + Version::ZERO,
+                price,
+                VecBuilderOptions::default()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_percentiles()
+                    .add_minmax()
+                    .add_average(),
+            )?,
+            indexes_to_fee_rate: computed_tx!("fee_rate", Source::None, stats()),
+            indexes_to_tx_vsize: computed_tx!("tx_vsize", Source::None, stats()),
+            indexes_to_tx_weight: computed_tx!("tx_weight", Source::None, stats()),
+            indexes_to_subsidy: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "subsidy",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default()
+                    .add_percentiles()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_minmax()
+                    .add_average(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_coinbase: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "coinbase",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default()
+                    .add_sum()
+                    .add_cumulative()
+                    .add_percentiles()
+                    .add_minmax()
+                    .add_average(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_unclaimed_rewards: ComputedValueVecsFromHeight::forced_import(
+                &db,
+                "unclaimed_rewards",
+                Source::Compute,
+                version + Version::ZERO,
+                VecBuilderOptions::default().add_sum().add_cumulative(),
+                compute_dollars,
+                indexes,
+            )?,
+            indexes_to_p2a_count: computed_h!("p2a_count", Source::Compute, full_stats()),
+            indexes_to_p2ms_count: computed_h!("p2ms_count", Source::Compute, full_stats()),
+            indexes_to_p2pk33_count: computed_h!("p2pk33_count", Source::Compute, full_stats()),
+            indexes_to_p2pk65_count: computed_h!("p2pk65_count", Source::Compute, full_stats()),
+            indexes_to_p2pkh_count: computed_h!("p2pkh_count", Source::Compute, full_stats()),
+            indexes_to_p2sh_count: computed_h!("p2sh_count", Source::Compute, full_stats()),
+            indexes_to_p2tr_count: computed_h!("p2tr_count", Source::Compute, full_stats()),
+            indexes_to_p2wpkh_count: computed_h!("p2wpkh_count", Source::Compute, full_stats()),
+            indexes_to_p2wsh_count: computed_h!("p2wsh_count", Source::Compute, full_stats()),
+            indexes_to_opreturn_count: computed_h!("opreturn_count", Source::Compute, full_stats()),
+            indexes_to_unknownoutput_count: computed_h!(
+                "unknownoutput_count",
+                Source::Compute,
+                full_stats()
+            ),
+            indexes_to_emptyoutput_count: computed_h!(
+                "emptyoutput_count",
+                Source::Compute,
+                full_stats()
+            ),
+            indexes_to_exact_utxo_count: computed_h!("exact_utxo_count", Source::Compute, last()),
+            indexes_to_subsidy_usd_1y_sma: compute_dollars
+                .then(|| {
+                    ComputedVecsFromDateIndex::forced_import(
+                        &db,
+                        "subsidy_usd_1y_sma",
+                        Source::Compute,
+                        version + v0,
+                        indexes,
+                        last(),
+                    )
+                })
+                .transpose()?,
+            indexes_to_puell_multiple: compute_dollars
+                .then(|| {
+                    ComputedVecsFromDateIndex::forced_import(
+                        &db,
+                        "puell_multiple",
+                        Source::Compute,
+                        version + v0,
+                        indexes,
+                        last(),
+                    )
+                })
+                .transpose()?,
+            indexes_to_hash_rate: computed_h!("hash_rate", Source::Compute, v5, last()),
+            indexes_to_hash_rate_1w_sma: computed_di!("hash_rate_1w_sma", last()),
+            indexes_to_hash_rate_1m_sma: computed_di!("hash_rate_1m_sma", last()),
+            indexes_to_hash_rate_2m_sma: computed_di!("hash_rate_2m_sma", last()),
+            indexes_to_hash_rate_1y_sma: computed_di!("hash_rate_1y_sma", last()),
+            indexes_to_difficulty_as_hash: computed_h!(
+                "difficulty_as_hash",
+                Source::Compute,
+                last()
+            ),
+            indexes_to_difficulty_adjustment: computed_h!(
+                "difficulty_adjustment",
+                Source::Compute,
+                sum()
+            ),
+            indexes_to_blocks_before_next_difficulty_adjustment: computed_h!(
+                "blocks_before_next_difficulty_adjustment",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_days_before_next_difficulty_adjustment: computed_h!(
+                "days_before_next_difficulty_adjustment",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_blocks_before_next_halving: computed_h!(
+                "blocks_before_next_halving",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_days_before_next_halving: computed_h!(
+                "days_before_next_halving",
+                Source::Compute,
+                v2,
+                last()
+            ),
+            indexes_to_hash_price_ths: computed_h!("hash_price_ths", Source::Compute, v4, last()),
+            indexes_to_hash_price_phs: computed_h!("hash_price_phs", Source::Compute, v4, last()),
+            indexes_to_hash_value_ths: computed_h!("hash_value_ths", Source::Compute, v4, last()),
+            indexes_to_hash_value_phs: computed_h!("hash_value_phs", Source::Compute, v4, last()),
+            indexes_to_hash_price_ths_min: computed_h!(
+                "hash_price_ths_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_price_phs_min: computed_h!(
+                "hash_price_phs_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_price_rebound: computed_h!(
+                "hash_price_rebound",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_ths_min: computed_h!(
+                "hash_value_ths_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_phs_min: computed_h!(
+                "hash_value_phs_min",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_hash_value_rebound: computed_h!(
+                "hash_value_rebound",
+                Source::Compute,
+                v4,
+                last()
+            ),
+            indexes_to_inflation_rate: computed_di!("inflation_rate", last()),
+            indexes_to_annualized_volume: computed_di!("annualized_volume", last()),
+            indexes_to_annualized_volume_btc: computed_di!("annualized_volume_btc", last()),
+            indexes_to_annualized_volume_usd: computed_di!("annualized_volume_usd", last()),
+            indexes_to_tx_btc_velocity: computed_di!("tx_btc_velocity", last()),
+            indexes_to_tx_usd_velocity: computed_di!("tx_usd_velocity", last()),
+            indexes_to_tx_per_sec: computed_di!("tx_per_sec", v2, last()),
+            indexes_to_outputs_per_sec: computed_di!("outputs_per_sec", v2, last()),
+            indexes_to_inputs_per_sec: computed_di!("inputs_per_sec", v2, last()),
+
+            txindex_to_is_coinbase,
+            txinindex_to_value,
+            txindex_to_input_value,
+            txindex_to_output_value,
+            txindex_to_fee,
+            txindex_to_fee_rate,
+            txindex_to_vsize,
+            txindex_to_weight,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+}

crates/brk_computer/src/chain/mod.rs

@@ -0,0 +1,132 @@
+mod compute;
+mod import;
+
+use brk_traversable::Traversable;
+use brk_types::{
+    Bitcoin, DateIndex, DecadeIndex, DifficultyEpoch, Dollars, FeeRate, HalvingEpoch, Height,
+    MonthIndex, QuarterIndex, Sats, SemesterIndex, StoredBool, StoredF32, StoredF64, StoredU32,
+    StoredU64, Timestamp, TxInIndex, TxIndex, VSize, WeekIndex, Weight, YearIndex,
+};
+use vecdb::{Database, EagerVec, LazyVecFrom1, LazyVecFrom2, PcoVec};
+
+use crate::grouped::{
+    ComputedValueVecsFromHeight, ComputedValueVecsFromTxindex, ComputedVecsFromDateIndex,
+    ComputedVecsFromHeight, ComputedVecsFromTxindex,
+};
+
+pub(crate) const TARGET_BLOCKS_PER_DAY_F64: f64 = 144.0;
+pub(crate) const TARGET_BLOCKS_PER_DAY_F32: f32 = 144.0;
+pub(crate) const TARGET_BLOCKS_PER_DAY: u64 = 144;
+pub(crate) const TARGET_BLOCKS_PER_WEEK: u64 = 7 * TARGET_BLOCKS_PER_DAY;
+pub(crate) const TARGET_BLOCKS_PER_MONTH: u64 = 30 * TARGET_BLOCKS_PER_DAY;
+pub(crate) const TARGET_BLOCKS_PER_QUARTER: u64 = 3 * TARGET_BLOCKS_PER_MONTH;
+pub(crate) const TARGET_BLOCKS_PER_SEMESTER: u64 = 2 * TARGET_BLOCKS_PER_QUARTER;
+pub(crate) const TARGET_BLOCKS_PER_YEAR: u64 = 2 * TARGET_BLOCKS_PER_SEMESTER;
+pub(crate) const TARGET_BLOCKS_PER_DECADE: u64 = 10 * TARGET_BLOCKS_PER_YEAR;
+pub(crate) const ONE_TERA_HASH: f64 = 1_000_000_000_000.0;
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    pub(crate) db: Database,
+
+    pub dateindex_to_block_count_target: LazyVecFrom1<DateIndex, StoredU64, DateIndex, DateIndex>,
+    pub weekindex_to_block_count_target: LazyVecFrom1<WeekIndex, StoredU64, WeekIndex, WeekIndex>,
+    pub monthindex_to_block_count_target:
+        LazyVecFrom1<MonthIndex, StoredU64, MonthIndex, MonthIndex>,
+    pub quarterindex_to_block_count_target:
+        LazyVecFrom1<QuarterIndex, StoredU64, QuarterIndex, QuarterIndex>,
+    pub semesterindex_to_block_count_target:
+        LazyVecFrom1<SemesterIndex, StoredU64, SemesterIndex, SemesterIndex>,
+    pub yearindex_to_block_count_target: LazyVecFrom1<YearIndex, StoredU64, YearIndex, YearIndex>,
+    pub decadeindex_to_block_count_target:
+        LazyVecFrom1<DecadeIndex, StoredU64, DecadeIndex, DecadeIndex>,
+    pub height_to_interval: EagerVec<PcoVec<Height, Timestamp>>,
+    pub height_to_24h_block_count: EagerVec<PcoVec<Height, StoredU32>>,
+    pub height_to_24h_coinbase_sum: EagerVec<PcoVec<Height, Sats>>,
+    pub height_to_24h_coinbase_usd_sum: EagerVec<PcoVec<Height, Dollars>>,
+    pub height_to_vbytes: EagerVec<PcoVec<Height, StoredU64>>,
+    pub difficultyepoch_to_timestamp: EagerVec<PcoVec<DifficultyEpoch, Timestamp>>,
+    pub halvingepoch_to_timestamp: EagerVec<PcoVec<HalvingEpoch, Timestamp>>,
+    pub timeindexes_to_timestamp: ComputedVecsFromDateIndex<Timestamp>,
+    pub indexes_to_block_count: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_1w_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_1m_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_1y_block_count: ComputedVecsFromDateIndex<StoredU32>,
+    pub indexes_to_block_interval: ComputedVecsFromHeight<Timestamp>,
+    pub indexes_to_block_size: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_block_vbytes: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_block_weight: ComputedVecsFromHeight<Weight>,
+    pub indexes_to_difficulty: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_difficultyepoch: ComputedVecsFromDateIndex<DifficultyEpoch>,
+    pub indexes_to_halvingepoch: ComputedVecsFromDateIndex<HalvingEpoch>,
+    pub indexes_to_coinbase: ComputedValueVecsFromHeight,
+    pub indexes_to_emptyoutput_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_fee: ComputedValueVecsFromTxindex,
+    pub indexes_to_fee_rate: ComputedVecsFromTxindex<FeeRate>,
+    /// Value == 0 when Coinbase
+    pub txindex_to_input_value: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub indexes_to_sent_sum: ComputedValueVecsFromHeight,
+    pub indexes_to_opreturn_count: ComputedVecsFromHeight<StoredU64>,
+    pub txindex_to_output_value: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub indexes_to_p2a_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2ms_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pk33_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pk65_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2pkh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2sh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2tr_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2wpkh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_p2wsh_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_subsidy: ComputedValueVecsFromHeight,
+    pub indexes_to_unclaimed_rewards: ComputedValueVecsFromHeight,
+    pub indexes_to_tx_count: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v1: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v2: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_v3: ComputedVecsFromHeight<StoredU64>,
+    pub indexes_to_tx_vsize: ComputedVecsFromTxindex<VSize>,
+    pub indexes_to_tx_weight: ComputedVecsFromTxindex<Weight>,
+    pub indexes_to_unknownoutput_count: ComputedVecsFromHeight<StoredU64>,
+    pub txinindex_to_value: EagerVec<PcoVec<TxInIndex, Sats>>,
+    pub indexes_to_input_count: ComputedVecsFromTxindex<StoredU64>,
+    pub txindex_to_is_coinbase: LazyVecFrom2<TxIndex, StoredBool, TxIndex, Height, Height, TxIndex>,
+    pub indexes_to_output_count: ComputedVecsFromTxindex<StoredU64>,
+    pub txindex_to_vsize: LazyVecFrom1<TxIndex, VSize, TxIndex, Weight>,
+    pub txindex_to_weight: LazyVecFrom2<TxIndex, Weight, TxIndex, StoredU32, TxIndex, StoredU32>,
+    pub txindex_to_fee: EagerVec<PcoVec<TxIndex, Sats>>,
+    pub txindex_to_fee_rate: EagerVec<PcoVec<TxIndex, FeeRate>>,
+    pub indexes_to_exact_utxo_count: ComputedVecsFromHeight<StoredU64>,
+    pub dateindex_to_fee_dominance: EagerVec<PcoVec<DateIndex, StoredF32>>,
+    pub dateindex_to_subsidy_dominance: EagerVec<PcoVec<DateIndex, StoredF32>>,
+    pub indexes_to_subsidy_usd_1y_sma: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub indexes_to_puell_multiple: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub indexes_to_hash_rate: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_hash_rate_1w_sma: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_hash_rate_1m_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_rate_2m_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_rate_1y_sma: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_hash_price_ths: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_ths_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_phs: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_phs_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_price_rebound: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_ths: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_ths_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_phs: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_phs_min: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_hash_value_rebound: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_difficulty_as_hash: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_difficulty_adjustment: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_blocks_before_next_difficulty_adjustment: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_days_before_next_difficulty_adjustment: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_blocks_before_next_halving: ComputedVecsFromHeight<StoredU32>,
+    pub indexes_to_days_before_next_halving: ComputedVecsFromHeight<StoredF32>,
+    pub indexes_to_inflation_rate: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_annualized_volume: ComputedVecsFromDateIndex<Sats>,
+    pub indexes_to_annualized_volume_btc: ComputedVecsFromDateIndex<Bitcoin>,
+    pub indexes_to_annualized_volume_usd: ComputedVecsFromDateIndex<Dollars>,
+    pub indexes_to_tx_btc_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_tx_usd_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_tx_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_outputs_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_inputs_per_sec: ComputedVecsFromDateIndex<StoredF32>,
+}

crates/brk_computer/src/cointime.rs

@@ -0,0 +1,577 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, CheckedSub, Dollars, StoredF32, StoredF64, Version};
+use vecdb::{Database, Exit, PAGE_SIZE, TypedVecIterator};
+
+use crate::{grouped::ComputedVecsFromDateIndex, utils::OptionExt};
+
+use super::{
+    Indexes, chain,
+    grouped::{
+        ComputedRatioVecsFromDateIndex, ComputedValueVecsFromHeight, ComputedVecsFromHeight,
+        Source, VecBuilderOptions,
+    },
+    indexes, price, stateful,
+};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub indexes_to_coinblocks_created: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_coinblocks_stored: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_liveliness: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_vaultedness: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_activity_to_vaultedness_ratio: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_vaulted_supply: ComputedValueVecsFromHeight,
+    pub indexes_to_active_supply: ComputedValueVecsFromHeight,
+    pub indexes_to_thermo_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_investor_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_active_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_vaulted_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_active_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_active_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_true_market_mean: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_true_market_mean_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_cointime_value_destroyed: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_value_created: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_value_stored: ComputedVecsFromHeight<StoredF64>,
+    pub indexes_to_cointime_price: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_cointime_cap: ComputedVecsFromHeight<Dollars>,
+    pub indexes_to_cointime_price_ratio: ComputedRatioVecsFromDateIndex,
+    pub indexes_to_cointime_adj_inflation_rate: ComputedVecsFromDateIndex<StoredF32>,
+    pub indexes_to_cointime_adj_tx_btc_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    pub indexes_to_cointime_adj_tx_usd_velocity: ComputedVecsFromDateIndex<StoredF64>,
+    // pub indexes_to_thermo_cap_rel_to_investor_cap: ComputedValueVecsFromHeight,
+}
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("cointime"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let compute_dollars = price.is_some();
+        let v0 = parent_version;
+        let v1 = parent_version + Version::ONE;
+
+        let last = || VecBuilderOptions::default().add_last();
+        let sum_cum = || VecBuilderOptions::default().add_sum().add_cumulative();
+
+        macro_rules! computed_h {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+            ($name:expr, $v:expr, $opts:expr) => {
+                ComputedVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    $v,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! computed_di {
+            ($name:expr, $opts:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v0,
+                    indexes,
+                    $opts,
+                )?
+            };
+        }
+        macro_rules! ratio_di {
+            ($name:expr) => {
+                ComputedRatioVecsFromDateIndex::forced_import(
+                    &db,
+                    $name,
+                    Source::None,
+                    v0,
+                    indexes,
+                    true,
+                )?
+            };
+        }
+        macro_rules! value_h {
+            ($name:expr) => {
+                ComputedValueVecsFromHeight::forced_import(
+                    &db,
+                    $name,
+                    Source::Compute,
+                    v1,
+                    last(),
+                    compute_dollars,
+                    indexes,
+                )?
+            };
+        }
+
+        let this = Self {
+            indexes_to_coinblocks_created: computed_h!("coinblocks_created", sum_cum()),
+            indexes_to_coinblocks_stored: computed_h!("coinblocks_stored", sum_cum()),
+            indexes_to_liveliness: computed_h!("liveliness", last()),
+            indexes_to_vaultedness: computed_h!("vaultedness", last()),
+            indexes_to_activity_to_vaultedness_ratio: computed_h!(
+                "activity_to_vaultedness_ratio",
+                last()
+            ),
+            indexes_to_vaulted_supply: value_h!("vaulted_supply"),
+            indexes_to_active_supply: value_h!("active_supply"),
+            indexes_to_thermo_cap: computed_h!("thermo_cap", v1, last()),
+            indexes_to_investor_cap: computed_h!("investor_cap", v1, last()),
+            indexes_to_vaulted_cap: computed_h!("vaulted_cap", v1, last()),
+            indexes_to_active_cap: computed_h!("active_cap", v1, last()),
+            indexes_to_vaulted_price: computed_h!("vaulted_price", last()),
+            indexes_to_vaulted_price_ratio: ratio_di!("vaulted_price"),
+            indexes_to_active_price: computed_h!("active_price", last()),
+            indexes_to_active_price_ratio: ratio_di!("active_price"),
+            indexes_to_true_market_mean: computed_h!("true_market_mean", last()),
+            indexes_to_true_market_mean_ratio: ratio_di!("true_market_mean"),
+            indexes_to_cointime_value_destroyed: computed_h!("cointime_value_destroyed", sum_cum()),
+            indexes_to_cointime_value_created: computed_h!("cointime_value_created", sum_cum()),
+            indexes_to_cointime_value_stored: computed_h!("cointime_value_stored", sum_cum()),
+            indexes_to_cointime_price: computed_h!("cointime_price", last()),
+            indexes_to_cointime_cap: computed_h!("cointime_cap", last()),
+            indexes_to_cointime_price_ratio: ratio_di!("cointime_price"),
+            indexes_to_cointime_adj_inflation_rate: computed_di!(
+                "cointime_adj_inflation_rate",
+                last()
+            ),
+            indexes_to_cointime_adj_tx_btc_velocity: computed_di!(
+                "cointime_adj_tx_btc_velocity",
+                last()
+            ),
+            indexes_to_cointime_adj_tx_usd_velocity: computed_di!(
+                "cointime_adj_tx_usd_velocity",
+                last()
+            ),
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    #[allow(clippy::too_many_arguments)]
+    pub fn compute(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        price: Option<&price::Vecs>,
+        chain: &chain::Vecs,
+        stateful: &stateful::Vecs,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexes, starting_indexes, price, chain, stateful, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    #[allow(clippy::too_many_arguments)]
+    fn compute_(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        price: Option<&price::Vecs>,
+        chain: &chain::Vecs,
+        stateful: &stateful::Vecs,
+        exit: &Exit,
+    ) -> Result<()> {
+        let circulating_supply = &stateful.utxo_cohorts.all.metrics.supply.height_to_supply;
+
+        self.indexes_to_coinblocks_created
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_transform(
+                    starting_indexes.height,
+                    circulating_supply,
+                    |(i, v, ..)| (i, StoredF64::from(Bitcoin::from(v))),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        let indexes_to_coinblocks_destroyed = &stateful
+            .utxo_cohorts
+            .all
+            .metrics
+            .activity
+            .indexes_to_coinblocks_destroyed;
+
+        self.indexes_to_coinblocks_stored
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                let mut coinblocks_destroyed_iter = indexes_to_coinblocks_destroyed
+                    .height
+                    .as_ref()
+                    .unwrap()
+                    .into_iter();
+                vec.compute_transform(
+                    starting_indexes.height,
+                    self.indexes_to_coinblocks_created.height.u(),
+                    |(i, created, ..)| {
+                        let destroyed = coinblocks_destroyed_iter.get_unwrap(i);
+                        (i, created.checked_sub(destroyed).unwrap())
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_liveliness
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_divide(
+                    starting_indexes.height,
+                    indexes_to_coinblocks_destroyed
+                        .height_extra
+                        .unwrap_cumulative(),
+                    self.indexes_to_coinblocks_created
+                        .height_extra
+                        .unwrap_cumulative(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        let liveliness = &self.indexes_to_liveliness;
+
+        self.indexes_to_vaultedness
+            .compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_transform(
+                    starting_indexes.height,
+                    liveliness.height.u(),
+                    |(i, v, ..)| (i, StoredF64::from(1.0).checked_sub(v).unwrap()),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        let vaultedness = &self.indexes_to_vaultedness;
+
+        self.indexes_to_activity_to_vaultedness_ratio.compute_all(
+            indexes,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_divide(
+                    starting_indexes.height,
+                    liveliness.height.u(),
+                    vaultedness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_vaulted_supply.compute_all(
+            indexes,
+            price,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_multiply(
+                    starting_indexes.height,
+                    circulating_supply,
+                    vaultedness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_active_supply.compute_all(
+            indexes,
+            price,
+            starting_indexes,
+            exit,
+            |vec| {
+                vec.compute_multiply(
+                    starting_indexes.height,
+                    circulating_supply,
+                    liveliness.height.u(),
+                    exit,
+                )?;
+                Ok(())
+            },
+        )?;
+
+        self.indexes_to_cointime_adj_inflation_rate
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_multiply(
+                    starting_indexes.dateindex,
+                    self.indexes_to_activity_to_vaultedness_ratio
+                        .dateindex
+                        .unwrap_last(),
+                    chain.indexes_to_inflation_rate.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_cointime_adj_tx_btc_velocity
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_multiply(
+                    starting_indexes.dateindex,
+                    self.indexes_to_activity_to_vaultedness_ratio
+                        .dateindex
+                        .unwrap_last(),
+                    chain.indexes_to_tx_btc_velocity.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        if let Some(price) = price {
+            let realized_cap = &stateful
+                .utxo_cohorts
+                .all
+                .metrics
+                .realized
+                .u()
+                .height_to_realized_cap;
+            let realized_price = stateful
+                .utxo_cohorts
+                .all
+                .metrics
+                .realized
+                .u()
+                .indexes_to_realized_price
+                .height
+                .u();
+
+            self.indexes_to_thermo_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_transform(
+                        starting_indexes.height,
+                        chain
+                            .indexes_to_subsidy
+                            .dollars
+                            .as_ref()
+                            .unwrap()
+                            .height_extra
+                            .unwrap_cumulative(),
+                        |(i, v, ..)| (i, v),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_investor_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_subtract(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_thermo_cap.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_vaultedness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_active_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        realized_cap,
+                        self.indexes_to_liveliness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        realized_price,
+                        self.indexes_to_vaultedness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_vaulted_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_vaulted_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_active_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        realized_price,
+                        self.indexes_to_liveliness.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_active_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_active_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_true_market_mean.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        self.indexes_to_investor_cap.height.u(),
+                        self.indexes_to_active_supply
+                            .bitcoin
+                            .height
+                            .as_ref()
+                            .unwrap(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_true_market_mean_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_true_market_mean.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_cointime_value_destroyed.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    // TODO: Another example when the callback should be applied to each index, instead of to base then merging from more granular to less
+                    // The price taken won't be correct for time based indexes
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        indexes_to_coinblocks_destroyed.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_value_created.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        self.indexes_to_coinblocks_created.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_value_stored.compute_all(
+                indexes,
+                starting_indexes,
+                exit,
+                |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        &price.chainindexes_to_price_close.height,
+                        self.indexes_to_coinblocks_stored.height.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+
+            self.indexes_to_cointime_price
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_divide(
+                        starting_indexes.height,
+                        self.indexes_to_cointime_value_destroyed
+                            .height_extra
+                            .unwrap_cumulative(),
+                        self.indexes_to_coinblocks_stored
+                            .height_extra
+                            .unwrap_cumulative(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_cointime_cap
+                .compute_all(indexes, starting_indexes, exit, |vec| {
+                    vec.compute_multiply(
+                        starting_indexes.height,
+                        self.indexes_to_cointime_price.height.u(),
+                        circulating_supply,
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+            self.indexes_to_cointime_price_ratio.compute_rest(
+                price,
+                starting_indexes,
+                exit,
+                Some(self.indexes_to_cointime_price.dateindex.unwrap_last()),
+            )?;
+
+            self.indexes_to_cointime_adj_tx_usd_velocity.compute_all(
+                starting_indexes,
+                exit,
+                |v| {
+                    v.compute_multiply(
+                        starting_indexes.dateindex,
+                        self.indexes_to_activity_to_vaultedness_ratio
+                            .dateindex
+                            .unwrap_last(),
+                        chain.indexes_to_tx_usd_velocity.dateindex.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                },
+            )?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/constants.rs

@@ -0,0 +1,256 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{StoredF32, StoredI16, StoredU16, Version};
+use vecdb::{AnyVec, Database, Exit, PAGE_SIZE};
+
+use crate::grouped::Source;
+
+use super::{
+    Indexes,
+    grouped::{ComputedVecsFromHeight, VecBuilderOptions},
+    indexes,
+};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub constant_0: ComputedVecsFromHeight<StoredU16>,
+    pub constant_1: ComputedVecsFromHeight<StoredU16>,
+    pub constant_2: ComputedVecsFromHeight<StoredU16>,
+    pub constant_3: ComputedVecsFromHeight<StoredU16>,
+    pub constant_4: ComputedVecsFromHeight<StoredU16>,
+    pub constant_38_2: ComputedVecsFromHeight<StoredF32>,
+    pub constant_50: ComputedVecsFromHeight<StoredU16>,
+    pub constant_61_8: ComputedVecsFromHeight<StoredF32>,
+    pub constant_100: ComputedVecsFromHeight<StoredU16>,
+    pub constant_600: ComputedVecsFromHeight<StoredU16>,
+    pub constant_minus_1: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_2: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_3: ComputedVecsFromHeight<StoredI16>,
+    pub constant_minus_4: ComputedVecsFromHeight<StoredI16>,
+}
+
+impl Vecs {
+    pub fn forced_import(
+        parent_path: &Path,
+        parent_version: Version,
+        indexes: &indexes::Vecs,
+    ) -> Result<Self> {
+        let db = Database::open(&parent_path.join("constants"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let version = parent_version + Version::ZERO;
+
+        let this = Self {
+            constant_0: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_0",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_1: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_1",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_3: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_3",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_4: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_4",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_38_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_38_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_50: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_50",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_61_8: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_61_8",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_100: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_100",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_600: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_600",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_1: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_1",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_2: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_2",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_3: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_3",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+            constant_minus_4: ComputedVecsFromHeight::forced_import(
+                &db,
+                "constant_minus_4",
+                Source::Compute,
+                version + Version::ZERO,
+                indexes,
+                VecBuilderOptions::default().add_last(),
+            )?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexes, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        [
+            (&mut self.constant_0, 0),
+            (&mut self.constant_1, 1),
+            (&mut self.constant_2, 2),
+            (&mut self.constant_3, 3),
+            (&mut self.constant_4, 4),
+            (&mut self.constant_50, 50),
+            (&mut self.constant_100, 100),
+            (&mut self.constant_600, 600),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredU16::new(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        [
+            (&mut self.constant_minus_1, -1),
+            (&mut self.constant_minus_2, -2),
+            (&mut self.constant_minus_3, 3),
+            (&mut self.constant_minus_4, 4),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredI16::new(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        [
+            (&mut self.constant_38_2, 38.2),
+            (&mut self.constant_61_8, 61.8),
+        ]
+        .into_iter()
+        .try_for_each(|(vec, value)| {
+            vec.compute_all(indexes, starting_indexes, exit, |vec| {
+                vec.compute_to(
+                    starting_indexes.height,
+                    indexes.height_to_date.len(),
+                    indexes.height_to_date.version(),
+                    |i| (i, StoredF32::from(value)),
+                    exit,
+                )?;
+                Ok(())
+            })
+        })?;
+
+        Ok(())
+    }
+}

crates/brk_computer/src/fetched.rs

@@ -0,0 +1,137 @@
+use std::path::Path;
+
+use brk_error::Result;
+use brk_fetcher::Fetcher;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{DateIndex, Height, OHLCCents, Version};
+use vecdb::{
+    AnyStoredVec, AnyVec, BytesVec, Database, Exit, GenericStoredVec, ImportableVec, IterableVec,
+    PAGE_SIZE, TypedVecIterator, VecIndex,
+};
+
+use super::{Indexes, indexes, utils::OptionExt};
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+    fetcher: Fetcher,
+
+    pub dateindex_to_price_ohlc_in_cents: BytesVec<DateIndex, OHLCCents>,
+    pub height_to_price_ohlc_in_cents: BytesVec<Height, OHLCCents>,
+}
+
+impl Vecs {
+    pub fn forced_import(parent: &Path, fetcher: Fetcher, version: Version) -> Result<Self> {
+        let db = Database::open(&parent.join("fetched"))?;
+        db.set_min_len(PAGE_SIZE * 1_000_000)?;
+
+        let this = Self {
+            fetcher,
+
+            dateindex_to_price_ohlc_in_cents: BytesVec::forced_import(
+                &db,
+                "price_ohlc_in_cents",
+                version + Version::ZERO,
+            )?,
+            height_to_price_ohlc_in_cents: BytesVec::forced_import(
+                &db,
+                "price_ohlc_in_cents",
+                version + Version::ZERO,
+            )?,
+
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(indexer, indexes, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        let height_to_timestamp = &indexer.vecs.block.height_to_timestamp;
+        let index = starting_indexes
+            .height
+            .min(Height::from(self.height_to_price_ohlc_in_cents.len()));
+        let mut prev_timestamp = index
+            .decremented()
+            .map(|prev_i| height_to_timestamp.iter().unwrap().get_unwrap(prev_i));
+        height_to_timestamp
+            .iter()?
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(i, v)| -> Result<()> {
+                self.height_to_price_ohlc_in_cents.truncate_push_at(
+                    i,
+                    self.fetcher
+                        .get_height(i.into(), v, prev_timestamp)
+                        .unwrap(),
+                )?;
+                prev_timestamp = Some(v);
+                Ok(())
+            })?;
+        self.height_to_price_ohlc_in_cents.safe_write(exit)?;
+
+        let index = starting_indexes
+            .dateindex
+            .min(DateIndex::from(self.dateindex_to_price_ohlc_in_cents.len()));
+        let mut prev = Some(index.decremented().map_or(OHLCCents::default(), |prev_i| {
+            self.dateindex_to_price_ohlc_in_cents
+                .iter()
+                .unwrap()
+                .get_unwrap(prev_i)
+        }));
+        indexes
+            .dateindex_to_date
+            .iter()
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(i, d)| -> Result<()> {
+                let ohlc = if i.to_usize() + 100 >= self.dateindex_to_price_ohlc_in_cents.len()
+                    && let Ok(mut ohlc) = self.fetcher.get_date(d)
+                {
+                    let prev_open = *prev.u().close;
+                    *ohlc.open = prev_open;
+                    *ohlc.high = (*ohlc.high).max(prev_open);
+                    *ohlc.low = (*ohlc.low).min(prev_open);
+                    ohlc
+                } else {
+                    prev.clone().unwrap()
+                };
+
+                prev.replace(ohlc.clone());
+
+                self.dateindex_to_price_ohlc_in_cents
+                    .truncate_push_at(i, ohlc)?;
+
+                Ok(())
+            })?;
+        self.dateindex_to_price_ohlc_in_cents.safe_write(exit)?;
+
+        Ok(())
+    }
+}

crates/brk_computer/src/grouped/builder_eager.rs

@@ -0,0 +1,838 @@
+use brk_error::{Error, Result};
+use brk_traversable::Traversable;
+use brk_types::{CheckedSub, StoredU64, Version};
+use schemars::JsonSchema;
+use vecdb::{
+    AnyStoredVec, Database, EagerVec, Exit, GenericStoredVec, ImportableVec, IterableVec, PcoVec,
+    VecIndex, VecValue,
+};
+
+use crate::utils::{OptionExt, get_percentile};
+
+use super::ComputedVecValue;
+
+const VERSION: Version = Version::ZERO;
+
+#[derive(Clone, Debug, Traversable)]
+pub struct EagerVecsBuilder<I, T>
+where
+    I: VecIndex,
+    T: ComputedVecValue + JsonSchema,
+{
+    pub first: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub average: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub sum: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub max: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub pct90: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub pct75: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub median: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub pct25: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub pct10: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub min: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub last: Option<Box<EagerVec<PcoVec<I, T>>>>,
+    pub cumulative: Option<Box<EagerVec<PcoVec<I, T>>>>,
+}
+
+impl<I, T> EagerVecsBuilder<I, T>
+where
+    I: VecIndex,
+    T: ComputedVecValue + JsonSchema,
+{
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        version: Version,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let only_one_active = options.is_only_one_active();
+        let suffix = |s: &str| format!("{name}_{s}");
+        let maybe_suffix = |s: &str| {
+            if only_one_active {
+                name.to_string()
+            } else {
+                suffix(s)
+            }
+        };
+        let v = version + VERSION;
+
+        macro_rules! import {
+            ($s:expr) => {
+                Box::new(EagerVec::forced_import(db, &maybe_suffix($s), v).unwrap())
+            };
+        }
+
+        let s = Self {
+            first: options.first.then(|| import!("first")),
+            last: options
+                .last
+                .then(|| Box::new(EagerVec::forced_import(db, name, v).unwrap())),
+            min: options.min.then(|| import!("min")),
+            max: options.max.then(|| import!("max")),
+            median: options.median.then(|| import!("median")),
+            average: options.average.then(|| import!("avg")),
+            sum: options.sum.then(|| {
+                let sum_name = if !options.last && !options.average && !options.min && !options.max
+                {
+                    name.to_string()
+                } else {
+                    maybe_suffix("sum")
+                };
+                Box::new(EagerVec::forced_import(db, &sum_name, v).unwrap())
+            }),
+            cumulative: options
+                .cumulative
+                .then(|| Box::new(EagerVec::forced_import(db, &suffix("cumulative"), v).unwrap())),
+            pct90: options.pct90.then(|| import!("pct90")),
+            pct75: options.pct75.then(|| import!("pct75")),
+            pct25: options.pct25.then(|| import!("pct25")),
+            pct10: options.pct10.then(|| import!("pct10")),
+        };
+
+        Ok(s)
+    }
+
+    #[inline]
+    fn needs_percentiles(&self) -> bool {
+        self.pct90.is_some()
+            || self.pct75.is_some()
+            || self.median.is_some()
+            || self.pct25.is_some()
+            || self.pct10.is_some()
+    }
+
+    #[inline]
+    fn needs_minmax(&self) -> bool {
+        self.max.is_some() || self.min.is_some()
+    }
+
+    #[inline]
+    fn needs_sum_or_cumulative(&self) -> bool {
+        self.sum.is_some() || self.cumulative.is_some()
+    }
+
+    #[inline]
+    fn needs_average_sum_or_cumulative(&self) -> bool {
+        self.needs_sum_or_cumulative() || self.average.is_some()
+    }
+
+    /// Compute min/max in O(n) without sorting or collecting
+    #[inline]
+    fn compute_minmax_streaming(
+        &mut self,
+        index: usize,
+        iter: impl Iterator<Item = T>,
+    ) -> Result<()> {
+        let mut min_val: Option<T> = None;
+        let mut max_val: Option<T> = None;
+        let need_min = self.min.is_some();
+        let need_max = self.max.is_some();
+
+        for val in iter {
+            if need_min {
+                min_val = Some(min_val.map_or(val, |m| if val < m { val } else { m }));
+            }
+            if need_max {
+                max_val = Some(max_val.map_or(val, |m| if val > m { val } else { m }));
+            }
+        }
+
+        if let Some(min) = self.min.as_mut() {
+            min.truncate_push_at(index, min_val.unwrap())?;
+        }
+        if let Some(max) = self.max.as_mut() {
+            max.truncate_push_at(index, max_val.unwrap())?;
+        }
+        Ok(())
+    }
+
+    /// Compute min/max from collected values in O(n) without sorting
+    #[inline]
+    fn compute_minmax_from_slice(&mut self, index: usize, values: &[T]) -> Result<()> {
+        if let Some(min) = self.min.as_mut() {
+            min.truncate_push_at(index, *values.iter().min().unwrap())?;
+        }
+        if let Some(max) = self.max.as_mut() {
+            max.truncate_push_at(index, *values.iter().max().unwrap())?;
+        }
+        Ok(())
+    }
+
+    /// Compute percentiles from sorted values (assumes values is already sorted)
+    fn compute_percentiles_from_sorted(&mut self, index: usize, values: &[T]) -> Result<()> {
+        if let Some(max) = self.max.as_mut() {
+            max.truncate_push_at(
+                index,
+                *values
+                    .last()
+                    .ok_or(Error::Internal("Empty values for percentiles"))?,
+            )?;
+        }
+        if let Some(pct90) = self.pct90.as_mut() {
+            pct90.truncate_push_at(index, get_percentile(values, 0.90))?;
+        }
+        if let Some(pct75) = self.pct75.as_mut() {
+            pct75.truncate_push_at(index, get_percentile(values, 0.75))?;
+        }
+        if let Some(median) = self.median.as_mut() {
+            median.truncate_push_at(index, get_percentile(values, 0.50))?;
+        }
+        if let Some(pct25) = self.pct25.as_mut() {
+            pct25.truncate_push_at(index, get_percentile(values, 0.25))?;
+        }
+        if let Some(pct10) = self.pct10.as_mut() {
+            pct10.truncate_push_at(index, get_percentile(values, 0.10))?;
+        }
+        if let Some(min) = self.min.as_mut() {
+            min.truncate_push_at(index, *values.first().unwrap())?;
+        }
+        Ok(())
+    }
+
+    /// Compute sum, average, and cumulative from values
+    fn compute_aggregates(
+        &mut self,
+        index: usize,
+        values: Vec<T>,
+        cumulative: &mut Option<T>,
+    ) -> Result<()> {
+        let len = values.len();
+        let sum = values.into_iter().fold(T::from(0), |a, b| a + b);
+
+        if let Some(average) = self.average.as_mut() {
+            average.truncate_push_at(index, sum / len)?;
+        }
+
+        if self.needs_sum_or_cumulative() {
+            if let Some(sum_vec) = self.sum.as_mut() {
+                sum_vec.truncate_push_at(index, sum)?;
+            }
+            if let Some(cumulative_vec) = self.cumulative.as_mut() {
+                let t = cumulative.unwrap() + sum;
+                cumulative.replace(t);
+                cumulative_vec.truncate_push_at(index, t)?;
+            }
+        }
+        Ok(())
+    }
+
+    pub fn extend(
+        &mut self,
+        max_from: I,
+        source: &impl IterableVec<I, T>,
+        exit: &Exit,
+    ) -> Result<()> {
+        if self.cumulative.is_none() {
+            return Ok(());
+        };
+
+        self.validate_computed_version_or_reset(source.version())?;
+
+        let index = self.starting_index(max_from);
+
+        let cumulative_vec = self.cumulative.um();
+
+        let mut cumulative = index.decremented().map_or(T::from(0_usize), |index| {
+            cumulative_vec.iter().get_unwrap(index)
+        });
+        source
+            .iter()
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(i, v)| -> Result<()> {
+                cumulative += v;
+                cumulative_vec.truncate_push_at(i, cumulative)?;
+                Ok(())
+            })?;
+
+        self.safe_write(exit)?;
+
+        Ok(())
+    }
+
+    pub fn compute<A>(
+        &mut self,
+        max_from: I,
+        source: &impl IterableVec<A, T>,
+        first_indexes: &impl IterableVec<I, A>,
+        count_indexes: &impl IterableVec<I, StoredU64>,
+        exit: &Exit,
+    ) -> Result<()>
+    where
+        A: VecIndex + VecValue + CheckedSub<A>,
+    {
+        self.validate_computed_version_or_reset(
+            source.version() + first_indexes.version() + count_indexes.version(),
+        )?;
+
+        let index = self.starting_index(max_from);
+
+        let mut source_iter = source.iter();
+
+        let cumulative_vec = self.cumulative.as_mut();
+
+        let mut cumulative = cumulative_vec.map(|cumulative_vec| {
+            index.decremented().map_or(T::from(0_usize), |index| {
+                cumulative_vec.iter().get_unwrap(index)
+            })
+        });
+
+        let mut count_indexes_iter = count_indexes.iter().skip(index.to_usize());
+
+        first_indexes
+            .iter()
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(index, first_index)| -> Result<()> {
+                let count_index = count_indexes_iter.next().unwrap();
+
+                if let Some(first) = self.first.as_mut() {
+                    let f = source_iter
+                        .get(first_index)
+                        .unwrap_or_else(|| T::from(0_usize));
+                    first.truncate_push_at(index, f)?;
+                }
+
+                if let Some(last) = self.last.as_mut() {
+                    let count_index = *count_index as usize;
+                    if count_index == 0 {
+                        panic!("should compute last if count can be 0")
+                    }
+                    let last_index = first_index + (count_index - 1);
+                    let v = source_iter.get_unwrap(last_index);
+                    // .context("to work")
+                    // .inspect_err(|_| {
+                    //     dbg!(first_index, count_index, last_index);
+                    // })
+                    // .unwrap()
+                    // ;
+                    last.truncate_push_at(index, v)?;
+                }
+
+                let needs_percentiles = self.needs_percentiles();
+                let needs_minmax = self.needs_minmax();
+                let needs_aggregates = self.needs_average_sum_or_cumulative();
+
+                // Fast path: only min/max needed, no sorting or allocation required
+                if needs_minmax && !needs_percentiles && !needs_aggregates {
+                    source_iter.set_position(first_index);
+                    self.compute_minmax_streaming(
+                        index,
+                        (&mut source_iter).take(*count_index as usize),
+                    )?;
+                } else if needs_percentiles || needs_aggregates {
+                    source_iter.set_position(first_index);
+                    let mut values = (&mut source_iter)
+                        .take(*count_index as usize)
+                        .collect::<Vec<_>>();
+
+                    if needs_percentiles {
+                        values.sort_unstable();
+                        self.compute_percentiles_from_sorted(index, &values)?;
+                    } else if needs_minmax {
+                        // We have values collected but only need min/max (along with aggregates)
+                        self.compute_minmax_from_slice(index, &values)?;
+                    }
+
+                    if needs_aggregates {
+                        self.compute_aggregates(index, values, &mut cumulative)?;
+                    }
+                }
+
+                Ok(())
+            })?;
+
+        self.safe_write(exit)?;
+
+        Ok(())
+    }
+
+    #[allow(clippy::wrong_self_convention)]
+    pub fn from_aligned<A>(
+        &mut self,
+        max_from: I,
+        source: &EagerVecsBuilder<A, T>,
+        first_indexes: &impl IterableVec<I, A>,
+        count_indexes: &impl IterableVec<I, StoredU64>,
+        exit: &Exit,
+    ) -> Result<()>
+    where
+        A: VecIndex + VecValue + CheckedSub<A>,
+    {
+        if self.needs_percentiles() {
+            panic!("percentiles unsupported in from_aligned");
+        }
+
+        self.validate_computed_version_or_reset(
+            VERSION + first_indexes.version() + count_indexes.version(),
+        )?;
+
+        let index = self.starting_index(max_from);
+
+        let mut source_first_iter = source.first.as_ref().map(|f| f.iter());
+        let mut source_last_iter = source.last.as_ref().map(|f| f.iter());
+        let mut source_max_iter = source.max.as_ref().map(|f| f.iter());
+        let mut source_min_iter = source.min.as_ref().map(|f| f.iter());
+        let mut source_average_iter = source.average.as_ref().map(|f| f.iter());
+        let mut source_sum_iter = source.sum.as_ref().map(|f| f.iter());
+
+        let mut cumulative = self.cumulative.as_mut().map(|cumulative_vec| {
+            index.decremented().map_or(T::from(0_usize), |index| {
+                cumulative_vec.iter().get_unwrap(index)
+            })
+        });
+
+        let mut count_indexes_iter = count_indexes.iter().skip(index.to_usize());
+
+        first_indexes
+            .iter()
+            .enumerate()
+            .skip(index.to_usize())
+            .try_for_each(|(index, first_index, ..)| -> Result<()> {
+                let count_index = count_indexes_iter.next().unwrap();
+
+                if let Some(first) = self.first.as_mut() {
+                    let v = source_first_iter.um().get_unwrap(first_index);
+                    first.truncate_push_at(index, v)?;
+                }
+
+                if let Some(last) = self.last.as_mut() {
+                    let count_index = *count_index as usize;
+                    if count_index == 0 {
+                        panic!("should compute last if count can be 0")
+                    }
+                    let last_index = first_index + (count_index - 1);
+                    let v = source_last_iter.um().get_unwrap(last_index);
+                    last.truncate_push_at(index, v)?;
+                }
+
+                let needs_minmax = self.needs_minmax();
+                let needs_aggregates = self.needs_average_sum_or_cumulative();
+
+                if needs_minmax || needs_aggregates {
+                    // Min/max: use streaming O(n) instead of sort O(n log n)
+                    if needs_minmax {
+                        if let Some(max) = self.max.as_mut() {
+                            let source_max_iter = source_max_iter.um();
+                            source_max_iter.set_position(first_index);
+                            let max_val =
+                                source_max_iter.take(*count_index as usize).max().unwrap();
+                            max.truncate_push_at(index, max_val)?;
+                        }
+
+                        if let Some(min) = self.min.as_mut() {
+                            let source_min_iter = source_min_iter.um();
+                            source_min_iter.set_position(first_index);
+                            let min_val =
+                                source_min_iter.take(*count_index as usize).min().unwrap();
+                            min.truncate_push_at(index, min_val)?;
+                        }
+                    }
+
+                    if needs_aggregates {
+                        if let Some(average) = self.average.as_mut() {
+                            let source_average_iter = source_average_iter.um();
+                            source_average_iter.set_position(first_index);
+                            let mut len = 0usize;
+                            let sum = (&mut *source_average_iter)
+                                .take(*count_index as usize)
+                                .inspect(|_| len += 1)
+                                .fold(T::from(0), |a, b| a + b);
+                            // TODO: Multiply by count then divide by cumulative
+                            // Right now it's not 100% accurate as there could be more or less elements in the lower timeframe (28 days vs 31 days in a month for example)
+                            let avg = sum / len;
+                            average.truncate_push_at(index, avg)?;
+                        }
+
+                        if self.needs_sum_or_cumulative() {
+                            let source_sum_iter = source_sum_iter.um();
+                            source_sum_iter.set_position(first_index);
+                            let sum = source_sum_iter
+                                .take(*count_index as usize)
+                                .fold(T::from(0), |a, b| a + b);
+
+                            if let Some(sum_vec) = self.sum.as_mut() {
+                                sum_vec.truncate_push_at(index, sum)?;
+                            }
+
+                            if let Some(cumulative_vec) = self.cumulative.as_mut() {
+                                let t = cumulative.unwrap() + sum;
+                                cumulative.replace(t);
+                                cumulative_vec.truncate_push_at(index, t)?;
+                            }
+                        }
+                    }
+                }
+
+                Ok(())
+            })?;
+
+        self.safe_write(exit)?;
+
+        Ok(())
+    }
+
+    pub fn starting_index(&self, max_from: I) -> I {
+        max_from.min(I::from(
+            self.iter_any_exportable().map(|v| v.len()).min().unwrap(),
+        ))
+    }
+
+    #[inline]
+    pub fn unwrap_first(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.first.u()
+    }
+    #[inline]
+    pub fn unwrap_average(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.average.u()
+    }
+    #[inline]
+    pub fn unwrap_sum(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.sum.u()
+    }
+    #[inline]
+    pub fn unwrap_max(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.max.u()
+    }
+    #[inline]
+    pub fn unwrap_pct90(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.pct90.u()
+    }
+    #[inline]
+    pub fn unwrap_pct75(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.pct75.u()
+    }
+    #[inline]
+    pub fn unwrap_median(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.median.u()
+    }
+    #[inline]
+    pub fn unwrap_pct25(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.pct25.u()
+    }
+    #[inline]
+    pub fn unwrap_pct10(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.pct10.u()
+    }
+    #[inline]
+    pub fn unwrap_min(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.min.u()
+    }
+    #[inline]
+    pub fn unwrap_last(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.last.u()
+    }
+    #[inline]
+    pub fn unwrap_cumulative(&self) -> &EagerVec<PcoVec<I, T>> {
+        self.cumulative.u()
+    }
+
+    pub fn safe_write(&mut self, exit: &Exit) -> Result<()> {
+        if let Some(first) = self.first.as_mut() {
+            first.safe_write(exit)?;
+        }
+        if let Some(last) = self.last.as_mut() {
+            last.safe_write(exit)?;
+        }
+        if let Some(min) = self.min.as_mut() {
+            min.safe_write(exit)?;
+        }
+        if let Some(max) = self.max.as_mut() {
+            max.safe_write(exit)?;
+        }
+        if let Some(median) = self.median.as_mut() {
+            median.safe_write(exit)?;
+        }
+        if let Some(average) = self.average.as_mut() {
+            average.safe_write(exit)?;
+        }
+        if let Some(sum) = self.sum.as_mut() {
+            sum.safe_write(exit)?;
+        }
+        if let Some(cumulative) = self.cumulative.as_mut() {
+            cumulative.safe_write(exit)?;
+        }
+        if let Some(pct90) = self.pct90.as_mut() {
+            pct90.safe_write(exit)?;
+        }
+        if let Some(pct75) = self.pct75.as_mut() {
+            pct75.safe_write(exit)?;
+        }
+        if let Some(pct25) = self.pct25.as_mut() {
+            pct25.safe_write(exit)?;
+        }
+        if let Some(pct10) = self.pct10.as_mut() {
+            pct10.safe_write(exit)?;
+        }
+
+        Ok(())
+    }
+
+    pub fn validate_computed_version_or_reset(&mut self, version: Version) -> Result<()> {
+        if let Some(first) = self.first.as_mut() {
+            first.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(last) = self.last.as_mut() {
+            last.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(min) = self.min.as_mut() {
+            min.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(max) = self.max.as_mut() {
+            max.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(median) = self.median.as_mut() {
+            median.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(average) = self.average.as_mut() {
+            average.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(sum) = self.sum.as_mut() {
+            sum.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(cumulative) = self.cumulative.as_mut() {
+            cumulative.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(pct90) = self.pct90.as_mut() {
+            pct90.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(pct75) = self.pct75.as_mut() {
+            pct75.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(pct25) = self.pct25.as_mut() {
+            pct25.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+        if let Some(pct10) = self.pct10.as_mut() {
+            pct10.validate_computed_version_or_reset(Version::ZERO + version)?;
+        }
+
+        Ok(())
+    }
+}
+
+#[derive(Default, Clone, Copy)]
+pub struct VecBuilderOptions {
+    average: bool,
+    sum: bool,
+    max: bool,
+    pct90: bool,
+    pct75: bool,
+    median: bool,
+    pct25: bool,
+    pct10: bool,
+    min: bool,
+    first: bool,
+    last: bool,
+    cumulative: bool,
+}
+
+impl VecBuilderOptions {
+    pub fn average(&self) -> bool {
+        self.average
+    }
+
+    pub fn sum(&self) -> bool {
+        self.sum
+    }
+
+    pub fn max(&self) -> bool {
+        self.max
+    }
+
+    pub fn pct90(&self) -> bool {
+        self.pct90
+    }
+
+    pub fn pct75(&self) -> bool {
+        self.pct75
+    }
+
+    pub fn median(&self) -> bool {
+        self.median
+    }
+
+    pub fn pct25(&self) -> bool {
+        self.pct25
+    }
+
+    pub fn pct10(&self) -> bool {
+        self.pct10
+    }
+
+    pub fn min(&self) -> bool {
+        self.min
+    }
+
+    pub fn first(&self) -> bool {
+        self.first
+    }
+
+    pub fn last(&self) -> bool {
+        self.last
+    }
+
+    pub fn cumulative(&self) -> bool {
+        self.cumulative
+    }
+
+    pub fn add_first(mut self) -> Self {
+        self.first = true;
+        self
+    }
+
+    pub fn add_last(mut self) -> Self {
+        self.last = true;
+        self
+    }
+
+    pub fn add_min(mut self) -> Self {
+        self.min = true;
+        self
+    }
+
+    pub fn add_max(mut self) -> Self {
+        self.max = true;
+        self
+    }
+
+    pub fn add_median(mut self) -> Self {
+        self.median = true;
+        self
+    }
+
+    pub fn add_average(mut self) -> Self {
+        self.average = true;
+        self
+    }
+
+    pub fn add_sum(mut self) -> Self {
+        self.sum = true;
+        self
+    }
+
+    pub fn add_pct90(mut self) -> Self {
+        self.pct90 = true;
+        self
+    }
+
+    pub fn add_pct75(mut self) -> Self {
+        self.pct75 = true;
+        self
+    }
+
+    pub fn add_pct25(mut self) -> Self {
+        self.pct25 = true;
+        self
+    }
+
+    pub fn add_pct10(mut self) -> Self {
+        self.pct10 = true;
+        self
+    }
+
+    pub fn add_cumulative(mut self) -> Self {
+        self.cumulative = true;
+        self
+    }
+
+    pub fn rm_min(mut self) -> Self {
+        self.min = false;
+        self
+    }
+
+    pub fn rm_max(mut self) -> Self {
+        self.max = false;
+        self
+    }
+
+    pub fn rm_median(mut self) -> Self {
+        self.median = false;
+        self
+    }
+
+    pub fn rm_average(mut self) -> Self {
+        self.average = false;
+        self
+    }
+
+    pub fn rm_sum(mut self) -> Self {
+        self.sum = false;
+        self
+    }
+
+    pub fn rm_pct90(mut self) -> Self {
+        self.pct90 = false;
+        self
+    }
+
+    pub fn rm_pct75(mut self) -> Self {
+        self.pct75 = false;
+        self
+    }
+
+    pub fn rm_pct25(mut self) -> Self {
+        self.pct25 = false;
+        self
+    }
+
+    pub fn rm_pct10(mut self) -> Self {
+        self.pct10 = false;
+        self
+    }
+
+    pub fn rm_cumulative(mut self) -> Self {
+        self.cumulative = false;
+        self
+    }
+
+    pub fn add_minmax(mut self) -> Self {
+        self.min = true;
+        self.max = true;
+        self
+    }
+
+    pub fn add_percentiles(mut self) -> Self {
+        self.pct90 = true;
+        self.pct75 = true;
+        self.median = true;
+        self.pct25 = true;
+        self.pct10 = true;
+        self
+    }
+
+    pub fn remove_percentiles(mut self) -> Self {
+        self.pct90 = false;
+        self.pct75 = false;
+        self.median = false;
+        self.pct25 = false;
+        self.pct10 = false;
+        self
+    }
+
+    pub fn is_only_one_active(&self) -> bool {
+        [
+            self.average,
+            self.sum,
+            self.max,
+            self.pct90,
+            self.pct75,
+            self.median,
+            self.pct25,
+            self.pct10,
+            self.min,
+            self.first,
+            self.last,
+            self.cumulative,
+        ]
+        .iter()
+        .filter(|b| **b)
+        .count()
+            == 1
+    }
+
+    pub fn copy_self_extra(&self) -> Self {
+        Self {
+            cumulative: self.cumulative,
+            ..Self::default()
+        }
+    }
+}

crates/brk_computer/src/grouped/builder_lazy.rs

@@ -0,0 +1,361 @@
+use brk_traversable::Traversable;
+use brk_types::Version;
+use schemars::JsonSchema;
+use vecdb::{FromCoarserIndex, IterableBoxedVec, IterableCloneableVec, LazyVecFrom2, VecIndex};
+
+use crate::grouped::{EagerVecsBuilder, VecBuilderOptions};
+use crate::utils::OptionExt;
+
+use super::ComputedVecValue;
+
+#[allow(clippy::type_complexity)]
+#[derive(Clone, Traversable)]
+pub struct LazyVecsBuilder<I, T, S1I, S2T>
+where
+    I: VecIndex,
+    T: ComputedVecValue + JsonSchema,
+    S1I: VecIndex,
+    S2T: ComputedVecValue,
+{
+    pub first: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub average: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub sum: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub max: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub min: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub last: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+    pub cumulative: Option<Box<LazyVecFrom2<I, T, S1I, T, I, S2T>>>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl<I, T, S1I, S2T> LazyVecsBuilder<I, T, S1I, S2T>
+where
+    I: VecIndex,
+    T: ComputedVecValue + JsonSchema + 'static,
+    S1I: VecIndex + 'static + FromCoarserIndex<I>,
+    S2T: ComputedVecValue,
+{
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        name: &str,
+        version: Version,
+        source: Option<IterableBoxedVec<S1I, T>>,
+        source_extra: &EagerVecsBuilder<S1I, T>,
+        len_source: IterableBoxedVec<I, S2T>,
+        options: LazyVecBuilderOptions,
+    ) -> Self {
+        let only_one_active = options.is_only_one_active();
+
+        let suffix = |s: &str| format!("{name}_{s}");
+
+        let maybe_suffix = |s: &str| {
+            if only_one_active {
+                name.to_string()
+            } else {
+                suffix(s)
+            }
+        };
+
+        Self {
+            first: options.first.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &maybe_suffix("first"),
+                    version + VERSION + Version::ZERO,
+                    source_extra
+                        .first
+                        .as_ref()
+                        .map_or_else(|| source.u().clone(), |v| v.clone()),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        source.get_at(S1I::min_from(i))
+                    },
+                ))
+            }),
+            last: options.last.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    name,
+                    version + VERSION + Version::ZERO,
+                    source_extra.last.as_ref().map_or_else(
+                        || {
+                            source
+                                .as_ref()
+                                .unwrap_or_else(|| {
+                                    dbg!(name, I::to_string());
+                                    panic!()
+                                })
+                                .clone()
+                        },
+                        |v| v.clone(),
+                    ),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        source.get_at(S1I::max_from(i, source.vec_len()))
+                    },
+                ))
+            }),
+            min: options.min.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &maybe_suffix("min"),
+                    version + VERSION + Version::ZERO,
+                    source_extra
+                        .min
+                        .as_ref()
+                        .map_or_else(|| source.u().clone(), |v| v.clone()),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        S1I::inclusive_range_from(i, source.vec_len())
+                            .flat_map(|i| source.get_at(i))
+                            .min()
+                    },
+                ))
+            }),
+            max: options.max.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &maybe_suffix("max"),
+                    version + VERSION + Version::ZERO,
+                    source_extra
+                        .max
+                        .as_ref()
+                        .map_or_else(|| source.u().clone(), |v| v.clone()),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        S1I::inclusive_range_from(i, source.vec_len())
+                            .flat_map(|i| source.get_at(i))
+                            .max()
+                    },
+                ))
+            }),
+            average: options.average.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &maybe_suffix("avg"),
+                    version + VERSION + Version::ZERO,
+                    source_extra
+                        .average
+                        .as_ref()
+                        .map_or_else(|| source.u().clone(), |v| v.clone()),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        let mut sum = T::from(0);
+                        let mut len = 0usize;
+                        for v in S1I::inclusive_range_from(i, source.vec_len())
+                            .flat_map(|i| source.get_at(i))
+                        {
+                            sum += v;
+                            len += 1;
+                        }
+                        if len == 0 {
+                            return None;
+                        }
+                        Some(sum / len)
+                    },
+                ))
+            }),
+            sum: options.sum.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &(if !options.last && !options.average && !options.min && !options.max {
+                        name.to_string()
+                    } else {
+                        maybe_suffix("sum")
+                    }),
+                    version + VERSION + Version::ZERO,
+                    source_extra
+                        .sum
+                        .as_ref()
+                        .map_or_else(|| source.u().clone(), |v| v.clone()),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        let mut sum = T::from(0);
+                        let mut has_values = false;
+                        for v in S1I::inclusive_range_from(i, source.vec_len())
+                            .flat_map(|i| source.get_at(i))
+                        {
+                            sum += v;
+                            has_values = true;
+                        }
+                        if !has_values {
+                            return None;
+                        }
+                        Some(sum)
+                    },
+                ))
+            }),
+            cumulative: options.cumulative.then(|| {
+                Box::new(LazyVecFrom2::init(
+                    &suffix("cumulative"),
+                    version + VERSION + Version::ZERO,
+                    source_extra.cumulative.u().boxed_clone(),
+                    len_source.clone(),
+                    |i: I, source, len_source| {
+                        if i.to_usize() >= len_source.vec_len() {
+                            return None;
+                        }
+                        source.get_at(S1I::max_from(i, source.vec_len()))
+                    },
+                ))
+            }),
+        }
+    }
+
+    pub fn starting_index(&self, max_from: I) -> I {
+        max_from.min(I::from(
+            self.iter_any_exportable().map(|v| v.len()).min().unwrap(),
+        ))
+    }
+
+    pub fn unwrap_first(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.first.u()
+    }
+    pub fn unwrap_average(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.average.u()
+    }
+    pub fn unwrap_sum(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.sum.u()
+    }
+    pub fn unwrap_max(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.max.u()
+    }
+    pub fn unwrap_min(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.min.u()
+    }
+    pub fn unwrap_last(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.last.u()
+    }
+    pub fn unwrap_cumulative(&self) -> &LazyVecFrom2<I, T, S1I, T, I, S2T> {
+        self.cumulative.u()
+    }
+}
+
+#[derive(Default, Clone, Copy)]
+pub struct LazyVecBuilderOptions {
+    average: bool,
+    sum: bool,
+    max: bool,
+    min: bool,
+    first: bool,
+    last: bool,
+    cumulative: bool,
+}
+
+impl From<VecBuilderOptions> for LazyVecBuilderOptions {
+    #[inline]
+    fn from(value: VecBuilderOptions) -> Self {
+        Self {
+            average: value.average(),
+            sum: value.sum(),
+            max: value.max(),
+            min: value.min(),
+            first: value.first(),
+            last: value.last(),
+            cumulative: value.cumulative(),
+        }
+    }
+}
+
+impl LazyVecBuilderOptions {
+    pub fn add_first(mut self) -> Self {
+        self.first = true;
+        self
+    }
+
+    pub fn add_last(mut self) -> Self {
+        self.last = true;
+        self
+    }
+
+    pub fn add_min(mut self) -> Self {
+        self.min = true;
+        self
+    }
+
+    pub fn add_max(mut self) -> Self {
+        self.max = true;
+        self
+    }
+
+    pub fn add_average(mut self) -> Self {
+        self.average = true;
+        self
+    }
+
+    pub fn add_sum(mut self) -> Self {
+        self.sum = true;
+        self
+    }
+
+    pub fn add_cumulative(mut self) -> Self {
+        self.cumulative = true;
+        self
+    }
+
+    pub fn rm_min(mut self) -> Self {
+        self.min = false;
+        self
+    }
+
+    pub fn rm_max(mut self) -> Self {
+        self.max = false;
+        self
+    }
+
+    pub fn rm_average(mut self) -> Self {
+        self.average = false;
+        self
+    }
+
+    pub fn rm_sum(mut self) -> Self {
+        self.sum = false;
+        self
+    }
+
+    pub fn rm_cumulative(mut self) -> Self {
+        self.cumulative = false;
+        self
+    }
+
+    pub fn add_minmax(mut self) -> Self {
+        self.min = true;
+        self.max = true;
+        self
+    }
+
+    pub fn is_only_one_active(&self) -> bool {
+        [
+            self.average,
+            self.sum,
+            self.max,
+            self.min,
+            self.first,
+            self.last,
+            self.cumulative,
+        ]
+        .iter()
+        .filter(|b| **b)
+        .count()
+            == 1
+    }
+
+    pub fn copy_self_extra(&self) -> Self {
+        Self {
+            cumulative: self.cumulative,
+            ..Self::default()
+        }
+    }
+}

crates/brk_computer/src/grouped/computed.rs

@@ -0,0 +1,28 @@
+use std::ops::{Add, AddAssign, Div};
+
+use serde::Serialize;
+use vecdb::{Formattable, PcoVecValue};
+
+pub trait ComputedVecValue
+where
+    Self: PcoVecValue
+        + From<usize>
+        + Div<usize, Output = Self>
+        + Add<Output = Self>
+        + AddAssign
+        + Ord
+        + Formattable
+        + Serialize,
+{
+}
+impl<T> ComputedVecValue for T where
+    T: PcoVecValue
+        + From<usize>
+        + Div<usize, Output = Self>
+        + Add<Output = Self>
+        + AddAssign
+        + Ord
+        + Formattable
+        + Serialize
+{
+}

crates/brk_computer/src/grouped/from_dateindex.rs

@@ -0,0 +1,199 @@
+use brk_error::Result;
+
+use brk_traversable::Traversable;
+use brk_types::{
+    DateIndex, DecadeIndex, MonthIndex, QuarterIndex, SemesterIndex, Version, WeekIndex, YearIndex,
+};
+use schemars::JsonSchema;
+use vecdb::{
+    AnyExportableVec, Database, EagerVec, Exit, ImportableVec, IterableCloneableVec, IterableVec,
+    PcoVec,
+};
+
+use crate::{Indexes, grouped::LazyVecsBuilder, indexes, utils::OptionExt};
+
+use super::{ComputedVecValue, EagerVecsBuilder, Source, VecBuilderOptions};
+
+#[derive(Clone)]
+pub struct ComputedVecsFromDateIndex<T>
+where
+    T: ComputedVecValue + PartialOrd + JsonSchema,
+{
+    pub dateindex: Option<EagerVec<PcoVec<DateIndex, T>>>,
+    pub dateindex_extra: EagerVecsBuilder<DateIndex, T>,
+    pub weekindex: LazyVecsBuilder<WeekIndex, T, DateIndex, WeekIndex>,
+    pub monthindex: LazyVecsBuilder<MonthIndex, T, DateIndex, MonthIndex>,
+    pub quarterindex: LazyVecsBuilder<QuarterIndex, T, DateIndex, QuarterIndex>,
+    pub semesterindex: LazyVecsBuilder<SemesterIndex, T, DateIndex, SemesterIndex>,
+    pub yearindex: LazyVecsBuilder<YearIndex, T, DateIndex, YearIndex>,
+    pub decadeindex: LazyVecsBuilder<DecadeIndex, T, DateIndex, DecadeIndex>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl<T> ComputedVecsFromDateIndex<T>
+where
+    T: ComputedVecValue + JsonSchema + 'static,
+{
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<DateIndex, T>,
+        version: Version,
+        indexes: &indexes::Vecs,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let dateindex = source
+            .is_compute()
+            .then(|| EagerVec::forced_import(db, name, version + VERSION + Version::ZERO).unwrap());
+
+        let dateindex_extra = EagerVecsBuilder::forced_import(
+            db,
+            name,
+            version + VERSION + Version::ZERO,
+            options.copy_self_extra(),
+        )?;
+
+        let options = options.remove_percentiles();
+
+        let dateindex_source = source.vec().or(dateindex.as_ref().map(|v| v.boxed_clone()));
+
+        Ok(Self {
+            weekindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.weekindex_to_weekindex.boxed_clone(),
+                options.into(),
+            ),
+            monthindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.monthindex_to_monthindex.boxed_clone(),
+                options.into(),
+            ),
+            quarterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.quarterindex_to_quarterindex.boxed_clone(),
+                options.into(),
+            ),
+            semesterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.semesterindex_to_semesterindex.boxed_clone(),
+                options.into(),
+            ),
+            yearindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.yearindex_to_yearindex.boxed_clone(),
+                options.into(),
+            ),
+            decadeindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                dateindex_source.clone(),
+                &dateindex_extra,
+                indexes.decadeindex_to_decadeindex.boxed_clone(),
+                options.into(),
+            ),
+            dateindex,
+            dateindex_extra,
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<DateIndex, T>>) -> Result<()>,
+    {
+        compute(self.dateindex.um())?;
+
+        let dateindex: Option<&EagerVec<PcoVec<DateIndex, T>>> = None;
+        self.compute_rest(starting_indexes, exit, dateindex)
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        dateindex: Option<&impl IterableVec<DateIndex, T>>,
+    ) -> Result<()> {
+        if let Some(dateindex) = dateindex {
+            self.dateindex_extra
+                .extend(starting_indexes.dateindex, dateindex, exit)?;
+        } else {
+            let dateindex = self.dateindex.u();
+
+            self.dateindex_extra
+                .extend(starting_indexes.dateindex, dateindex, exit)?;
+        }
+
+        Ok(())
+    }
+}
+
+impl<T> Traversable for ComputedVecsFromDateIndex<T>
+where
+    T: ComputedVecValue + JsonSchema,
+{
+    fn to_tree_node(&self) -> brk_traversable::TreeNode {
+        let dateindex_extra_node = self.dateindex_extra.to_tree_node();
+        brk_traversable::TreeNode::Branch(
+            [
+                self.dateindex
+                    .as_ref()
+                    .map(|nested| ("dateindex".to_string(), nested.to_tree_node())),
+                if dateindex_extra_node.is_empty() {
+                    None
+                } else {
+                    Some(("dateindex_extra".to_string(), dateindex_extra_node))
+                },
+                Some(("weekindex".to_string(), self.weekindex.to_tree_node())),
+                Some(("monthindex".to_string(), self.monthindex.to_tree_node())),
+                Some(("quarterindex".to_string(), self.quarterindex.to_tree_node())),
+                Some((
+                    "semesterindex".to_string(),
+                    self.semesterindex.to_tree_node(),
+                )),
+                Some(("yearindex".to_string(), self.yearindex.to_tree_node())),
+                Some(("decadeindex".to_string(), self.decadeindex.to_tree_node())),
+            ]
+            .into_iter()
+            .flatten()
+            .collect(),
+        )
+        .merge_branches()
+        .unwrap()
+    }
+
+    fn iter_any_exportable(&self) -> impl Iterator<Item = &dyn AnyExportableVec> {
+        let mut regular_iter: Box<dyn Iterator<Item = &dyn AnyExportableVec>> =
+            Box::new(self.dateindex_extra.iter_any_exportable());
+        regular_iter = Box::new(regular_iter.chain(self.weekindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.monthindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.quarterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.semesterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.yearindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.decadeindex.iter_any_exportable()));
+        if let Some(ref x) = self.dateindex {
+            regular_iter = Box::new(regular_iter.chain(x.iter_any_exportable()));
+        }
+        regular_iter
+    }
+}

crates/brk_computer/src/grouped/from_height.rs

@@ -0,0 +1,259 @@
+use brk_error::Result;
+
+use brk_traversable::Traversable;
+use brk_types::{
+    DateIndex, DecadeIndex, DifficultyEpoch, Height, MonthIndex, QuarterIndex, SemesterIndex,
+    Version, WeekIndex, YearIndex,
+};
+use schemars::JsonSchema;
+use vecdb::{
+    AnyExportableVec, Database, EagerVec, Exit, ImportableVec, IterableCloneableVec, IterableVec,
+    PcoVec,
+};
+
+use crate::{
+    Indexes,
+    grouped::{LazyVecsBuilder, Source},
+    indexes,
+    utils::OptionExt,
+};
+
+use super::{ComputedVecValue, EagerVecsBuilder, VecBuilderOptions};
+
+#[derive(Clone)]
+pub struct ComputedVecsFromHeight<T>
+where
+    T: ComputedVecValue + PartialOrd + JsonSchema,
+{
+    pub height: Option<EagerVec<PcoVec<Height, T>>>,
+    pub height_extra: EagerVecsBuilder<Height, T>,
+    pub dateindex: EagerVecsBuilder<DateIndex, T>,
+    pub weekindex: LazyVecsBuilder<WeekIndex, T, DateIndex, WeekIndex>,
+    pub difficultyepoch: EagerVecsBuilder<DifficultyEpoch, T>,
+    pub monthindex: LazyVecsBuilder<MonthIndex, T, DateIndex, MonthIndex>,
+    pub quarterindex: LazyVecsBuilder<QuarterIndex, T, DateIndex, QuarterIndex>,
+    pub semesterindex: LazyVecsBuilder<SemesterIndex, T, DateIndex, SemesterIndex>,
+    pub yearindex: LazyVecsBuilder<YearIndex, T, DateIndex, YearIndex>,
+    // TODO: pub halvingepoch: StorableVecGeneator<Halvingepoch, T>,
+    pub decadeindex: LazyVecsBuilder<DecadeIndex, T, DateIndex, DecadeIndex>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl<T> ComputedVecsFromHeight<T>
+where
+    T: ComputedVecValue + Ord + From<f64> + JsonSchema + 'static,
+    f64: From<T>,
+{
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<Height, T>,
+        version: Version,
+        indexes: &indexes::Vecs,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let height = source
+            .is_compute()
+            .then(|| EagerVec::forced_import(db, name, version + VERSION + Version::ZERO).unwrap());
+
+        let height_extra = EagerVecsBuilder::forced_import(
+            db,
+            name,
+            version + VERSION + Version::ZERO,
+            options.copy_self_extra(),
+        )?;
+
+        let dateindex =
+            EagerVecsBuilder::forced_import(db, name, version + VERSION + Version::ZERO, options)?;
+
+        let options = options.remove_percentiles();
+
+        Ok(Self {
+            weekindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.weekindex_to_weekindex.boxed_clone(),
+                options.into(),
+            ),
+            monthindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.monthindex_to_monthindex.boxed_clone(),
+                options.into(),
+            ),
+            quarterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.quarterindex_to_quarterindex.boxed_clone(),
+                options.into(),
+            ),
+            semesterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.semesterindex_to_semesterindex.boxed_clone(),
+                options.into(),
+            ),
+            yearindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.yearindex_to_yearindex.boxed_clone(),
+                options.into(),
+            ),
+            decadeindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.decadeindex_to_decadeindex.boxed_clone(),
+                options.into(),
+            ),
+            // halvingepoch: StorableVecGeneator::forced_import(db, name, version + VERSION + Version::ZERO, format, options)?,
+            height,
+            height_extra,
+            dateindex,
+            difficultyepoch: EagerVecsBuilder::forced_import(
+                db,
+                name,
+                version + VERSION + Version::ZERO,
+                options,
+            )?,
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<Height, T>>) -> Result<()>,
+    {
+        compute(self.height.um())?;
+
+        let height: Option<&EagerVec<PcoVec<Height, T>>> = None;
+        self.compute_rest(indexes, starting_indexes, exit, height)
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        height_vec: Option<&impl IterableVec<Height, T>>,
+    ) -> Result<()> {
+        if let Some(height) = height_vec {
+            self.height_extra
+                .extend(starting_indexes.height, height, exit)?;
+
+            self.dateindex.compute(
+                starting_indexes.dateindex,
+                height,
+                &indexes.dateindex_to_first_height,
+                &indexes.dateindex_to_height_count,
+                exit,
+            )?;
+
+            self.difficultyepoch.compute(
+                starting_indexes.difficultyepoch,
+                height,
+                &indexes.difficultyepoch_to_first_height,
+                &indexes.difficultyepoch_to_height_count,
+                exit,
+            )?;
+        } else {
+            let height = self.height.u();
+
+            self.height_extra
+                .extend(starting_indexes.height, height, exit)?;
+
+            self.dateindex.compute(
+                starting_indexes.dateindex,
+                height,
+                &indexes.dateindex_to_first_height,
+                &indexes.dateindex_to_height_count,
+                exit,
+            )?;
+
+            self.difficultyepoch.compute(
+                starting_indexes.difficultyepoch,
+                height,
+                &indexes.difficultyepoch_to_first_height,
+                &indexes.difficultyepoch_to_height_count,
+                exit,
+            )?;
+        }
+
+        Ok(())
+    }
+}
+
+impl<T> Traversable for ComputedVecsFromHeight<T>
+where
+    T: ComputedVecValue + JsonSchema,
+{
+    fn to_tree_node(&self) -> brk_traversable::TreeNode {
+        let height_extra_node = self.height_extra.to_tree_node();
+        brk_traversable::TreeNode::Branch(
+            [
+                self.height
+                    .as_ref()
+                    .map(|nested| ("height".to_string(), nested.to_tree_node())),
+                if height_extra_node.is_empty() {
+                    None
+                } else {
+                    Some(("height_extra".to_string(), height_extra_node))
+                },
+                Some(("dateindex".to_string(), self.dateindex.to_tree_node())),
+                Some(("weekindex".to_string(), self.weekindex.to_tree_node())),
+                Some((
+                    "difficultyepoch".to_string(),
+                    self.difficultyepoch.to_tree_node(),
+                )),
+                Some(("monthindex".to_string(), self.monthindex.to_tree_node())),
+                Some(("quarterindex".to_string(), self.quarterindex.to_tree_node())),
+                Some((
+                    "semesterindex".to_string(),
+                    self.semesterindex.to_tree_node(),
+                )),
+                Some(("yearindex".to_string(), self.yearindex.to_tree_node())),
+                Some(("decadeindex".to_string(), self.decadeindex.to_tree_node())),
+            ]
+            .into_iter()
+            .flatten()
+            .collect(),
+        )
+        .merge_branches()
+        .unwrap()
+    }
+
+    fn iter_any_exportable(&self) -> impl Iterator<Item = &dyn AnyExportableVec> {
+        let mut regular_iter: Box<dyn Iterator<Item = &dyn AnyExportableVec>> =
+            Box::new(self.height_extra.iter_any_exportable());
+        regular_iter = Box::new(regular_iter.chain(self.dateindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.weekindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.difficultyepoch.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.monthindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.quarterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.semesterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.yearindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.decadeindex.iter_any_exportable()));
+        if let Some(ref x) = self.height {
+            regular_iter = Box::new(regular_iter.chain(x.iter_any_exportable()));
+        }
+        regular_iter
+    }
+}

crates/brk_computer/src/grouped/from_height_strict.rs

@@ -0,0 +1,121 @@
+use brk_error::Result;
+
+use brk_traversable::Traversable;
+use brk_types::{DifficultyEpoch, Height, Version};
+use schemars::JsonSchema;
+use vecdb::{AnyExportableVec, Database, EagerVec, Exit, ImportableVec, PcoVec};
+
+use crate::{Indexes, indexes};
+
+use super::{ComputedVecValue, EagerVecsBuilder, VecBuilderOptions};
+
+#[derive(Clone)]
+pub struct ComputedVecsFromHeightStrict<T>
+where
+    T: ComputedVecValue + PartialOrd + JsonSchema,
+{
+    pub height: EagerVec<PcoVec<Height, T>>,
+    pub height_extra: EagerVecsBuilder<Height, T>,
+    pub difficultyepoch: EagerVecsBuilder<DifficultyEpoch, T>,
+    // TODO: pub halvingepoch: StorableVecGeneator<Halvingepoch, T>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl<T> ComputedVecsFromHeightStrict<T>
+where
+    T: ComputedVecValue + Ord + From<f64> + JsonSchema,
+    f64: From<T>,
+{
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        version: Version,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let height = EagerVec::forced_import(db, name, version + VERSION + Version::ZERO)?;
+
+        let height_extra = EagerVecsBuilder::forced_import(
+            db,
+            name,
+            version + VERSION + Version::ZERO,
+            options.copy_self_extra(),
+        )?;
+
+        let options = options.remove_percentiles();
+
+        Ok(Self {
+            height,
+            height_extra,
+            difficultyepoch: EagerVecsBuilder::forced_import(
+                db,
+                name,
+                version + VERSION + Version::ZERO,
+                options,
+            )?,
+            // halvingepoch: StorableVecGeneator::forced_import(db, name, version + VERSION + Version::ZERO, format, options)?,
+        })
+    }
+
+    pub fn compute<F>(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<Height, T>>) -> Result<()>,
+    {
+        compute(&mut self.height)?;
+
+        self.height_extra
+            .extend(starting_indexes.height, &self.height, exit)?;
+
+        self.difficultyepoch.compute(
+            starting_indexes.difficultyepoch,
+            &self.height,
+            &indexes.difficultyepoch_to_first_height,
+            &indexes.difficultyepoch_to_height_count,
+            exit,
+        )?;
+
+        Ok(())
+    }
+}
+
+impl<T> Traversable for ComputedVecsFromHeightStrict<T>
+where
+    T: ComputedVecValue + JsonSchema,
+{
+    fn to_tree_node(&self) -> brk_traversable::TreeNode {
+        let height_extra_node = self.height_extra.to_tree_node();
+        brk_traversable::TreeNode::Branch(
+            [
+                Some(("height".to_string(), self.height.to_tree_node())),
+                if height_extra_node.is_empty() {
+                    None
+                } else {
+                    Some(("height_extra".to_string(), height_extra_node))
+                },
+                Some((
+                    "difficultyepoch".to_string(),
+                    self.difficultyepoch.to_tree_node(),
+                )),
+            ]
+            .into_iter()
+            .flatten()
+            .collect(),
+        )
+        .merge_branches()
+        .unwrap()
+    }
+
+    fn iter_any_exportable(&self) -> impl Iterator<Item = &dyn AnyExportableVec> {
+        let mut regular_iter: Box<dyn Iterator<Item = &dyn AnyExportableVec>> =
+            Box::new(self.height.iter_any_exportable());
+        regular_iter = Box::new(regular_iter.chain(self.height_extra.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.difficultyepoch.iter_any_exportable()));
+        regular_iter
+    }
+}

crates/brk_computer/src/grouped/from_txindex.rs

@@ -0,0 +1,457 @@
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{
+    Bitcoin, DateIndex, DecadeIndex, DifficultyEpoch, Dollars, Height, MonthIndex, QuarterIndex,
+    Sats, SemesterIndex, TxIndex, Version, WeekIndex, YearIndex,
+};
+use schemars::JsonSchema;
+use vecdb::{
+    AnyExportableVec, AnyVec, CollectableVec, Database, EagerVec, Exit, GenericStoredVec,
+    ImportableVec, IterableCloneableVec, PcoVec, TypedVecIterator, VecIndex,
+};
+
+use crate::{
+    Indexes,
+    grouped::{LazyVecsBuilder, Source},
+    indexes, price,
+    utils::OptionExt,
+};
+
+use super::{ComputedVecValue, EagerVecsBuilder, VecBuilderOptions};
+
+#[derive(Clone)]
+pub struct ComputedVecsFromTxindex<T>
+where
+    T: ComputedVecValue + PartialOrd + JsonSchema,
+{
+    pub txindex: Option<Box<EagerVec<PcoVec<TxIndex, T>>>>,
+    pub height: EagerVecsBuilder<Height, T>,
+    pub dateindex: EagerVecsBuilder<DateIndex, T>,
+    pub weekindex: LazyVecsBuilder<WeekIndex, T, DateIndex, WeekIndex>,
+    pub difficultyepoch: EagerVecsBuilder<DifficultyEpoch, T>,
+    pub monthindex: LazyVecsBuilder<MonthIndex, T, DateIndex, MonthIndex>,
+    pub quarterindex: LazyVecsBuilder<QuarterIndex, T, DateIndex, QuarterIndex>,
+    pub semesterindex: LazyVecsBuilder<SemesterIndex, T, DateIndex, SemesterIndex>,
+    pub yearindex: LazyVecsBuilder<YearIndex, T, DateIndex, YearIndex>,
+    // TODO: pub halvingepoch: StorableVecGeneator<Halvingepoch, T>,
+    pub decadeindex: LazyVecsBuilder<DecadeIndex, T, DateIndex, DecadeIndex>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl<T> ComputedVecsFromTxindex<T>
+where
+    T: ComputedVecValue + Ord + From<f64> + JsonSchema + 'static,
+    f64: From<T>,
+{
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<TxIndex, T>,
+        version: Version,
+        indexes: &indexes::Vecs,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let txindex = source.is_compute().then(|| {
+            Box::new(EagerVec::forced_import(db, name, version + VERSION + Version::ZERO).unwrap())
+        });
+
+        let height =
+            EagerVecsBuilder::forced_import(db, name, version + VERSION + Version::ZERO, options)?;
+
+        let options = options.remove_percentiles();
+
+        let dateindex =
+            EagerVecsBuilder::forced_import(db, name, version + VERSION + Version::ZERO, options)?;
+
+        Ok(Self {
+            weekindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.weekindex_to_weekindex.boxed_clone(),
+                options.into(),
+            ),
+            monthindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.monthindex_to_monthindex.boxed_clone(),
+                options.into(),
+            ),
+            quarterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.quarterindex_to_quarterindex.boxed_clone(),
+                options.into(),
+            ),
+            semesterindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.semesterindex_to_semesterindex.boxed_clone(),
+                options.into(),
+            ),
+            yearindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.yearindex_to_yearindex.boxed_clone(),
+                options.into(),
+            ),
+            decadeindex: LazyVecsBuilder::forced_import(
+                name,
+                version + VERSION + Version::ZERO,
+                None,
+                &dateindex,
+                indexes.decadeindex_to_decadeindex.boxed_clone(),
+                options.into(),
+            ),
+
+            txindex,
+            height,
+            dateindex,
+            difficultyepoch: EagerVecsBuilder::forced_import(
+                db,
+                name,
+                version + VERSION + Version::ZERO,
+                options,
+            )?,
+            // halvingepoch: StorableVecGeneator::forced_import(db, name, version + VERSION + Version::ZERO, format, options)?,
+        })
+    }
+
+    // pub fn compute_all<F>(
+    //     &mut self,
+    //     indexer: &Indexer,
+    //     indexes: &indexes::Vecs,
+    //     starting_indexes: &Indexes,
+    //     exit: &Exit,
+    //     mut compute: F,
+    // ) -> Result<()>
+    // where
+    //     F: FnMut(
+    //         &mut EagerVec<PcoVec<TxIndex, T>>,
+    //         &Indexer,
+    //         &indexes::Vecs,
+    //         &Indexes,
+    //         &Exit,
+    //     ) -> Result<()>,
+    // {
+    //     compute(
+    //         self.txindex.um(),
+    //         indexer,
+    //         indexes,
+    //         starting_indexes,
+    //         exit,
+    //     )?;
+
+    //     let txindex: Option<&StoredVec<TxIndex, T>> = None;
+    //     self.compute_rest(indexer, indexes, starting_indexes, exit, txindex)?;
+
+    //     Ok(())
+    // }
+
+    pub fn compute_rest(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        txindex: Option<&impl CollectableVec<TxIndex, T>>,
+    ) -> Result<()> {
+        if let Some(txindex) = txindex {
+            self.height.compute(
+                starting_indexes.height,
+                txindex,
+                &indexer.vecs.tx.height_to_first_txindex,
+                &indexes.height_to_txindex_count,
+                exit,
+            )?;
+        } else {
+            let txindex = self.txindex.u().as_ref();
+
+            self.height.compute(
+                starting_indexes.height,
+                txindex,
+                &indexer.vecs.tx.height_to_first_txindex,
+                &indexes.height_to_txindex_count,
+                exit,
+            )?;
+        }
+
+        self.compute_after_height(indexes, starting_indexes, exit)
+    }
+
+    fn compute_after_height(
+        &mut self,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.dateindex.from_aligned(
+            starting_indexes.dateindex,
+            &self.height,
+            &indexes.dateindex_to_first_height,
+            &indexes.dateindex_to_height_count,
+            exit,
+        )?;
+
+        self.difficultyepoch.from_aligned(
+            starting_indexes.difficultyepoch,
+            &self.height,
+            &indexes.difficultyepoch_to_first_height,
+            &indexes.difficultyepoch_to_height_count,
+            exit,
+        )?;
+
+        Ok(())
+    }
+}
+
+impl ComputedVecsFromTxindex<Bitcoin> {
+    pub fn compute_rest_from_sats(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        sats: &ComputedVecsFromTxindex<Sats>,
+        txindex: Option<&impl CollectableVec<TxIndex, Bitcoin>>,
+    ) -> Result<()> {
+        let txindex_version = if let Some(txindex) = txindex {
+            txindex.version()
+        } else {
+            self.txindex.u().as_ref().version()
+        };
+
+        self.height
+            .validate_computed_version_or_reset(txindex_version)?;
+
+        let starting_index = self.height.starting_index(starting_indexes.height);
+
+        // Create iterators once before the loop to avoid repeated iterator creation
+        let mut first_iter = sats.height.first.as_ref().map(|v| v.into_iter());
+        let mut average_iter = sats.height.average.as_ref().map(|v| v.into_iter());
+        let mut sum_iter = sats.height.sum.as_ref().map(|v| v.into_iter());
+        let mut max_iter = sats.height.max.as_ref().map(|v| v.into_iter());
+        let mut pct90_iter = sats.height.pct90.as_ref().map(|v| v.into_iter());
+        let mut pct75_iter = sats.height.pct75.as_ref().map(|v| v.into_iter());
+        let mut median_iter = sats.height.median.as_ref().map(|v| v.into_iter());
+        let mut pct25_iter = sats.height.pct25.as_ref().map(|v| v.into_iter());
+        let mut pct10_iter = sats.height.pct10.as_ref().map(|v| v.into_iter());
+        let mut min_iter = sats.height.min.as_ref().map(|v| v.into_iter());
+        let mut last_iter = sats.height.last.as_ref().map(|v| v.into_iter());
+        let mut cumulative_iter = sats.height.cumulative.as_ref().map(|v| v.into_iter());
+
+        (starting_index.to_usize()..indexer.vecs.block.height_to_weight.len())
+            .map(Height::from)
+            .try_for_each(|height| -> Result<()> {
+                if let Some(first) = self.height.first.as_mut() {
+                    first
+                        .truncate_push(height, Bitcoin::from(first_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(average) = self.height.average.as_mut() {
+                    average.truncate_push(
+                        height,
+                        Bitcoin::from(average_iter.um().get_unwrap(height)),
+                    )?;
+                }
+                if let Some(sum) = self.height.sum.as_mut() {
+                    sum.truncate_push(height, Bitcoin::from(sum_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(max) = self.height.max.as_mut() {
+                    max.truncate_push(height, Bitcoin::from(max_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(pct90) = self.height.pct90.as_mut() {
+                    pct90
+                        .truncate_push(height, Bitcoin::from(pct90_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(pct75) = self.height.pct75.as_mut() {
+                    pct75
+                        .truncate_push(height, Bitcoin::from(pct75_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(median) = self.height.median.as_mut() {
+                    median.truncate_push(
+                        height,
+                        Bitcoin::from(median_iter.um().get_unwrap(height)),
+                    )?;
+                }
+                if let Some(pct25) = self.height.pct25.as_mut() {
+                    pct25
+                        .truncate_push(height, Bitcoin::from(pct25_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(pct10) = self.height.pct10.as_mut() {
+                    pct10
+                        .truncate_push(height, Bitcoin::from(pct10_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(min) = self.height.min.as_mut() {
+                    min.truncate_push(height, Bitcoin::from(min_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(last) = self.height.last.as_mut() {
+                    last.truncate_push(height, Bitcoin::from(last_iter.um().get_unwrap(height)))?;
+                }
+                if let Some(cumulative) = self.height.cumulative.as_mut() {
+                    cumulative.truncate_push(
+                        height,
+                        Bitcoin::from(cumulative_iter.um().get_unwrap(height)),
+                    )?;
+                }
+                Ok(())
+            })?;
+
+        self.height.safe_write(exit)?;
+
+        self.compute_after_height(indexes, starting_indexes, exit)
+    }
+}
+
+impl ComputedVecsFromTxindex<Dollars> {
+    #[allow(clippy::too_many_arguments)]
+    pub fn compute_rest_from_bitcoin(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        bitcoin: &ComputedVecsFromTxindex<Bitcoin>,
+        txindex: Option<&impl CollectableVec<TxIndex, Dollars>>,
+        price: &price::Vecs,
+    ) -> Result<()> {
+        let txindex_version = if let Some(txindex) = txindex {
+            txindex.version()
+        } else {
+            self.txindex.u().as_ref().version()
+        };
+
+        self.height
+            .validate_computed_version_or_reset(txindex_version)?;
+
+        let starting_index = self.height.starting_index(starting_indexes.height);
+
+        let mut close_iter = price.chainindexes_to_price_close.height.into_iter();
+
+        // Create iterators once before the loop to avoid repeated iterator creation
+        let mut first_iter = bitcoin.height.first.as_ref().map(|v| v.into_iter());
+        let mut average_iter = bitcoin.height.average.as_ref().map(|v| v.into_iter());
+        let mut sum_iter = bitcoin.height.sum.as_ref().map(|v| v.into_iter());
+        let mut max_iter = bitcoin.height.max.as_ref().map(|v| v.into_iter());
+        let mut pct90_iter = bitcoin.height.pct90.as_ref().map(|v| v.into_iter());
+        let mut pct75_iter = bitcoin.height.pct75.as_ref().map(|v| v.into_iter());
+        let mut median_iter = bitcoin.height.median.as_ref().map(|v| v.into_iter());
+        let mut pct25_iter = bitcoin.height.pct25.as_ref().map(|v| v.into_iter());
+        let mut pct10_iter = bitcoin.height.pct10.as_ref().map(|v| v.into_iter());
+        let mut min_iter = bitcoin.height.min.as_ref().map(|v| v.into_iter());
+        let mut last_iter = bitcoin.height.last.as_ref().map(|v| v.into_iter());
+        let mut cumulative_iter = bitcoin.height.cumulative.as_ref().map(|v| v.into_iter());
+
+        (starting_index.to_usize()..indexer.vecs.block.height_to_weight.len())
+            .map(Height::from)
+            .try_for_each(|height| -> Result<()> {
+                let price = *close_iter.get_unwrap(height);
+
+                if let Some(first) = self.height.first.as_mut() {
+                    first.truncate_push(height, price * first_iter.um().get_unwrap(height))?;
+                }
+                if let Some(average) = self.height.average.as_mut() {
+                    average.truncate_push(height, price * average_iter.um().get_unwrap(height))?;
+                }
+                if let Some(sum) = self.height.sum.as_mut() {
+                    sum.truncate_push(height, price * sum_iter.um().get_unwrap(height))?;
+                }
+                if let Some(max) = self.height.max.as_mut() {
+                    max.truncate_push(height, price * max_iter.um().get_unwrap(height))?;
+                }
+                if let Some(pct90) = self.height.pct90.as_mut() {
+                    pct90.truncate_push(height, price * pct90_iter.um().get_unwrap(height))?;
+                }
+                if let Some(pct75) = self.height.pct75.as_mut() {
+                    pct75.truncate_push(height, price * pct75_iter.um().get_unwrap(height))?;
+                }
+                if let Some(median) = self.height.median.as_mut() {
+                    median.truncate_push(height, price * median_iter.um().get_unwrap(height))?;
+                }
+                if let Some(pct25) = self.height.pct25.as_mut() {
+                    pct25.truncate_push(height, price * pct25_iter.um().get_unwrap(height))?;
+                }
+                if let Some(pct10) = self.height.pct10.as_mut() {
+                    pct10.truncate_push(height, price * pct10_iter.um().get_unwrap(height))?;
+                }
+                if let Some(min) = self.height.min.as_mut() {
+                    min.truncate_push(height, price * min_iter.um().get_unwrap(height))?;
+                }
+                if let Some(last) = self.height.last.as_mut() {
+                    last.truncate_push(height, price * last_iter.um().get_unwrap(height))?;
+                }
+                if let Some(cumulative) = self.height.cumulative.as_mut() {
+                    cumulative
+                        .truncate_push(height, price * cumulative_iter.um().get_unwrap(height))?;
+                }
+                Ok(())
+            })?;
+
+        self.height.safe_write(exit)?;
+
+        self.compute_after_height(indexes, starting_indexes, exit)
+    }
+}
+
+impl<T> Traversable for ComputedVecsFromTxindex<T>
+where
+    T: ComputedVecValue + JsonSchema,
+{
+    fn to_tree_node(&self) -> brk_traversable::TreeNode {
+        brk_traversable::TreeNode::Branch(
+            [
+                self.txindex
+                    .as_ref()
+                    .map(|nested| ("txindex".to_string(), nested.to_tree_node())),
+                Some(("height".to_string(), self.height.to_tree_node())),
+                Some(("dateindex".to_string(), self.dateindex.to_tree_node())),
+                Some(("weekindex".to_string(), self.weekindex.to_tree_node())),
+                Some((
+                    "difficultyepoch".to_string(),
+                    self.difficultyepoch.to_tree_node(),
+                )),
+                Some(("monthindex".to_string(), self.monthindex.to_tree_node())),
+                Some(("quarterindex".to_string(), self.quarterindex.to_tree_node())),
+                Some((
+                    "semesterindex".to_string(),
+                    self.semesterindex.to_tree_node(),
+                )),
+                Some(("yearindex".to_string(), self.yearindex.to_tree_node())),
+                Some(("decadeindex".to_string(), self.decadeindex.to_tree_node())),
+            ]
+            .into_iter()
+            .flatten()
+            .collect(),
+        )
+        .merge_branches()
+        .unwrap()
+    }
+
+    fn iter_any_exportable(&self) -> impl Iterator<Item = &dyn AnyExportableVec> {
+        let mut regular_iter: Box<dyn Iterator<Item = &dyn AnyExportableVec>> =
+            Box::new(self.height.iter_any_exportable());
+        regular_iter = Box::new(regular_iter.chain(self.dateindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.weekindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.difficultyepoch.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.monthindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.quarterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.semesterindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.yearindex.iter_any_exportable()));
+        regular_iter = Box::new(regular_iter.chain(self.decadeindex.iter_any_exportable()));
+        if let Some(ref x) = self.txindex {
+            regular_iter = Box::new(regular_iter.chain(x.iter_any_exportable()));
+        }
+        regular_iter
+    }
+}

crates/brk_computer/src/grouped/mod.rs

@@ -1,22 +1,30 @@
-mod builder;
+mod builder_eager;
+mod builder_lazy;
+mod computed;
 mod from_dateindex;
 mod from_height;
 mod from_height_strict;
 mod from_txindex;
+mod price_percentiles;
 mod ratio_from_dateindex;
-mod r#type;
+mod sd_from_dateindex;
+mod source;
 mod value_from_dateindex;
 mod value_from_height;
 mod value_from_txindex;
 mod value_height;
 
-pub use builder::*;
+pub use builder_eager::*;
+pub use builder_lazy::*;
+use computed::*;
 pub use from_dateindex::*;
 pub use from_height::*;
 pub use from_height_strict::*;
 pub use from_txindex::*;
+pub use price_percentiles::*;
 pub use ratio_from_dateindex::*;
-use r#type::*;
+pub use sd_from_dateindex::*;
+pub use source::*;
 pub use value_from_dateindex::*;
 pub use value_from_height::*;
 pub use value_from_txindex::*;

crates/brk_computer/src/grouped/price_percentiles.rs

@@ -0,0 +1,122 @@
+use brk_error::Result;
+use brk_traversable::{Traversable, TreeNode};
+use brk_types::{DateIndex, Dollars, Version};
+use vecdb::{AnyExportableVec, AnyStoredVec, Database, EagerVec, Exit, GenericStoredVec, PcoVec};
+
+use crate::{Indexes, indexes};
+
+use super::{ComputedVecsFromDateIndex, Source, VecBuilderOptions};
+
+pub const PERCENTILES: [u8; 19] = [
+    5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95,
+];
+pub const PERCENTILES_LEN: usize = PERCENTILES.len();
+
+#[derive(Clone)]
+pub struct PricePercentiles {
+    pub vecs: [Option<ComputedVecsFromDateIndex<Dollars>>; PERCENTILES_LEN],
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl PricePercentiles {
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        version: Version,
+        indexes: &indexes::Vecs,
+        compute: bool,
+    ) -> Result<Self> {
+        let vecs = PERCENTILES.map(|p| {
+            compute.then(|| {
+                ComputedVecsFromDateIndex::forced_import(
+                    db,
+                    &format!("{name}_price_pct{p:02}"),
+                    Source::Compute,
+                    version + VERSION,
+                    indexes,
+                    VecBuilderOptions::default().add_last(),
+                )
+                .unwrap()
+            })
+        });
+
+        Ok(Self { vecs })
+    }
+
+    /// Push percentile prices at date boundary.
+    /// Only called when dateindex is Some (last height of the day).
+    pub fn truncate_push(
+        &mut self,
+        dateindex: DateIndex,
+        percentile_prices: &[Dollars; PERCENTILES_LEN],
+    ) -> Result<()> {
+        for (i, vec) in self.vecs.iter_mut().enumerate() {
+            if let Some(v) = vec {
+                v.dateindex
+                    .as_mut()
+                    .unwrap()
+                    .truncate_push(dateindex, percentile_prices[i])?;
+            }
+        }
+        Ok(())
+    }
+
+    pub fn compute_rest(&mut self, starting_indexes: &Indexes, exit: &Exit) -> Result<()> {
+        for vec in self.vecs.iter_mut().flatten() {
+            vec.compute_rest(
+                starting_indexes,
+                exit,
+                None::<&EagerVec<PcoVec<DateIndex, Dollars>>>,
+            )?;
+        }
+        Ok(())
+    }
+
+    pub fn get(&self, percentile: u8) -> Option<&ComputedVecsFromDateIndex<Dollars>> {
+        PERCENTILES
+            .iter()
+            .position(|&p| p == percentile)
+            .and_then(|i| self.vecs[i].as_ref())
+    }
+}
+
+impl PricePercentiles {
+    pub fn write(&mut self) -> Result<()> {
+        for vec in self.vecs.iter_mut().flatten() {
+            if let Some(dateindex_vec) = vec.dateindex.as_mut() {
+                dateindex_vec.write()?;
+            }
+        }
+        Ok(())
+    }
+
+    /// Validate computed versions or reset if mismatched.
+    pub fn validate_computed_version_or_reset(&mut self, version: Version) -> Result<()> {
+        for vec in self.vecs.iter_mut().flatten() {
+            if let Some(dateindex_vec) = vec.dateindex.as_mut() {
+                dateindex_vec.validate_computed_version_or_reset(version)?;
+            }
+        }
+        Ok(())
+    }
+}
+
+impl Traversable for PricePercentiles {
+    fn to_tree_node(&self) -> TreeNode {
+        TreeNode::Branch(
+            PERCENTILES
+                .iter()
+                .zip(self.vecs.iter())
+                .filter_map(|(p, v)| v.as_ref().map(|v| (format!("pct{p:02}"), v.to_tree_node())))
+                .collect(),
+        )
+    }
+
+    fn iter_any_exportable(&self) -> impl Iterator<Item = &dyn AnyExportableVec> {
+        self.vecs
+            .iter()
+            .flatten()
+            .flat_map(|p| p.iter_any_exportable())
+    }
+}

crates/brk_computer/src/grouped/ratio_from_dateindex.rs

@@ -0,0 +1,389 @@
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Date, DateIndex, Dollars, StoredF32, Version};
+use vecdb::{PcoVec, 
+    AnyStoredVec, AnyVec, CollectableVec, Database, EagerVec, Exit, GenericStoredVec, IterableVec,
+    TypedVecIterator, VecIndex,
+};
+
+use crate::{
+    Indexes,
+    grouped::{
+        ComputedStandardDeviationVecsFromDateIndex, StandardDeviationVecsOptions, source::Source,
+    },
+    indexes, price,
+    utils::{get_percentile, OptionExt},
+};
+
+use super::{ComputedVecsFromDateIndex, VecBuilderOptions};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedRatioVecsFromDateIndex {
+    pub price: Option<ComputedVecsFromDateIndex<Dollars>>,
+
+    pub ratio: ComputedVecsFromDateIndex<StoredF32>,
+    pub ratio_1w_sma: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_1m_sma: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct99: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct98: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct95: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct5: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct2: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct1: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub ratio_pct99_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub ratio_pct98_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub ratio_pct95_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub ratio_pct5_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub ratio_pct2_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub ratio_pct1_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+
+    pub ratio_sd: Option<ComputedStandardDeviationVecsFromDateIndex>,
+    pub ratio_4y_sd: Option<ComputedStandardDeviationVecsFromDateIndex>,
+    pub ratio_2y_sd: Option<ComputedStandardDeviationVecsFromDateIndex>,
+    pub ratio_1y_sd: Option<ComputedStandardDeviationVecsFromDateIndex>,
+}
+
+const VERSION: Version = Version::TWO;
+
+impl ComputedRatioVecsFromDateIndex {
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<DateIndex, Dollars>,
+        version: Version,
+        indexes: &indexes::Vecs,
+        extended: bool,
+    ) -> Result<Self> {
+        let opts = VecBuilderOptions::default().add_last();
+        let v = version + VERSION;
+
+        macro_rules! import {
+            ($suffix:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    db, &format!("{name}_{}", $suffix), Source::Compute, v, indexes, opts,
+                ).unwrap()
+            };
+        }
+        macro_rules! import_sd {
+            ($suffix:expr, $days:expr) => {
+                ComputedStandardDeviationVecsFromDateIndex::forced_import(
+                    db, &format!("{name}_{}", $suffix), $days, Source::Compute, v, indexes,
+                    StandardDeviationVecsOptions::default().add_all(),
+                ).unwrap()
+            };
+        }
+
+        Ok(Self {
+            price: source.is_compute().then(|| {
+                ComputedVecsFromDateIndex::forced_import(db, name, Source::Compute, v, indexes, opts).unwrap()
+            }),
+            ratio: import!("ratio"),
+            ratio_1w_sma: extended.then(|| import!("ratio_1w_sma")),
+            ratio_1m_sma: extended.then(|| import!("ratio_1m_sma")),
+            ratio_sd: extended.then(|| import_sd!("ratio", usize::MAX)),
+            ratio_1y_sd: extended.then(|| import_sd!("ratio_1y", 365)),
+            ratio_2y_sd: extended.then(|| import_sd!("ratio_2y", 2 * 365)),
+            ratio_4y_sd: extended.then(|| import_sd!("ratio_4y", 4 * 365)),
+            ratio_pct99: extended.then(|| import!("ratio_pct99")),
+            ratio_pct98: extended.then(|| import!("ratio_pct98")),
+            ratio_pct95: extended.then(|| import!("ratio_pct95")),
+            ratio_pct5: extended.then(|| import!("ratio_pct5")),
+            ratio_pct2: extended.then(|| import!("ratio_pct2")),
+            ratio_pct1: extended.then(|| import!("ratio_pct1")),
+            ratio_pct99_usd: extended.then(|| import!("ratio_pct99_usd")),
+            ratio_pct98_usd: extended.then(|| import!("ratio_pct98_usd")),
+            ratio_pct95_usd: extended.then(|| import!("ratio_pct95_usd")),
+            ratio_pct5_usd: extended.then(|| import!("ratio_pct5_usd")),
+            ratio_pct2_usd: extended.then(|| import!("ratio_pct2_usd")),
+            ratio_pct1_usd: extended.then(|| import!("ratio_pct1_usd")),
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        price: &price::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<DateIndex, Dollars>>) -> Result<()>,
+    {
+        self.price
+            .as_mut()
+            .unwrap()
+            .compute_all(starting_indexes, exit, compute)?;
+
+        let date_to_price_opt: Option<&EagerVec<PcoVec<DateIndex, Dollars>>> = None;
+        self.compute_rest(price, starting_indexes, exit, date_to_price_opt)
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        price: &price::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        price_opt: Option<&impl IterableVec<DateIndex, Dollars>>,
+    ) -> Result<()> {
+        let closes = price.timeindexes_to_price_close.dateindex.u();
+
+        let price = price_opt.unwrap_or_else(|| unsafe {
+            std::mem::transmute(&self.price.u().dateindex)
+        });
+
+        self.ratio.compute_all(starting_indexes, exit, |v| {
+            v.compute_transform2(
+                starting_indexes.dateindex,
+                closes,
+                price,
+                |(i, close, price, ..)| {
+                    if price == Dollars::ZERO {
+                        (i, StoredF32::from(1.0))
+                    } else {
+                        (i, StoredF32::from(*close / price))
+                    }
+                },
+                exit,
+            )?;
+            Ok(())
+        })?;
+
+        if self.ratio_1w_sma.is_none() {
+            return Ok(());
+        }
+
+        let min_ratio_date = DateIndex::try_from(Date::MIN_RATIO).unwrap();
+
+        self.ratio_1w_sma
+            .as_mut()
+            .unwrap()
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_sma_(
+                    starting_indexes.dateindex,
+                    self.ratio.dateindex.u(),
+                    7,
+                    exit,
+                    Some(min_ratio_date),
+                )?;
+                Ok(())
+            })?;
+
+        self.ratio_1m_sma
+            .as_mut()
+            .unwrap()
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_sma_(
+                    starting_indexes.dateindex,
+                    self.ratio.dateindex.u(),
+                    30,
+                    exit,
+                    Some(min_ratio_date),
+                )?;
+                Ok(())
+            })?;
+
+        let ratio_version = self.ratio.dateindex.u().version();
+        self.mut_ratio_vecs()
+            .iter_mut()
+            .try_for_each(|v| -> Result<()> {
+                v.validate_computed_version_or_reset(
+                    Version::ZERO + v.inner_version() + ratio_version,
+                )?;
+                Ok(())
+            })?;
+
+        let starting_dateindex = self
+            .mut_ratio_vecs()
+            .iter()
+            .map(|v| DateIndex::from(v.len()))
+            .min()
+            .unwrap()
+            .min(starting_indexes.dateindex);
+
+        let min_ratio_date_usize = min_ratio_date.to_usize();
+
+        let mut sorted = self.ratio.dateindex.u().collect_range(
+            Some(min_ratio_date_usize),
+            Some(starting_dateindex.to_usize()),
+        );
+
+        sorted.sort_unstable();
+
+        // Cache mutable refs before the loop to avoid repeated unwrap chains
+        let pct1_vec = self.ratio_pct1.um().dateindex.um();
+        let pct2_vec = self.ratio_pct2.um().dateindex.um();
+        let pct5_vec = self.ratio_pct5.um().dateindex.um();
+        let pct95_vec = self.ratio_pct95.um().dateindex.um();
+        let pct98_vec = self.ratio_pct98.um().dateindex.um();
+        let pct99_vec = self.ratio_pct99.um().dateindex.um();
+
+        self.ratio
+            .dateindex
+            .as_ref()
+            .unwrap()
+            .iter()
+            .enumerate()
+            .skip(starting_dateindex.to_usize())
+            .try_for_each(|(index, ratio)| -> Result<()> {
+                if index < min_ratio_date_usize {
+                    pct1_vec.truncate_push_at(index, StoredF32::NAN)?;
+                    pct2_vec.truncate_push_at(index, StoredF32::NAN)?;
+                    pct5_vec.truncate_push_at(index, StoredF32::NAN)?;
+                    pct95_vec.truncate_push_at(index, StoredF32::NAN)?;
+                    pct98_vec.truncate_push_at(index, StoredF32::NAN)?;
+                    pct99_vec.truncate_push_at(index, StoredF32::NAN)?;
+                } else {
+                    let pos = sorted.binary_search(&ratio).unwrap_or_else(|pos| pos);
+                    sorted.insert(pos, ratio);
+
+                    pct1_vec.truncate_push_at(index, get_percentile(&sorted, 0.01))?;
+                    pct2_vec.truncate_push_at(index, get_percentile(&sorted, 0.02))?;
+                    pct5_vec.truncate_push_at(index, get_percentile(&sorted, 0.05))?;
+                    pct95_vec.truncate_push_at(index, get_percentile(&sorted, 0.95))?;
+                    pct98_vec.truncate_push_at(index, get_percentile(&sorted, 0.98))?;
+                    pct99_vec.truncate_push_at(index, get_percentile(&sorted, 0.99))?;
+                }
+
+                Ok(())
+            })?;
+
+        self.mut_ratio_vecs()
+            .into_iter()
+            .try_for_each(|v| v.safe_flush(exit))?;
+
+        self.ratio_pct1.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+        self.ratio_pct2.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+        self.ratio_pct5.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+        self.ratio_pct95.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+        self.ratio_pct98.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+        self.ratio_pct99.um().compute_rest(
+            starting_indexes,
+            exit,
+            None as Option<&EagerVec<PcoVec<_, _>>>,
+        )?;
+
+        let date_to_price = price_opt.unwrap_or_else(|| unsafe {
+            std::mem::transmute(&self.price.u().dateindex)
+        });
+
+        self.ratio_pct99_usd
+            .as_mut()
+            .unwrap()
+            .compute_all(starting_indexes, exit, |vec| {
+                let mut iter = self
+                    .ratio_pct99
+                    .as_ref()
+                    .unwrap()
+                    .dateindex
+                    .as_ref()
+                    .unwrap()
+                    .into_iter();
+                vec.compute_transform(
+                    starting_indexes.dateindex,
+                    date_to_price,
+                    |(i, price, ..)| {
+                        let multiplier = iter.get_unwrap(i);
+                        (i, price * multiplier)
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        let compute_usd =
+            |usd: Option<&mut ComputedVecsFromDateIndex<Dollars>>,
+             source: Option<&ComputedVecsFromDateIndex<StoredF32>>| {
+                usd.unwrap().compute_all(starting_indexes, exit, |vec| {
+                    let mut iter = source.unwrap().dateindex.u().into_iter();
+                    vec.compute_transform(
+                        starting_indexes.dateindex,
+                        date_to_price,
+                        |(i, price, ..)| {
+                            let multiplier = iter.get_unwrap(i);
+                            (i, price * multiplier)
+                        },
+                        exit,
+                    )?;
+                    Ok(())
+                })
+            };
+
+        compute_usd(self.ratio_pct1_usd.as_mut(), self.ratio_pct1.as_ref())?;
+        compute_usd(self.ratio_pct2_usd.as_mut(), self.ratio_pct2.as_ref())?;
+        compute_usd(self.ratio_pct5_usd.as_mut(), self.ratio_pct5.as_ref())?;
+        compute_usd(self.ratio_pct95_usd.as_mut(), self.ratio_pct95.as_ref())?;
+        compute_usd(self.ratio_pct98_usd.as_mut(), self.ratio_pct98.as_ref())?;
+        compute_usd(self.ratio_pct99_usd.as_mut(), self.ratio_pct99.as_ref())?;
+
+        self.ratio_sd.um().compute_all(
+            starting_indexes,
+            exit,
+            self.ratio.dateindex.u(),
+            Some(date_to_price),
+        )?;
+        self.ratio_4y_sd.um().compute_all(
+            starting_indexes,
+            exit,
+            self.ratio.dateindex.u(),
+            Some(date_to_price),
+        )?;
+        self.ratio_2y_sd.um().compute_all(
+            starting_indexes,
+            exit,
+            self.ratio.dateindex.u(),
+            Some(date_to_price),
+        )?;
+        self.ratio_1y_sd.um().compute_all(
+            starting_indexes,
+            exit,
+            self.ratio.dateindex.u(),
+            Some(date_to_price),
+        )?;
+
+        Ok(())
+    }
+
+    fn mut_ratio_vecs(&mut self) -> Vec<&mut EagerVec<PcoVec<DateIndex, StoredF32>>> {
+        let mut vecs = Vec::with_capacity(6);
+        if let Some(v) = self.ratio_pct1.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        if let Some(v) = self.ratio_pct2.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        if let Some(v) = self.ratio_pct5.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        if let Some(v) = self.ratio_pct95.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        if let Some(v) = self.ratio_pct98.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        if let Some(v) = self.ratio_pct99.as_mut() {
+            vecs.push(v.dateindex.um());
+        }
+        vecs
+    }
+}

crates/brk_computer/src/grouped/sd_from_dateindex.rs

@@ -0,0 +1,551 @@
+use std::mem;
+
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Date, DateIndex, Dollars, StoredF32, Version};
+use vecdb::{
+    AnyStoredVec, AnyVec, BoxedVecIterator, CollectableVec, Database, EagerVec, Exit,
+    GenericStoredVec, IterableVec, PcoVec, VecIndex,
+};
+
+use crate::{Indexes, grouped::source::Source, indexes, utils::OptionExt};
+
+use super::{ComputedVecsFromDateIndex, VecBuilderOptions};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedStandardDeviationVecsFromDateIndex {
+    days: usize,
+
+    pub sma: Option<ComputedVecsFromDateIndex<StoredF32>>,
+
+    pub sd: ComputedVecsFromDateIndex<StoredF32>,
+
+    pub zscore: Option<ComputedVecsFromDateIndex<StoredF32>>,
+
+    pub p0_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub p1sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub p1_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub p2sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub p2_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub p3sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m0_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m1sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m1_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m2sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m2_5sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+    pub m3sd: Option<ComputedVecsFromDateIndex<StoredF32>>,
+
+    pub _0sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p0_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p1sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p1_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p2sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p2_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub p3sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m0_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m1sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m1_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m2sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m2_5sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+    pub m3sd_usd: Option<ComputedVecsFromDateIndex<Dollars>>,
+}
+
+#[derive(Debug, Default)]
+pub struct StandardDeviationVecsOptions {
+    zscore: bool,
+    bands: bool,
+    price_bands: bool,
+}
+
+impl StandardDeviationVecsOptions {
+    pub fn add_all(mut self) -> Self {
+        self.zscore = true;
+        self.bands = true;
+        self.price_bands = true;
+        self
+    }
+
+    pub fn add_zscore(mut self) -> Self {
+        self.zscore = true;
+        self
+    }
+
+    pub fn add_bands(mut self) -> Self {
+        self.bands = true;
+        self
+    }
+
+    pub fn add_price_bands(mut self) -> Self {
+        self.bands = true;
+        self.price_bands = true;
+        self
+    }
+
+    pub fn zscore(&self) -> bool {
+        self.zscore
+    }
+
+    pub fn bands(&self) -> bool {
+        self.bands
+    }
+
+    pub fn price_bands(&self) -> bool {
+        self.price_bands
+    }
+}
+
+impl ComputedStandardDeviationVecsFromDateIndex {
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        days: usize,
+        sma: Source<DateIndex, StoredF32>,
+        parent_version: Version,
+        indexes: &indexes::Vecs,
+        options: StandardDeviationVecsOptions,
+    ) -> Result<Self> {
+        let opts = VecBuilderOptions::default().add_last();
+        let version = parent_version + Version::ONE;
+
+        macro_rules! import {
+            ($suffix:expr) => {
+                ComputedVecsFromDateIndex::forced_import(
+                    db,
+                    &format!("{name}_{}", $suffix),
+                    Source::Compute,
+                    version,
+                    indexes,
+                    opts,
+                )
+                .unwrap()
+            };
+        }
+
+        Ok(Self {
+            days,
+            sma: sma.is_compute().then(|| import!("sma")),
+            sd: import!("sd"),
+            p0_5sd: options.bands().then(|| import!("p0_5sd")),
+            p1sd: options.bands().then(|| import!("p1sd")),
+            p1_5sd: options.bands().then(|| import!("p1_5sd")),
+            p2sd: options.bands().then(|| import!("p2sd")),
+            p2_5sd: options.bands().then(|| import!("p2_5sd")),
+            p3sd: options.bands().then(|| import!("p3sd")),
+            m0_5sd: options.bands().then(|| import!("m0_5sd")),
+            m1sd: options.bands().then(|| import!("m1sd")),
+            m1_5sd: options.bands().then(|| import!("m1_5sd")),
+            m2sd: options.bands().then(|| import!("m2sd")),
+            m2_5sd: options.bands().then(|| import!("m2_5sd")),
+            m3sd: options.bands().then(|| import!("m3sd")),
+            _0sd_usd: options.price_bands().then(|| import!("0sd_usd")),
+            p0_5sd_usd: options.price_bands().then(|| import!("p0_5sd_usd")),
+            p1sd_usd: options.price_bands().then(|| import!("p1sd_usd")),
+            p1_5sd_usd: options.price_bands().then(|| import!("p1_5sd_usd")),
+            p2sd_usd: options.price_bands().then(|| import!("p2sd_usd")),
+            p2_5sd_usd: options.price_bands().then(|| import!("p2_5sd_usd")),
+            p3sd_usd: options.price_bands().then(|| import!("p3sd_usd")),
+            m0_5sd_usd: options.price_bands().then(|| import!("m0_5sd_usd")),
+            m1sd_usd: options.price_bands().then(|| import!("m1sd_usd")),
+            m1_5sd_usd: options.price_bands().then(|| import!("m1_5sd_usd")),
+            m2sd_usd: options.price_bands().then(|| import!("m2sd_usd")),
+            m2_5sd_usd: options.price_bands().then(|| import!("m2_5sd_usd")),
+            m3sd_usd: options.price_bands().then(|| import!("m3sd_usd")),
+            zscore: options.zscore().then(|| import!("zscore")),
+        })
+    }
+
+    pub fn compute_all(
+        &mut self,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        source: &impl CollectableVec<DateIndex, StoredF32>,
+        price_opt: Option<&impl IterableVec<DateIndex, Dollars>>,
+    ) -> Result<()> {
+        let min_date = DateIndex::try_from(Date::MIN_RATIO).unwrap();
+
+        self.sma
+            .as_mut()
+            .unwrap()
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_sma_(
+                    starting_indexes.dateindex,
+                    source,
+                    self.days,
+                    exit,
+                    Some(min_date),
+                )?;
+                Ok(())
+            })?;
+
+        let sma_opt: Option<&EagerVec<PcoVec<DateIndex, StoredF32>>> = None;
+        self.compute_rest(starting_indexes, exit, sma_opt, source, price_opt)
+    }
+
+    #[allow(clippy::too_many_arguments)]
+    pub fn compute_rest(
+        &mut self,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        sma_opt: Option<&impl IterableVec<DateIndex, StoredF32>>,
+        source: &impl CollectableVec<DateIndex, StoredF32>,
+        price_opt: Option<&impl IterableVec<DateIndex, Dollars>>,
+    ) -> Result<()> {
+        let sma = sma_opt.unwrap_or_else(|| unsafe { mem::transmute(&self.sma.u().dateindex) });
+
+        let min_date = DateIndex::try_from(Date::MIN_RATIO).unwrap();
+
+        let source_version = source.version();
+
+        self.mut_stateful_date_vecs()
+            .try_for_each(|v| -> Result<()> {
+                v.validate_computed_version_or_reset(
+                    Version::ZERO + v.inner_version() + source_version,
+                )?;
+                Ok(())
+            })?;
+
+        let starting_dateindex = self
+            .mut_stateful_date_vecs()
+            .map(|v| DateIndex::from(v.len()))
+            .min()
+            .unwrap()
+            .min(starting_indexes.dateindex);
+
+        let mut sorted = source.collect_range(
+            Some(min_date.to_usize()),
+            Some(starting_dateindex.to_usize()),
+        );
+
+        sorted.sort_unstable();
+
+        let mut p0_5sd = self.p0_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut p1sd = self.p1sd.as_mut().map(|c| c.dateindex.um());
+        let mut p1_5sd = self.p1_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut p2sd = self.p2sd.as_mut().map(|c| c.dateindex.um());
+        let mut p2_5sd = self.p2_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut p3sd = self.p3sd.as_mut().map(|c| c.dateindex.um());
+        let mut m0_5sd = self.m0_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut m1sd = self.m1sd.as_mut().map(|c| c.dateindex.um());
+        let mut m1_5sd = self.m1_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut m2sd = self.m2sd.as_mut().map(|c| c.dateindex.um());
+        let mut m2_5sd = self.m2_5sd.as_mut().map(|c| c.dateindex.um());
+        let mut m3sd = self.m3sd.as_mut().map(|c| c.dateindex.um());
+
+        let min_date_usize = min_date.to_usize();
+        let mut sma_iter = sma.iter().skip(starting_dateindex.to_usize());
+
+        source
+            .iter()
+            .enumerate()
+            .skip(starting_dateindex.to_usize())
+            .try_for_each(|(index, ratio)| -> Result<()> {
+                if index < min_date_usize {
+                    self.sd
+                        .dateindex
+                        .as_mut()
+                        .unwrap()
+                        .truncate_push_at(index, StoredF32::NAN)?;
+
+                    if let Some(v) = p0_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = p1sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = p1_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = p2sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = p2_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = p3sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m0_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m1sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m1_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m2sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m2_5sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    if let Some(v) = m3sd.as_mut() {
+                        v.truncate_push_at(index, StoredF32::NAN)?
+                    }
+                    // Advance iterator to stay in sync
+                    sma_iter.next();
+                } else {
+                    let pos = sorted.binary_search(&ratio).unwrap_or_else(|pos| pos);
+                    sorted.insert(pos, ratio);
+
+                    let avg = sma_iter.next().unwrap();
+
+                    let population =
+                        index.checked_sub(min_date_usize).unwrap().to_usize() as f32 + 1.0;
+
+                    let sd = StoredF32::from(
+                        (sorted.iter().map(|v| (**v - *avg).powi(2)).sum::<f32>() / population)
+                            .sqrt(),
+                    );
+
+                    self.sd
+                        .dateindex
+                        .as_mut()
+                        .unwrap()
+                        .truncate_push_at(index, sd)?;
+                    if let Some(v) = p0_5sd.as_mut() {
+                        v.truncate_push_at(index, avg + StoredF32::from(0.5 * *sd))?
+                    }
+                    if let Some(v) = p1sd.as_mut() {
+                        v.truncate_push_at(index, avg + sd)?
+                    }
+                    if let Some(v) = p1_5sd.as_mut() {
+                        v.truncate_push_at(index, avg + StoredF32::from(1.5 * *sd))?
+                    }
+                    if let Some(v) = p2sd.as_mut() {
+                        v.truncate_push_at(index, avg + 2 * sd)?
+                    }
+                    if let Some(v) = p2_5sd.as_mut() {
+                        v.truncate_push_at(index, avg + StoredF32::from(2.5 * *sd))?
+                    }
+                    if let Some(v) = p3sd.as_mut() {
+                        v.truncate_push_at(index, avg + 3 * sd)?
+                    }
+                    if let Some(v) = m0_5sd.as_mut() {
+                        v.truncate_push_at(index, avg - StoredF32::from(0.5 * *sd))?
+                    }
+                    if let Some(v) = m1sd.as_mut() {
+                        v.truncate_push_at(index, avg - sd)?
+                    }
+                    if let Some(v) = m1_5sd.as_mut() {
+                        v.truncate_push_at(index, avg - StoredF32::from(1.5 * *sd))?
+                    }
+                    if let Some(v) = m2sd.as_mut() {
+                        v.truncate_push_at(index, avg - 2 * sd)?
+                    }
+                    if let Some(v) = m2_5sd.as_mut() {
+                        v.truncate_push_at(index, avg - StoredF32::from(2.5 * *sd))?
+                    }
+                    if let Some(v) = m3sd.as_mut() {
+                        v.truncate_push_at(index, avg - 3 * sd)?
+                    }
+                }
+
+                Ok(())
+            })?;
+
+        drop(sma_iter);
+
+        self.mut_stateful_date_vecs()
+            .try_for_each(|v| v.safe_flush(exit))?;
+
+        self.mut_stateful_computed().try_for_each(|v| {
+            v.compute_rest(
+                starting_indexes,
+                exit,
+                None as Option<&EagerVec<PcoVec<_, _>>>,
+            )
+        })?;
+
+        if let Some(zscore) = self.zscore.as_mut() {
+            zscore.compute_all(starting_indexes, exit, |vec| {
+                vec.compute_zscore(
+                    starting_indexes.dateindex,
+                    source,
+                    sma,
+                    self.sd.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        }
+
+        let Some(price) = price_opt else {
+            return Ok(());
+        };
+
+        let compute_usd =
+            |usd: &mut ComputedVecsFromDateIndex<Dollars>,
+             mut iter: BoxedVecIterator<DateIndex, StoredF32>| {
+                usd.compute_all(starting_indexes, exit, |vec| {
+                    vec.compute_transform(
+                        starting_indexes.dateindex,
+                        price,
+                        |(i, price, ..)| {
+                            let multiplier = iter.get_unwrap(i);
+                            (i, price * multiplier)
+                        },
+                        exit,
+                    )?;
+                    Ok(())
+                })
+            };
+
+        if self._0sd_usd.is_none() {
+            return Ok(());
+        }
+
+        compute_usd(self._0sd_usd.um(), sma.iter())?;
+        compute_usd(
+            self.p0_5sd_usd.um(),
+            self.p0_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.p1sd_usd.um(),
+            self.p1sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.p1_5sd_usd.um(),
+            self.p1_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.p2sd_usd.um(),
+            self.p2sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.p2_5sd_usd.um(),
+            self.p2_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.p3sd_usd.um(),
+            self.p3sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m0_5sd_usd.um(),
+            self.m0_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m1sd_usd.um(),
+            self.m1sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m1_5sd_usd.um(),
+            self.m1_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m2sd_usd.um(),
+            self.m2sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m2_5sd_usd.um(),
+            self.m2_5sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+        compute_usd(
+            self.m3sd_usd.um(),
+            self.m3sd
+                .as_ref()
+                .unwrap()
+                .dateindex
+                .as_ref()
+                .unwrap()
+                .iter(),
+        )?;
+
+        Ok(())
+    }
+
+    fn mut_stateful_computed(
+        &mut self,
+    ) -> impl Iterator<Item = &mut ComputedVecsFromDateIndex<StoredF32>> {
+        [
+            Some(&mut self.sd),
+            self.p0_5sd.as_mut(),
+            self.p1sd.as_mut(),
+            self.p1_5sd.as_mut(),
+            self.p2sd.as_mut(),
+            self.p2_5sd.as_mut(),
+            self.p3sd.as_mut(),
+            self.m0_5sd.as_mut(),
+            self.m1sd.as_mut(),
+            self.m1_5sd.as_mut(),
+            self.m2sd.as_mut(),
+            self.m2_5sd.as_mut(),
+            self.m3sd.as_mut(),
+        ]
+        .into_iter()
+        .flatten()
+    }
+
+    fn mut_stateful_date_vecs(
+        &mut self,
+    ) -> impl Iterator<Item = &mut EagerVec<PcoVec<DateIndex, StoredF32>>> {
+        self.mut_stateful_computed().map(|c| c.dateindex.um())
+    }
+}

crates/brk_computer/src/grouped/source.rs

@@ -0,0 +1,54 @@
+use vecdb::IterableBoxedVec;
+
+#[derive(Clone)]
+pub enum Source<I, T> {
+    Compute,
+    None,
+    Vec(IterableBoxedVec<I, T>),
+}
+
+impl<I, T> Source<I, T> {
+    pub fn is_compute(&self) -> bool {
+        matches!(self, Self::Compute)
+    }
+
+    pub fn is_none(&self) -> bool {
+        matches!(self, Self::None)
+    }
+
+    pub fn is_vec(&self) -> bool {
+        matches!(self, Self::Vec(_))
+    }
+
+    pub fn vec(self) -> Option<IterableBoxedVec<I, T>> {
+        match self {
+            Self::Vec(v) => Some(v),
+            _ => None,
+        }
+    }
+}
+
+impl<I, T> From<bool> for Source<I, T> {
+    #[inline]
+    fn from(value: bool) -> Self {
+        if value { Self::Compute } else { Self::None }
+    }
+}
+
+impl<I, T> From<IterableBoxedVec<I, T>> for Source<I, T> {
+    #[inline]
+    fn from(value: IterableBoxedVec<I, T>) -> Self {
+        Self::Vec(value)
+    }
+}
+
+impl<I, T> From<Option<IterableBoxedVec<I, T>>> for Source<I, T> {
+    #[inline]
+    fn from(value: Option<IterableBoxedVec<I, T>>) -> Self {
+        if let Some(v) = value {
+            Self::Vec(v)
+        } else {
+            Self::None
+        }
+    }
+}

crates/brk_computer/src/grouped/value_from_dateindex.rs

@@ -0,0 +1,135 @@
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, DateIndex, Dollars, Sats, Version};
+use vecdb::{CollectableVec, Database, EagerVec, Exit, PcoVec};
+
+use crate::{
+    Indexes,
+    grouped::ComputedVecsFromDateIndex,
+    indexes, price,
+    traits::{ComputeFromBitcoin, ComputeFromSats},
+    utils::OptionExt,
+};
+
+use super::{Source, VecBuilderOptions};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedValueVecsFromDateIndex {
+    pub sats: ComputedVecsFromDateIndex<Sats>,
+    pub bitcoin: ComputedVecsFromDateIndex<Bitcoin>,
+    pub dollars: Option<ComputedVecsFromDateIndex<Dollars>>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl ComputedValueVecsFromDateIndex {
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<DateIndex, Sats>,
+        version: Version,
+        options: VecBuilderOptions,
+        compute_dollars: bool,
+        indexes: &indexes::Vecs,
+    ) -> Result<Self> {
+        Ok(Self {
+            sats: ComputedVecsFromDateIndex::forced_import(
+                db,
+                name,
+                source,
+                version + VERSION,
+                indexes,
+                options,
+            )?,
+            bitcoin: ComputedVecsFromDateIndex::forced_import(
+                db,
+                &format!("{name}_btc"),
+                Source::Compute,
+                version + VERSION,
+                indexes,
+                options,
+            )?,
+            dollars: compute_dollars.then(|| {
+                ComputedVecsFromDateIndex::forced_import(
+                    db,
+                    &format!("{name}_usd"),
+                    Source::Compute,
+                    version + VERSION,
+                    indexes,
+                    options,
+                )
+                .unwrap()
+            }),
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<DateIndex, Sats>>) -> Result<()>,
+    {
+        compute(self.sats.dateindex.um())?;
+
+        let dateindex: Option<&PcoVec<DateIndex, Sats>> = None;
+        self.compute_rest(price, starting_indexes, exit, dateindex)?;
+
+        Ok(())
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        dateindex: Option<&impl CollectableVec<DateIndex, Sats>>,
+    ) -> Result<()> {
+        if let Some(dateindex) = dateindex {
+            self.sats
+                .compute_rest(starting_indexes, exit, Some(dateindex))?;
+
+            self.bitcoin.compute_all(starting_indexes, exit, |v| {
+                v.compute_from_sats(starting_indexes.dateindex, dateindex, exit)
+            })?;
+        } else {
+            let dateindex: Option<&PcoVec<DateIndex, Sats>> = None;
+
+            self.sats.compute_rest(starting_indexes, exit, dateindex)?;
+
+            self.bitcoin.compute_all(starting_indexes, exit, |v| {
+                v.compute_from_sats(
+                    starting_indexes.dateindex,
+                    self.sats.dateindex.u(),
+                    exit,
+                )
+            })?;
+        }
+
+        let dateindex_to_bitcoin = self.bitcoin.dateindex.u();
+        let dateindex_to_price_close = price
+            .as_ref()
+            .unwrap()
+            .timeindexes_to_price_close
+            .dateindex
+            .as_ref()
+            .unwrap();
+
+        if let Some(dollars) = self.dollars.as_mut() {
+            dollars.compute_all(starting_indexes, exit, |v| {
+                v.compute_from_bitcoin(
+                    starting_indexes.dateindex,
+                    dateindex_to_bitcoin,
+                    dateindex_to_price_close,
+                    exit,
+                )
+            })?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/grouped/value_from_height.rs

@@ -0,0 +1,134 @@
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, Dollars, Height, Sats, Version};
+use vecdb::{CollectableVec, Database, EagerVec, Exit, PcoVec};
+
+use crate::{
+    Indexes,
+    grouped::Source,
+    indexes, price,
+    traits::{ComputeFromBitcoin, ComputeFromSats},
+    utils::OptionExt,
+};
+
+use super::{ComputedVecsFromHeight, VecBuilderOptions};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedValueVecsFromHeight {
+    pub sats: ComputedVecsFromHeight<Sats>,
+    pub bitcoin: ComputedVecsFromHeight<Bitcoin>,
+    pub dollars: Option<ComputedVecsFromHeight<Dollars>>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl ComputedValueVecsFromHeight {
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<Height, Sats>,
+        version: Version,
+        options: VecBuilderOptions,
+        compute_dollars: bool,
+        indexes: &indexes::Vecs,
+    ) -> Result<Self> {
+        Ok(Self {
+            sats: ComputedVecsFromHeight::forced_import(
+                db,
+                name,
+                source,
+                version + VERSION,
+                indexes,
+                options,
+            )?,
+            bitcoin: ComputedVecsFromHeight::forced_import(
+                db,
+                &format!("{name}_btc"),
+                Source::Compute,
+                version + VERSION,
+                indexes,
+                options,
+            )?,
+            dollars: compute_dollars.then(|| {
+                ComputedVecsFromHeight::forced_import(
+                    db,
+                    &format!("{name}_usd"),
+                    Source::Compute,
+                    version + VERSION,
+                    indexes,
+                    options,
+                )
+                .unwrap()
+            }),
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<Height, Sats>>) -> Result<()>,
+    {
+        compute(self.sats.height.um())?;
+
+        let height: Option<&PcoVec<Height, Sats>> = None;
+        self.compute_rest(indexes, price, starting_indexes, exit, height)?;
+
+        Ok(())
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        indexes: &indexes::Vecs,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        height: Option<&impl CollectableVec<Height, Sats>>,
+    ) -> Result<()> {
+        if let Some(height) = height {
+            self.sats
+                .compute_rest(indexes, starting_indexes, exit, Some(height))?;
+
+            self.bitcoin
+                .compute_all(indexes, starting_indexes, exit, |v| {
+                    v.compute_from_sats(starting_indexes.height, height, exit)
+                })?;
+        } else {
+            let height: Option<&PcoVec<Height, Sats>> = None;
+
+            self.sats
+                .compute_rest(indexes, starting_indexes, exit, height)?;
+
+            self.bitcoin
+                .compute_all(indexes, starting_indexes, exit, |v| {
+                    v.compute_from_sats(
+                        starting_indexes.height,
+                        self.sats.height.u(),
+                        exit,
+                    )
+                })?;
+        }
+
+        let height_to_bitcoin = self.bitcoin.height.u();
+        let height_to_price_close = &price.u().chainindexes_to_price_close.height;
+
+        if let Some(dollars) = self.dollars.as_mut() {
+            dollars.compute_all(indexes, starting_indexes, exit, |v| {
+                v.compute_from_bitcoin(
+                    starting_indexes.height,
+                    height_to_bitcoin,
+                    height_to_price_close,
+                    exit,
+                )
+            })?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/grouped/value_from_txindex.rs

@@ -0,0 +1,196 @@
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, Close, Dollars, Height, Sats, TxIndex, Version};
+use vecdb::{
+    CollectableVec, Database, Exit, IterableCloneableVec, LazyVecFrom1, LazyVecFrom3, PcoVec,
+    VecIndex,
+};
+
+use crate::{Indexes, grouped::Source, indexes, price, utils::OptionExt};
+
+use super::{ComputedVecsFromTxindex, VecBuilderOptions};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedValueVecsFromTxindex {
+    pub sats: ComputedVecsFromTxindex<Sats>,
+    pub bitcoin_txindex: LazyVecFrom1<TxIndex, Bitcoin, TxIndex, Sats>,
+    pub bitcoin: ComputedVecsFromTxindex<Bitcoin>,
+    #[allow(clippy::type_complexity)]
+    pub dollars_txindex: Option<
+        LazyVecFrom3<TxIndex, Dollars, TxIndex, Bitcoin, TxIndex, Height, Height, Close<Dollars>>,
+    >,
+    pub dollars: Option<ComputedVecsFromTxindex<Dollars>>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl ComputedValueVecsFromTxindex {
+    #[allow(clippy::too_many_arguments)]
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        source: Source<TxIndex, Sats>,
+        version: Version,
+        price: Option<&price::Vecs>,
+        options: VecBuilderOptions,
+    ) -> Result<Self> {
+        let compute_dollars = price.is_some();
+
+        let name_btc = format!("{name}_btc");
+        let name_usd = format!("{name}_usd");
+
+        let sats = ComputedVecsFromTxindex::forced_import(
+            db,
+            name,
+            source.clone(),
+            version + VERSION,
+            indexes,
+            options,
+        )?;
+
+        let source_vec = source.vec();
+
+        let bitcoin_txindex = LazyVecFrom1::init(
+            &name_btc,
+            version + VERSION,
+            source_vec.unwrap_or_else(|| sats.txindex.u().boxed_clone()),
+            |txindex: TxIndex, iter| iter.get_at(txindex.to_usize()).map(Bitcoin::from),
+        );
+
+        let bitcoin = ComputedVecsFromTxindex::forced_import(
+            db,
+            &name_btc,
+            Source::None,
+            version + VERSION,
+            indexes,
+            options,
+        )?;
+
+        let dollars_txindex = price.map(|price| {
+            LazyVecFrom3::init(
+                &name_usd,
+                version + VERSION,
+                bitcoin_txindex.boxed_clone(),
+                indexer.vecs.tx.txindex_to_height.boxed_clone(),
+                price.chainindexes_to_price_close.height.boxed_clone(),
+                |txindex: TxIndex,
+                 txindex_to_btc_iter,
+                 txindex_to_height_iter,
+                 height_to_price_close_iter| {
+                    let txindex = txindex.to_usize();
+                    txindex_to_btc_iter.get_at(txindex).and_then(|btc| {
+                        txindex_to_height_iter.get_at(txindex).and_then(|height| {
+                            height_to_price_close_iter
+                                .get_at(height.to_usize())
+                                .map(|close| *close * btc)
+                        })
+                    })
+                },
+            )
+        });
+
+        Ok(Self {
+            sats,
+            bitcoin_txindex,
+            bitcoin,
+            dollars_txindex,
+            dollars: compute_dollars.then(|| {
+                ComputedVecsFromTxindex::forced_import(
+                    db,
+                    &name_usd,
+                    Source::None,
+                    version + VERSION,
+                    indexes,
+                    options,
+                )
+                .unwrap()
+            }),
+        })
+    }
+
+    // pub fn compute_all<F>(
+    //     &mut self,
+    //     indexer: &Indexer,
+    //     indexes: &indexes::Vecs,
+    //     price: Option<&marketprice::Vecs>,
+    //     starting_indexes: &Indexes,
+    //     exit: &Exit,
+    //     mut compute: F,
+    // ) -> Result<()>
+    // where
+    //     F: FnMut(
+    //         &mut EagerVec<PcoVec<TxIndex, Sats>>,
+    //         &Indexer,
+    //         &indexes::Vecs,
+    //         &Indexes,
+    //         &Exit,
+    //     ) -> Result<()>,
+    // {
+    //     compute(
+    //         self.sats.txindex.um(),
+    //         indexer,
+    //         indexes,
+    //         starting_indexes,
+    //         exit,
+    //     )?;
+
+    //     let txindex: Option<&PcoVec<TxIndex, Sats>> = None;
+    //     self.compute_rest(
+    //         indexer,
+    //         indexes,
+    //         fetched,
+    //         starting_indexes,
+    //         exit,
+    //         txindex,
+    //     )?;
+
+    //     Ok(())
+    // }
+
+    pub fn compute_rest(
+        &mut self,
+        indexer: &Indexer,
+        indexes: &indexes::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        txindex: Option<&impl CollectableVec<TxIndex, Sats>>,
+        price: Option<&price::Vecs>,
+    ) -> Result<()> {
+        if let Some(txindex) = txindex {
+            self.sats
+                .compute_rest(indexer, indexes, starting_indexes, exit, Some(txindex))?;
+        } else {
+            let txindex: Option<&PcoVec<TxIndex, Sats>> = None;
+            self.sats
+                .compute_rest(indexer, indexes, starting_indexes, exit, txindex)?;
+        }
+
+        self.bitcoin.compute_rest_from_sats(
+            indexer,
+            indexes,
+            starting_indexes,
+            exit,
+            &self.sats,
+            Some(&self.bitcoin_txindex),
+        )?;
+
+        if let Some(dollars) = self.dollars.as_mut() {
+            let dollars_txindex = self.dollars_txindex.um();
+
+            dollars.compute_rest_from_bitcoin(
+                indexer,
+                indexes,
+                starting_indexes,
+                exit,
+                &self.bitcoin,
+                Some(dollars_txindex),
+                price.u(),
+            )?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/grouped/value_height.rs

@@ -0,0 +1,101 @@
+use brk_error::Result;
+use brk_traversable::Traversable;
+use brk_types::{Bitcoin, Dollars, Height, Sats, Version};
+use vecdb::{CollectableVec, Database, EagerVec, Exit, ImportableVec, PcoVec};
+
+use crate::{
+    Indexes,
+    grouped::Source,
+    price,
+    traits::{ComputeFromBitcoin, ComputeFromSats},
+    utils::OptionExt,
+};
+
+#[derive(Clone, Traversable)]
+pub struct ComputedHeightValueVecs {
+    pub sats: Option<EagerVec<PcoVec<Height, Sats>>>,
+    pub bitcoin: EagerVec<PcoVec<Height, Bitcoin>>,
+    pub dollars: Option<EagerVec<PcoVec<Height, Dollars>>>,
+}
+
+const VERSION: Version = Version::ZERO;
+
+impl ComputedHeightValueVecs {
+    pub fn forced_import(
+        db: &Database,
+        name: &str,
+        source: Source<Height, Sats>,
+        version: Version,
+        compute_dollars: bool,
+    ) -> Result<Self> {
+        Ok(Self {
+            sats: source.is_compute().then(|| {
+                EagerVec::forced_import(db, name, version + VERSION + Version::ZERO).unwrap()
+            }),
+            bitcoin: EagerVec::forced_import(
+                db,
+                &format!("{name}_btc"),
+                version + VERSION + Version::ZERO,
+            )?,
+            dollars: compute_dollars.then(|| {
+                EagerVec::forced_import(
+                    db,
+                    &format!("{name}_usd"),
+                    version + VERSION + Version::ZERO,
+                )
+                .unwrap()
+            }),
+        })
+    }
+
+    pub fn compute_all<F>(
+        &mut self,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        mut compute: F,
+    ) -> Result<()>
+    where
+        F: FnMut(&mut EagerVec<PcoVec<Height, Sats>>) -> Result<()>,
+    {
+        compute(self.sats.um())?;
+
+        let height: Option<&PcoVec<Height, Sats>> = None;
+        self.compute_rest(price, starting_indexes, exit, height)?;
+
+        Ok(())
+    }
+
+    pub fn compute_rest(
+        &mut self,
+        price: Option<&price::Vecs>,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+        height: Option<&impl CollectableVec<Height, Sats>>,
+    ) -> Result<()> {
+        if let Some(height) = height {
+            self.bitcoin
+                .compute_from_sats(starting_indexes.height, height, exit)?;
+        } else {
+            self.bitcoin.compute_from_sats(
+                starting_indexes.height,
+                self.sats.u(),
+                exit,
+            )?;
+        }
+
+        let height_to_bitcoin = &self.bitcoin;
+        let height_to_price_close = &price.u().chainindexes_to_price_close.height;
+
+        if let Some(dollars) = self.dollars.as_mut() {
+            dollars.compute_from_bitcoin(
+                starting_indexes.height,
+                height_to_bitcoin,
+                height_to_price_close,
+                exit,
+            )?;
+        }
+
+        Ok(())
+    }
+}

crates/brk_computer/src/indexes.rs

@@ -0,0 +1,709 @@
+use std::{ops::Deref, path::Path};
+
+use brk_error::Result;
+use brk_indexer::Indexer;
+use brk_traversable::Traversable;
+use brk_types::{
+    Date, DateIndex, DecadeIndex, DifficultyEpoch, EmptyOutputIndex, HalvingEpoch, Height,
+    MonthIndex, OpReturnIndex, OutPoint, P2AAddressIndex, P2ABytes, P2MSOutputIndex,
+    P2PK33AddressIndex, P2PK33Bytes, P2PK65AddressIndex, P2PK65Bytes, P2PKHAddressIndex,
+    P2PKHBytes, P2SHAddressIndex, P2SHBytes, P2TRAddressIndex, P2TRBytes, P2WPKHAddressIndex,
+    P2WPKHBytes, P2WSHAddressIndex, P2WSHBytes, QuarterIndex, Sats, SemesterIndex, StoredU64,
+    Timestamp, TxInIndex, TxIndex, TxOutIndex, Txid, UnknownOutputIndex, Version, WeekIndex,
+    YearIndex,
+};
+use vecdb::{
+    Database, EagerVec, Exit, ImportableVec, IterableCloneableVec, LazyVecFrom1, PAGE_SIZE, PcoVec,
+    TypedVecIterator,
+};
+
+const VERSION: Version = Version::ZERO;
+
+#[derive(Clone, Traversable)]
+pub struct Vecs {
+    db: Database,
+
+    pub dateindex_to_date: EagerVec<PcoVec<DateIndex, Date>>,
+    pub dateindex_to_dateindex: EagerVec<PcoVec<DateIndex, DateIndex>>,
+    pub dateindex_to_first_height: EagerVec<PcoVec<DateIndex, Height>>,
+    pub dateindex_to_height_count: EagerVec<PcoVec<DateIndex, StoredU64>>,
+    pub dateindex_to_monthindex: EagerVec<PcoVec<DateIndex, MonthIndex>>,
+    pub dateindex_to_weekindex: EagerVec<PcoVec<DateIndex, WeekIndex>>,
+    pub decadeindex_to_decadeindex: EagerVec<PcoVec<DecadeIndex, DecadeIndex>>,
+    pub decadeindex_to_first_yearindex: EagerVec<PcoVec<DecadeIndex, YearIndex>>,
+    pub decadeindex_to_yearindex_count: EagerVec<PcoVec<DecadeIndex, StoredU64>>,
+    pub difficultyepoch_to_difficultyepoch: EagerVec<PcoVec<DifficultyEpoch, DifficultyEpoch>>,
+    pub difficultyepoch_to_first_height: EagerVec<PcoVec<DifficultyEpoch, Height>>,
+    pub difficultyepoch_to_height_count: EagerVec<PcoVec<DifficultyEpoch, StoredU64>>,
+    pub emptyoutputindex_to_emptyoutputindex:
+        LazyVecFrom1<EmptyOutputIndex, EmptyOutputIndex, EmptyOutputIndex, TxIndex>,
+    pub halvingepoch_to_first_height: EagerVec<PcoVec<HalvingEpoch, Height>>,
+    pub halvingepoch_to_halvingepoch: EagerVec<PcoVec<HalvingEpoch, HalvingEpoch>>,
+    pub height_to_date: EagerVec<PcoVec<Height, Date>>,
+    pub height_to_date_fixed: EagerVec<PcoVec<Height, Date>>,
+    pub height_to_dateindex: EagerVec<PcoVec<Height, DateIndex>>,
+    pub height_to_difficultyepoch: EagerVec<PcoVec<Height, DifficultyEpoch>>,
+    pub height_to_halvingepoch: EagerVec<PcoVec<Height, HalvingEpoch>>,
+    pub height_to_height: EagerVec<PcoVec<Height, Height>>,
+    pub height_to_timestamp_fixed: EagerVec<PcoVec<Height, Timestamp>>,
+    pub height_to_txindex_count: EagerVec<PcoVec<Height, StoredU64>>,
+    pub monthindex_to_dateindex_count: EagerVec<PcoVec<MonthIndex, StoredU64>>,
+    pub monthindex_to_first_dateindex: EagerVec<PcoVec<MonthIndex, DateIndex>>,
+    pub monthindex_to_monthindex: EagerVec<PcoVec<MonthIndex, MonthIndex>>,
+    pub monthindex_to_quarterindex: EagerVec<PcoVec<MonthIndex, QuarterIndex>>,
+    pub monthindex_to_semesterindex: EagerVec<PcoVec<MonthIndex, SemesterIndex>>,
+    pub monthindex_to_yearindex: EagerVec<PcoVec<MonthIndex, YearIndex>>,
+    pub opreturnindex_to_opreturnindex:
+        LazyVecFrom1<OpReturnIndex, OpReturnIndex, OpReturnIndex, TxIndex>,
+    pub p2aaddressindex_to_p2aaddressindex:
+        LazyVecFrom1<P2AAddressIndex, P2AAddressIndex, P2AAddressIndex, P2ABytes>,
+    pub p2msoutputindex_to_p2msoutputindex:
+        LazyVecFrom1<P2MSOutputIndex, P2MSOutputIndex, P2MSOutputIndex, TxIndex>,
+    pub p2pk33addressindex_to_p2pk33addressindex:
+        LazyVecFrom1<P2PK33AddressIndex, P2PK33AddressIndex, P2PK33AddressIndex, P2PK33Bytes>,
+    pub p2pk65addressindex_to_p2pk65addressindex:
+        LazyVecFrom1<P2PK65AddressIndex, P2PK65AddressIndex, P2PK65AddressIndex, P2PK65Bytes>,
+    pub p2pkhaddressindex_to_p2pkhaddressindex:
+        LazyVecFrom1<P2PKHAddressIndex, P2PKHAddressIndex, P2PKHAddressIndex, P2PKHBytes>,
+    pub p2shaddressindex_to_p2shaddressindex:
+        LazyVecFrom1<P2SHAddressIndex, P2SHAddressIndex, P2SHAddressIndex, P2SHBytes>,
+    pub p2traddressindex_to_p2traddressindex:
+        LazyVecFrom1<P2TRAddressIndex, P2TRAddressIndex, P2TRAddressIndex, P2TRBytes>,
+    pub p2wpkhaddressindex_to_p2wpkhaddressindex:
+        LazyVecFrom1<P2WPKHAddressIndex, P2WPKHAddressIndex, P2WPKHAddressIndex, P2WPKHBytes>,
+    pub p2wshaddressindex_to_p2wshaddressindex:
+        LazyVecFrom1<P2WSHAddressIndex, P2WSHAddressIndex, P2WSHAddressIndex, P2WSHBytes>,
+    pub quarterindex_to_first_monthindex: EagerVec<PcoVec<QuarterIndex, MonthIndex>>,
+    pub quarterindex_to_monthindex_count: EagerVec<PcoVec<QuarterIndex, StoredU64>>,
+    pub quarterindex_to_quarterindex: EagerVec<PcoVec<QuarterIndex, QuarterIndex>>,
+    pub semesterindex_to_first_monthindex: EagerVec<PcoVec<SemesterIndex, MonthIndex>>,
+    pub semesterindex_to_monthindex_count: EagerVec<PcoVec<SemesterIndex, StoredU64>>,
+    pub semesterindex_to_semesterindex: EagerVec<PcoVec<SemesterIndex, SemesterIndex>>,
+    pub txindex_to_input_count: EagerVec<PcoVec<TxIndex, StoredU64>>,
+    pub txindex_to_output_count: EagerVec<PcoVec<TxIndex, StoredU64>>,
+    pub txindex_to_txindex: LazyVecFrom1<TxIndex, TxIndex, TxIndex, Txid>,
+    pub txinindex_to_txinindex: LazyVecFrom1<TxInIndex, TxInIndex, TxInIndex, OutPoint>,
+    pub txoutindex_to_txoutindex: LazyVecFrom1<TxOutIndex, TxOutIndex, TxOutIndex, Sats>,
+    pub unknownoutputindex_to_unknownoutputindex:
+        LazyVecFrom1<UnknownOutputIndex, UnknownOutputIndex, UnknownOutputIndex, TxIndex>,
+    pub weekindex_to_dateindex_count: EagerVec<PcoVec<WeekIndex, StoredU64>>,
+    pub weekindex_to_first_dateindex: EagerVec<PcoVec<WeekIndex, DateIndex>>,
+    pub weekindex_to_weekindex: EagerVec<PcoVec<WeekIndex, WeekIndex>>,
+    pub yearindex_to_decadeindex: EagerVec<PcoVec<YearIndex, DecadeIndex>>,
+    pub yearindex_to_first_monthindex: EagerVec<PcoVec<YearIndex, MonthIndex>>,
+    pub yearindex_to_monthindex_count: EagerVec<PcoVec<YearIndex, StoredU64>>,
+    pub yearindex_to_yearindex: EagerVec<PcoVec<YearIndex, YearIndex>>,
+}
+
+impl Vecs {
+    pub fn forced_import(
+        parent: &Path,
+        parent_version: Version,
+        indexer: &Indexer,
+    ) -> Result<Self> {
+        let db = Database::open(&parent.join("indexes"))?;
+        db.set_min_len(PAGE_SIZE * 10_000_000)?;
+
+        let version = parent_version + VERSION;
+
+        macro_rules! eager {
+            ($name:expr) => {
+                EagerVec::forced_import(&db, $name, version)?
+            };
+        }
+        macro_rules! lazy {
+            ($name:expr, $source:expr) => {
+                LazyVecFrom1::init($name, version, $source.boxed_clone(), |index, _| {
+                    Some(index)
+                })
+            };
+        }
+
+        let this = Self {
+            txoutindex_to_txoutindex: lazy!("txoutindex", indexer.vecs.txout.txoutindex_to_value),
+            txinindex_to_txinindex: lazy!("txinindex", indexer.vecs.txin.txinindex_to_outpoint),
+            p2pk33addressindex_to_p2pk33addressindex: lazy!(
+                "p2pk33addressindex",
+                indexer.vecs.address.p2pk33addressindex_to_p2pk33bytes
+            ),
+            p2pk65addressindex_to_p2pk65addressindex: lazy!(
+                "p2pk65addressindex",
+                indexer.vecs.address.p2pk65addressindex_to_p2pk65bytes
+            ),
+            p2pkhaddressindex_to_p2pkhaddressindex: lazy!(
+                "p2pkhaddressindex",
+                indexer.vecs.address.p2pkhaddressindex_to_p2pkhbytes
+            ),
+            p2shaddressindex_to_p2shaddressindex: lazy!(
+                "p2shaddressindex",
+                indexer.vecs.address.p2shaddressindex_to_p2shbytes
+            ),
+            p2traddressindex_to_p2traddressindex: lazy!(
+                "p2traddressindex",
+                indexer.vecs.address.p2traddressindex_to_p2trbytes
+            ),
+            p2wpkhaddressindex_to_p2wpkhaddressindex: lazy!(
+                "p2wpkhaddressindex",
+                indexer.vecs.address.p2wpkhaddressindex_to_p2wpkhbytes
+            ),
+            p2wshaddressindex_to_p2wshaddressindex: lazy!(
+                "p2wshaddressindex",
+                indexer.vecs.address.p2wshaddressindex_to_p2wshbytes
+            ),
+            p2aaddressindex_to_p2aaddressindex: lazy!(
+                "p2aaddressindex",
+                indexer.vecs.address.p2aaddressindex_to_p2abytes
+            ),
+            p2msoutputindex_to_p2msoutputindex: lazy!(
+                "p2msoutputindex",
+                indexer.vecs.output.p2msoutputindex_to_txindex
+            ),
+            emptyoutputindex_to_emptyoutputindex: lazy!(
+                "emptyoutputindex",
+                indexer.vecs.output.emptyoutputindex_to_txindex
+            ),
+            unknownoutputindex_to_unknownoutputindex: lazy!(
+                "unknownoutputindex",
+                indexer.vecs.output.unknownoutputindex_to_txindex
+            ),
+            opreturnindex_to_opreturnindex: lazy!(
+                "opreturnindex",
+                indexer.vecs.output.opreturnindex_to_txindex
+            ),
+            txindex_to_txindex: lazy!("txindex", indexer.vecs.tx.txindex_to_txid),
+            txindex_to_input_count: eager!("input_count"),
+            txindex_to_output_count: eager!("output_count"),
+            dateindex_to_date: eager!("date"),
+            dateindex_to_dateindex: eager!("dateindex"),
+            dateindex_to_first_height: eager!("first_height"),
+            dateindex_to_monthindex: eager!("monthindex"),
+            dateindex_to_weekindex: eager!("weekindex"),
+            decadeindex_to_decadeindex: eager!("decadeindex"),
+            decadeindex_to_first_yearindex: eager!("first_yearindex"),
+            difficultyepoch_to_difficultyepoch: eager!("difficultyepoch"),
+            difficultyepoch_to_first_height: eager!("first_height"),
+            halvingepoch_to_first_height: eager!("first_height"),
+            halvingepoch_to_halvingepoch: eager!("halvingepoch"),
+            height_to_date: eager!("date"),
+            height_to_difficultyepoch: eager!("difficultyepoch"),
+            height_to_halvingepoch: eager!("halvingepoch"),
+            height_to_height: eager!("height"),
+            monthindex_to_first_dateindex: eager!("first_dateindex"),
+            monthindex_to_monthindex: eager!("monthindex"),
+            monthindex_to_quarterindex: eager!("quarterindex"),
+            monthindex_to_semesterindex: eager!("semesterindex"),
+            monthindex_to_yearindex: eager!("yearindex"),
+            quarterindex_to_first_monthindex: eager!("first_monthindex"),
+            semesterindex_to_first_monthindex: eager!("first_monthindex"),
+            weekindex_to_first_dateindex: eager!("first_dateindex"),
+            yearindex_to_first_monthindex: eager!("first_monthindex"),
+            quarterindex_to_quarterindex: eager!("quarterindex"),
+            semesterindex_to_semesterindex: eager!("semesterindex"),
+            weekindex_to_weekindex: eager!("weekindex"),
+            yearindex_to_decadeindex: eager!("decadeindex"),
+            yearindex_to_yearindex: eager!("yearindex"),
+            height_to_date_fixed: eager!("date_fixed"),
+            height_to_dateindex: eager!("dateindex"),
+            height_to_timestamp_fixed: eager!("timestamp_fixed"),
+            height_to_txindex_count: eager!("txindex_count"),
+            dateindex_to_height_count: eager!("height_count"),
+            weekindex_to_dateindex_count: eager!("dateindex_count"),
+            difficultyepoch_to_height_count: eager!("height_count"),
+            monthindex_to_dateindex_count: eager!("dateindex_count"),
+            quarterindex_to_monthindex_count: eager!("monthindex_count"),
+            semesterindex_to_monthindex_count: eager!("monthindex_count"),
+            yearindex_to_monthindex_count: eager!("monthindex_count"),
+            decadeindex_to_yearindex_count: eager!("yearindex_count"),
+            db,
+        };
+
+        this.db.retain_regions(
+            this.iter_any_exportable()
+                .flat_map(|v| v.region_names())
+                .collect(),
+        )?;
+
+        this.db.compact()?;
+
+        Ok(this)
+    }
+
+    pub fn compute(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: brk_indexer::Indexes,
+        exit: &Exit,
+    ) -> Result<Indexes> {
+        let indexes = self.compute_(indexer, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(indexes)
+    }
+
+    fn compute_(
+        &mut self,
+        indexer: &Indexer,
+        starting_indexes: brk_indexer::Indexes,
+        exit: &Exit,
+    ) -> Result<Indexes> {
+        // ---
+        // TxIndex
+        // ---
+
+        self.txindex_to_input_count.compute_count_from_indexes(
+            starting_indexes.txindex,
+            &indexer.vecs.tx.txindex_to_first_txinindex,
+            &indexer.vecs.txin.txinindex_to_outpoint,
+            exit,
+        )?;
+
+        self.txindex_to_output_count.compute_count_from_indexes(
+            starting_indexes.txindex,
+            &indexer.vecs.tx.txindex_to_first_txoutindex,
+            &indexer.vecs.txout.txoutindex_to_value,
+            exit,
+        )?;
+
+        self.height_to_txindex_count.compute_count_from_indexes(
+            starting_indexes.height,
+            &indexer.vecs.tx.height_to_first_txindex,
+            &indexer.vecs.tx.txindex_to_txid,
+            exit,
+        )?;
+
+        // ---
+        // Height
+        // ---
+
+        self.height_to_height.compute_from_index(
+            starting_indexes.height,
+            &indexer.vecs.block.height_to_weight,
+            exit,
+        )?;
+
+        self.height_to_date.compute_transform(
+            starting_indexes.height,
+            &indexer.vecs.block.height_to_timestamp,
+            |(h, t, ..)| (h, Date::from(t)),
+            exit,
+        )?;
+
+        let mut prev_timestamp_fixed = None;
+        self.height_to_timestamp_fixed.compute_transform(
+            starting_indexes.height,
+            &indexer.vecs.block.height_to_timestamp,
+            |(h, timestamp, height_to_timestamp_fixed_iter)| {
+                if prev_timestamp_fixed.is_none()
+                    && let Some(prev_h) = h.decremented()
+                {
+                    prev_timestamp_fixed.replace(
+                        height_to_timestamp_fixed_iter
+                            .into_iter()
+                            .get_unwrap(prev_h),
+                    );
+                }
+                let timestamp_fixed =
+                    prev_timestamp_fixed.map_or(timestamp, |prev_d| prev_d.max(timestamp));
+                prev_timestamp_fixed.replace(timestamp_fixed);
+                (h, timestamp_fixed)
+            },
+            exit,
+        )?;
+
+        self.height_to_date_fixed.compute_transform(
+            starting_indexes.height,
+            &self.height_to_timestamp_fixed,
+            |(h, t, ..)| (h, Date::from(t)),
+            exit,
+        )?;
+
+        let decremented_starting_height = starting_indexes.height.decremented().unwrap_or_default();
+
+        // ---
+        // DateIndex
+        // ---
+
+        let starting_dateindex = self
+            .height_to_dateindex
+            .into_iter()
+            .get(decremented_starting_height)
+            .unwrap_or_default();
+
+        self.height_to_dateindex.compute_transform(
+            starting_indexes.height,
+            &self.height_to_date_fixed,
+            |(h, d, ..)| (h, DateIndex::try_from(d).unwrap()),
+            exit,
+        )?;
+
+        let starting_dateindex = if let Some(dateindex) = self
+            .height_to_dateindex
+            .into_iter()
+            .get(decremented_starting_height)
+        {
+            starting_dateindex.min(dateindex)
+        } else {
+            starting_dateindex
+        };
+
+        self.dateindex_to_first_height.compute_coarser(
+            starting_indexes.height,
+            &self.height_to_dateindex,
+            exit,
+        )?;
+
+        self.dateindex_to_dateindex.compute_from_index(
+            starting_dateindex,
+            &self.dateindex_to_first_height,
+            exit,
+        )?;
+
+        self.dateindex_to_date.compute_from_index(
+            starting_dateindex,
+            &self.dateindex_to_first_height,
+            exit,
+        )?;
+
+        self.dateindex_to_height_count.compute_count_from_indexes(
+            starting_dateindex,
+            &self.dateindex_to_first_height,
+            &indexer.vecs.block.height_to_weight,
+            exit,
+        )?;
+
+        // ---
+        // WeekIndex
+        // ---
+
+        let starting_weekindex = self
+            .dateindex_to_weekindex
+            .into_iter()
+            .get(starting_dateindex)
+            .unwrap_or_default();
+
+        self.dateindex_to_weekindex.compute_range(
+            starting_dateindex,
+            &self.dateindex_to_dateindex,
+            |i| (i, WeekIndex::from(i)),
+            exit,
+        )?;
+
+        self.weekindex_to_first_dateindex.compute_coarser(
+            starting_dateindex,
+            &self.dateindex_to_weekindex,
+            exit,
+        )?;
+
+        self.weekindex_to_weekindex.compute_from_index(
+            starting_weekindex,
+            &self.weekindex_to_first_dateindex,
+            exit,
+        )?;
+
+        self.weekindex_to_dateindex_count
+            .compute_count_from_indexes(
+                starting_weekindex,
+                &self.weekindex_to_first_dateindex,
+                &self.dateindex_to_date,
+                exit,
+            )?;
+
+        // ---
+        // DifficultyEpoch
+        // ---
+
+        let starting_difficultyepoch = self
+            .height_to_difficultyepoch
+            .into_iter()
+            .get(decremented_starting_height)
+            .unwrap_or_default();
+
+        self.height_to_difficultyepoch.compute_from_index(
+            starting_indexes.height,
+            &indexer.vecs.block.height_to_weight,
+            exit,
+        )?;
+
+        self.difficultyepoch_to_first_height.compute_coarser(
+            starting_indexes.height,
+            &self.height_to_difficultyepoch,
+            exit,
+        )?;
+
+        self.difficultyepoch_to_difficultyepoch.compute_from_index(
+            starting_difficultyepoch,
+            &self.difficultyepoch_to_first_height,
+            exit,
+        )?;
+
+        self.difficultyepoch_to_height_count
+            .compute_count_from_indexes(
+                starting_difficultyepoch,
+                &self.difficultyepoch_to_first_height,
+                &self.height_to_date,
+                exit,
+            )?;
+
+        // ---
+        // MonthIndex
+        // ---
+
+        let starting_monthindex = self
+            .dateindex_to_monthindex
+            .into_iter()
+            .get(starting_dateindex)
+            .unwrap_or_default();
+
+        self.dateindex_to_monthindex.compute_range(
+            starting_dateindex,
+            &self.dateindex_to_dateindex,
+            |i| (i, MonthIndex::from(i)),
+            exit,
+        )?;
+
+        self.monthindex_to_first_dateindex.compute_coarser(
+            starting_dateindex,
+            &self.dateindex_to_monthindex,
+            exit,
+        )?;
+
+        self.monthindex_to_monthindex.compute_from_index(
+            starting_monthindex,
+            &self.monthindex_to_first_dateindex,
+            exit,
+        )?;
+
+        self.monthindex_to_dateindex_count
+            .compute_count_from_indexes(
+                starting_monthindex,
+                &self.monthindex_to_first_dateindex,
+                &self.dateindex_to_date,
+                exit,
+            )?;
+
+        // ---
+        // QuarterIndex
+        // ---
+
+        let starting_quarterindex = self
+            .monthindex_to_quarterindex
+            .into_iter()
+            .get(starting_monthindex)
+            .unwrap_or_default();
+
+        self.monthindex_to_quarterindex.compute_from_index(
+            starting_monthindex,
+            &self.monthindex_to_first_dateindex,
+            exit,
+        )?;
+
+        self.quarterindex_to_first_monthindex.compute_coarser(
+            starting_monthindex,
+            &self.monthindex_to_quarterindex,
+            exit,
+        )?;
+
+        // let quarter_count = self.quarterindex_to_first_monthindex.len();
+
+        self.quarterindex_to_quarterindex.compute_from_index(
+            starting_quarterindex,
+            &self.quarterindex_to_first_monthindex,
+            exit,
+        )?;
+
+        self.quarterindex_to_monthindex_count
+            .compute_count_from_indexes(
+                starting_quarterindex,
+                &self.quarterindex_to_first_monthindex,
+                &self.monthindex_to_monthindex,
+                exit,
+            )?;
+
+        // ---
+        // SemesterIndex
+        // ---
+
+        let starting_semesterindex = self
+            .monthindex_to_semesterindex
+            .into_iter()
+            .get(starting_monthindex)
+            .unwrap_or_default();
+
+        self.monthindex_to_semesterindex.compute_from_index(
+            starting_monthindex,
+            &self.monthindex_to_first_dateindex,
+            exit,
+        )?;
+
+        self.semesterindex_to_first_monthindex.compute_coarser(
+            starting_monthindex,
+            &self.monthindex_to_semesterindex,
+            exit,
+        )?;
+
+        // let semester_count = self.semesterindex_to_first_monthindex.len();
+
+        self.semesterindex_to_semesterindex.compute_from_index(
+            starting_semesterindex,
+            &self.semesterindex_to_first_monthindex,
+            exit,
+        )?;
+
+        self.semesterindex_to_monthindex_count
+            .compute_count_from_indexes(
+                starting_semesterindex,
+                &self.semesterindex_to_first_monthindex,
+                &self.monthindex_to_monthindex,
+                exit,
+            )?;
+
+        // ---
+        // YearIndex
+        // ---
+
+        let starting_yearindex = self
+            .monthindex_to_yearindex
+            .into_iter()
+            .get(starting_monthindex)
+            .unwrap_or_default();
+
+        self.monthindex_to_yearindex.compute_from_index(
+            starting_monthindex,
+            &self.monthindex_to_first_dateindex,
+            exit,
+        )?;
+
+        self.yearindex_to_first_monthindex.compute_coarser(
+            starting_monthindex,
+            &self.monthindex_to_yearindex,
+            exit,
+        )?;
+
+        self.yearindex_to_yearindex.compute_from_index(
+            starting_yearindex,
+            &self.yearindex_to_first_monthindex,
+            exit,
+        )?;
+
+        self.yearindex_to_monthindex_count
+            .compute_count_from_indexes(
+                starting_yearindex,
+                &self.yearindex_to_first_monthindex,
+                &self.monthindex_to_monthindex,
+                exit,
+            )?;
+        // ---
+        // HalvingEpoch
+        // ---
+
+        let starting_halvingepoch = self
+            .height_to_halvingepoch
+            .into_iter()
+            .get(decremented_starting_height)
+            .unwrap_or_default();
+
+        self.height_to_halvingepoch.compute_from_index(
+            starting_indexes.height,
+            &indexer.vecs.block.height_to_weight,
+            exit,
+        )?;
+
+        self.halvingepoch_to_first_height.compute_coarser(
+            starting_indexes.height,
+            &self.height_to_halvingepoch,
+            exit,
+        )?;
+
+        self.halvingepoch_to_halvingepoch.compute_from_index(
+            starting_halvingepoch,
+            &self.halvingepoch_to_first_height,
+            exit,
+        )?;
+
+        // ---
+        // DecadeIndex
+        // ---
+
+        let starting_decadeindex = self
+            .yearindex_to_decadeindex
+            .into_iter()
+            .get(starting_yearindex)
+            .unwrap_or_default();
+
+        self.yearindex_to_decadeindex.compute_from_index(
+            starting_yearindex,
+            &self.yearindex_to_first_monthindex,
+            exit,
+        )?;
+
+        self.decadeindex_to_first_yearindex.compute_coarser(
+            starting_yearindex,
+            &self.yearindex_to_decadeindex,
+            exit,
+        )?;
+
+        self.decadeindex_to_decadeindex.compute_from_index(
+            starting_decadeindex,
+            &self.decadeindex_to_first_yearindex,
+            exit,
+        )?;
+
+        self.decadeindex_to_yearindex_count
+            .compute_count_from_indexes(
+                starting_decadeindex,
+                &self.decadeindex_to_first_yearindex,
+                &self.yearindex_to_yearindex,
+                exit,
+            )?;
+
+        Ok(Indexes {
+            indexes: starting_indexes,
+            dateindex: starting_dateindex,
+            weekindex: starting_weekindex,
+            monthindex: starting_monthindex,
+            quarterindex: starting_quarterindex,
+            semesterindex: starting_semesterindex,
+            yearindex: starting_yearindex,
+            decadeindex: starting_decadeindex,
+            difficultyepoch: starting_difficultyepoch,
+            halvingepoch: starting_halvingepoch,
+        })
+    }
+}
+
+#[derive(Debug, Clone)]
+pub struct Indexes {
+    indexes: brk_indexer::Indexes,
+    pub dateindex: DateIndex,
+    pub weekindex: WeekIndex,
+    pub monthindex: MonthIndex,
+    pub quarterindex: QuarterIndex,
+    pub semesterindex: SemesterIndex,
+    pub yearindex: YearIndex,
+    pub decadeindex: DecadeIndex,
+    pub difficultyepoch: DifficultyEpoch,
+    pub halvingepoch: HalvingEpoch,
+}
+
+impl Indexes {
+    pub fn update_from_height(&mut self, height: Height, indexes: &Vecs) {
+        self.indexes.height = height;
+        self.dateindex =
+            DateIndex::try_from(indexes.height_to_date_fixed.into_iter().get_unwrap(height))
+                .unwrap();
+        self.weekindex = WeekIndex::from(self.dateindex);
+        self.monthindex = MonthIndex::from(self.dateindex);
+        self.quarterindex = QuarterIndex::from(self.monthindex);
+        self.semesterindex = SemesterIndex::from(self.monthindex);
+        self.yearindex = YearIndex::from(self.monthindex);
+        self.decadeindex = DecadeIndex::from(self.dateindex);
+        self.difficultyepoch = DifficultyEpoch::from(self.height);
+        self.halvingepoch = HalvingEpoch::from(self.height);
+    }
+}
+
+impl Deref for Indexes {
+    type Target = brk_indexer::Indexes;
+    fn deref(&self) -> &Self::Target {
+        &self.indexes
+    }
+}

crates/brk_computer/src/lib.rs

@@ -1,104 +1,306 @@
 #![doc = include_str!("../README.md")]
-#![doc = "\n## Example\n\n```rust"]
-#![doc = include_str!("../examples/main.rs")]
-#![doc = "```"]
 
-use std::path::{Path, PathBuf};
+use std::{path::Path, thread, time::Instant};
 
-use brk_core::Version;
-use brk_exit::Exit;
+use brk_error::Result;
 use brk_fetcher::Fetcher;
 use brk_indexer::Indexer;
-use brk_vec::{AnyCollectableVec, Computation, Format};
-
-mod stores;
-mod utils;
-mod vecs;
-
+use brk_reader::Reader;
+use brk_traversable::Traversable;
+use brk_types::Version;
 use log::info;
-use stores::Stores;
-use vecs::Vecs;
+use vecdb::Exit;
 
-#[derive(Clone)]
+mod blks;
+mod chain;
+mod cointime;
+mod constants;
+mod fetched;
+mod grouped;
+mod indexes;
+mod market;
+mod pools;
+mod price;
+mod stateful;
+mod traits;
+mod utils;
+
+use indexes::Indexes;
+use utils::OptionExt;
+
+#[derive(Clone, Traversable)]
 pub struct Computer {
-    path: PathBuf,
-    fetcher: Option<Fetcher>,
-    vecs: Option<Vecs>,
-    stores: Option<Stores>,
-    format: Format,
+    pub blks: blks::Vecs,
+    pub chain: chain::Vecs,
+    pub cointime: cointime::Vecs,
+    pub constants: constants::Vecs,
+    pub fetched: Option<fetched::Vecs>,
+    pub indexes: indexes::Vecs,
+    pub market: market::Vecs,
+    pub pools: pools::Vecs,
+    pub price: Option<price::Vecs>,
+    pub stateful: stateful::Vecs,
 }
 
-const VERSION: Version = Version::ONE;
+const VERSION: Version = Version::new(4);
 
 impl Computer {
-    pub fn new(outputs_dir: &Path, fetcher: Option<Fetcher>, format: Format) -> Self {
-        Self {
-            path: outputs_dir.to_owned(),
-            fetcher,
-            vecs: None,
-            stores: None,
-            format,
-        }
-    }
-
-    pub fn import_vecs(
-        &mut self,
-        indexer: &Indexer,
-        computation: Computation,
-    ) -> color_eyre::Result<()> {
-        self.vecs = Some(Vecs::import(
-            // TODO: Give self.path, join inside import
-            &self.path.join("vecs/computed"),
-            VERSION + Version::ZERO,
-            indexer,
-            self.fetcher.is_some(),
-            computation,
-            self.format,
-        )?);
-        Ok(())
-    }
-
     /// Do NOT import multiple times or things will break !!!
-    /// Clone struct instead
-    pub fn import_stores(&mut self, indexer: &Indexer) -> color_eyre::Result<()> {
-        self.stores = Some(Stores::import(
-            // TODO: Give self.path, join inside import
-            &self.path.join("stores"),
-            VERSION + Version::ZERO,
-            indexer.keyspace(),
-        )?);
-        Ok(())
-    }
-}
+    pub fn forced_import(
+        outputs_path: &Path,
+        indexer: &Indexer,
+        fetcher: Option<Fetcher>,
+    ) -> Result<Self> {
+        info!("Importing computer...");
+        let import_start = Instant::now();
+
+        let computed_path = outputs_path.join("computed");
+
+        const STACK_SIZE: usize = 512 * 1024 * 1024;
+        let big_thread = || thread::Builder::new().stack_size(STACK_SIZE);
+
+        let i = Instant::now();
+        let (indexes, fetched, blks) = thread::scope(|s| -> Result<_> {
+            let fetched_handle = fetcher
+                .map(|fetcher| {
+                    big_thread().spawn_scoped(s, move || {
+                        fetched::Vecs::forced_import(outputs_path, fetcher, VERSION)
+                    })
+                })
+                .transpose()?;
+
+            let blks_handle = big_thread()
+                .spawn_scoped(s, || blks::Vecs::forced_import(&computed_path, VERSION))?;
+
+            let indexes = indexes::Vecs::forced_import(&computed_path, VERSION, indexer)?;
+            let fetched = fetched_handle.map(|h| h.join().unwrap()).transpose()?;
+            let blks = blks_handle.join().unwrap()?;
+
+            Ok((indexes, fetched, blks))
+        })?;
+        info!("Imported indexes/fetched/blks in {:?}", i.elapsed());
+
+        let i = Instant::now();
+        let (price, constants, market) = thread::scope(|s| -> Result<_> {
+            let constants_handle = big_thread().spawn_scoped(s, || {
+                constants::Vecs::forced_import(&computed_path, VERSION, &indexes)
+            })?;
+
+            let market_handle = big_thread().spawn_scoped(s, || {
+                market::Vecs::forced_import(&computed_path, VERSION, &indexes)
+            })?;
+
+            let price = fetched
+                .is_some()
+                .then(|| price::Vecs::forced_import(&computed_path, VERSION, &indexes).unwrap());
+
+            let constants = constants_handle.join().unwrap()?;
+            let market = market_handle.join().unwrap()?;
+
+            Ok((price, constants, market))
+        })?;
+        info!("Imported price/constants/market in {:?}", i.elapsed());
+
+        let i = Instant::now();
+        let (chain, pools, cointime) = thread::scope(|s| -> Result<_> {
+            let chain_handle = big_thread().spawn_scoped(s, || {
+                chain::Vecs::forced_import(
+                    &computed_path,
+                    VERSION,
+                    indexer,
+                    &indexes,
+                    price.as_ref(),
+                )
+            })?;
+
+            let pools_handle = big_thread().spawn_scoped(s, || {
+                pools::Vecs::forced_import(&computed_path, VERSION, &indexes, price.as_ref())
+            })?;
+
+            let cointime =
+                cointime::Vecs::forced_import(&computed_path, VERSION, &indexes, price.as_ref())?;
+
+            let chain = chain_handle.join().unwrap()?;
+            let pools = pools_handle.join().unwrap()?;
+
+            Ok((chain, pools, cointime))
+        })?;
+        info!("Imported chain/pools/cointime in {:?}", i.elapsed());
+
+        // Threads inside
+        let i = Instant::now();
+        let stateful =
+            stateful::Vecs::forced_import(&computed_path, VERSION, &indexes, price.as_ref())?;
+        info!("Imported stateful in {:?}", i.elapsed());
+
+        info!("Total import time: {:?}", import_start.elapsed());
+
+        Ok(Self {
+            constants,
+            market,
+            stateful,
+            chain,
+            blks,
+            pools,
+            cointime,
+            indexes,
+            fetched,
+            price,
+        })
+    }
 
-impl Computer {
     pub fn compute(
         &mut self,
-        indexer: &mut Indexer,
+        indexer: &Indexer,
         starting_indexes: brk_indexer::Indexes,
+        reader: &Reader,
         exit: &Exit,
-    ) -> color_eyre::Result<()> {
-        info!("Computing...");
-        self.vecs
-            .as_mut()
-            .unwrap()
-            .compute(indexer, starting_indexes, self.fetcher.as_mut(), exit)
-    }
+    ) -> Result<()> {
+        let compute_start = Instant::now();
+        info!("Computing indexes...");
+        let i = Instant::now();
+        let mut starting_indexes = self.indexes.compute(indexer, starting_indexes, exit)?;
+        info!("Computed indexes in {:?}", i.elapsed());
 
-    pub fn vecs(&self) -> Vec<&dyn AnyCollectableVec> {
-        // pub fn vecs(&self) -> &Vecs {
-        self.vecs.as_ref().unwrap().vecs()
-    }
+        if let Some(fetched) = self.fetched.as_mut() {
+            info!("Computing fetched...");
+            let i = Instant::now();
+            fetched.compute(indexer, &self.indexes, &starting_indexes, exit)?;
+            info!("Computed fetched in {:?}", i.elapsed());
 
-    // pub fn mut_vecs(&mut self) -> &mut Vecs {
-    //     self.vecs.as_mut().unwrap()
-    // }
+            info!("Computing prices...");
+            let i = Instant::now();
+            self.price
+                .um()
+                .compute(&self.indexes, &starting_indexes, fetched, exit)?;
+            info!("Computed prices in {:?}", i.elapsed());
+        }
 
-    pub fn stores(&self) -> &Stores {
-        self.stores.as_ref().unwrap()
-    }
+        // thread::scope(|scope| -> Result<()> {
+        // let blks = scope.spawn(|| -> Result<()> {
+        info!("Computing BLKs metadata...");
+        let i = Instant::now();
+        self.blks
+            .compute(indexer, &starting_indexes, reader, exit)?;
+        info!("Computed blk in {:?}", i.elapsed());
+        //     Ok(())
+        // });
 
-    pub fn mut_stores(&mut self) -> &mut Stores {
-        self.stores.as_mut().unwrap()
+        // let constants = scope.spawn(|| -> Result<()> {
+        info!("Computing constants...");
+        let i = Instant::now();
+        self.constants
+            .compute(&self.indexes, &starting_indexes, exit)?;
+        info!("Computed constants in {:?}", i.elapsed());
+        //     Ok(())
+        // });
+
+        // let chain = scope.spawn(|| -> Result<()> {
+        info!("Computing chain...");
+        let i = Instant::now();
+        self.chain.compute(
+            indexer,
+            &self.indexes,
+            &starting_indexes,
+            self.price.as_ref(),
+            exit,
+        )?;
+        info!("Computed chain in {:?}", i.elapsed());
+        //     Ok(())
+        // });
+
+        if let Some(price) = self.price.as_ref() {
+            info!("Computing market...");
+            let i = Instant::now();
+            self.market.compute(price, &starting_indexes, exit)?;
+            info!("Computed market in {:?}", i.elapsed());
+        }
+
+        // blks.join().unwrap()?;
+        // constants.join().unwrap()?;
+        // chain.join().unwrap()?;
+        // Ok(())
+        // })?;
+
+        let starting_indexes_clone = starting_indexes.clone();
+        // thread::scope(|scope| -> Result<()> {
+        // let pools = scope.spawn(|| -> Result<()> {
+        info!("Computing pools...");
+        let i = Instant::now();
+        self.pools.compute(
+            indexer,
+            &self.indexes,
+            &starting_indexes_clone,
+            &self.chain,
+            self.price.as_ref(),
+            exit,
+        )?;
+        info!("Computed pools in {:?}", i.elapsed());
+        //     Ok(())
+        // });
+
+        info!("Computing stateful...");
+        let i = Instant::now();
+        self.stateful.compute(
+            indexer,
+            &self.indexes,
+            &self.chain,
+            self.price.as_ref(),
+            &mut starting_indexes,
+            exit,
+        )?;
+        info!("Computed stateful in {:?}", i.elapsed());
+
+        //     pools.join().unwrap()?;
+        //     Ok(())
+        // })?;
+
+        info!("Computing cointime...");
+        let i = Instant::now();
+        self.cointime.compute(
+            &self.indexes,
+            &starting_indexes,
+            self.price.as_ref(),
+            &self.chain,
+            &self.stateful,
+            exit,
+        )?;
+        info!("Computed cointime in {:?}", i.elapsed());
+
+        info!("Total compute time: {:?}", compute_start.elapsed());
+        Ok(())
     }
 }
+
+// pub fn generate_allocation_files(monitored: &pools::Vecs) -> Result<()> {
+//     info!("Generating allocative files...");
+
+//     let mut flamegraph = allocative::FlameGraphBuilder::default();
+//     flamegraph.visit_root(monitored);
+//     let output = flamegraph.finish();
+
+//     let folder = format!(
+//         "at-{}",
+//         jiff::Timestamp::now().strftime("%Y-%m-%d_%Hh%Mm%Ss"),
+//     );
+
+//     let path = std::path::PathBuf::from(&format!("./target/flamegraph/{folder}"));
+//     std::fs::create_dir_all(&path)?;
+
+//     // fs::write(path.join("flamegraph.src"), &output.flamegraph())?;
+
+//     let mut fg_svg = Vec::new();
+//     inferno::flamegraph::from_reader(
+//         &mut inferno::flamegraph::Options::default(),
+//         output.flamegraph().write().as_bytes(),
+//         &mut fg_svg,
+//     )?;
+
+//     std::fs::write(path.join("flamegraph.svg"), &fg_svg)?;
+
+//     std::fs::write(path.join("warnings.txt"), output.warnings())?;
+
+//     info!("Successfully generate allocative files");
+
+//     Ok(())
+// }

crates/brk_computer/src/market/compute.rs

@@ -0,0 +1,807 @@
+use std::thread;
+
+use brk_error::Result;
+use brk_types::{Date, DateIndex, Dollars, StoredF32, StoredU16};
+use vecdb::{EagerVec, Exit, GenericStoredVec, PcoVec, TypedVecIterator, VecIndex};
+
+use crate::{
+    price,
+    traits::{ComputeDCAAveragePriceViaLen, ComputeDCAStackViaLen, ComputeDrawdown},
+    utils::OptionExt,
+    Indexes,
+};
+
+use super::Vecs;
+
+impl Vecs {
+    pub fn compute(
+        &mut self,
+        price: &price::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.compute_(price, starting_indexes, exit)?;
+        self.db.compact()?;
+        Ok(())
+    }
+
+    fn compute_(
+        &mut self,
+        price: &price::Vecs,
+        starting_indexes: &Indexes,
+        exit: &Exit,
+    ) -> Result<()> {
+        self.height_to_price_ath.compute_all_time_high(
+            starting_indexes.height,
+            &price.chainindexes_to_price_high.height,
+            exit,
+        )?;
+        self.height_to_price_drawdown.compute_drawdown(
+            starting_indexes.height,
+            &price.chainindexes_to_price_close.height,
+            &self.height_to_price_ath,
+            exit,
+        )?;
+
+        self.indexes_to_price_ath
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_all_time_high(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_high.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_drawdown
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_drawdown(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_close.dateindex.u(),
+                    self.indexes_to_price_ath.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_days_since_price_ath
+            .compute_all(starting_indexes, exit, |v| {
+                let mut high_iter = price.timeindexes_to_price_high.dateindex.u().into_iter();
+                let mut prev = None;
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_price_ath.dateindex.u(),
+                    |(i, ath, slf)| {
+                        if prev.is_none() {
+                            let i = i.to_usize();
+                            prev.replace(if i > 0 {
+                                slf.get_pushed_or_read_at_unwrap_once(i - 1)
+                            } else {
+                                StoredU16::default()
+                            });
+                        }
+                        let days = if *high_iter.get_unwrap(i) == ath {
+                            StoredU16::default()
+                        } else {
+                            prev.unwrap() + StoredU16::new(1)
+                        };
+                        prev.replace(days);
+                        (i, days)
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_max_days_between_price_aths
+            .compute_all(starting_indexes, exit, |v| {
+                let mut prev = None;
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_days_since_price_ath.dateindex.u(),
+                    |(i, days, slf)| {
+                        if prev.is_none() {
+                            let i = i.to_usize();
+                            prev.replace(if i > 0 {
+                                slf.get_pushed_or_read_at_unwrap_once(i - 1)
+                            } else {
+                                StoredU16::ZERO
+                            });
+                        }
+                        let max = prev.unwrap().max(days);
+                        prev.replace(max);
+                        (i, max)
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_max_years_between_price_aths
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_max_days_between_price_aths.dateindex.u(),
+                    |(i, max, ..)| (i, StoredF32::from(*max as f64 / 365.0)),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        [
+            (1, &mut self.price_1d_ago, &mut self._1d_price_returns, None),
+            (7, &mut self.price_1w_ago, &mut self._1w_price_returns, None),
+            (
+                30,
+                &mut self.price_1m_ago,
+                &mut self._1m_price_returns,
+                None,
+            ),
+            (
+                3 * 30,
+                &mut self.price_3m_ago,
+                &mut self._3m_price_returns,
+                None,
+            ),
+            (
+                6 * 30,
+                &mut self.price_6m_ago,
+                &mut self._6m_price_returns,
+                None,
+            ),
+            (
+                365,
+                &mut self.price_1y_ago,
+                &mut self._1y_price_returns,
+                None,
+            ),
+            (
+                2 * 365,
+                &mut self.price_2y_ago,
+                &mut self._2y_price_returns,
+                Some(&mut self._2y_cagr),
+            ),
+            (
+                3 * 365,
+                &mut self.price_3y_ago,
+                &mut self._3y_price_returns,
+                Some(&mut self._3y_cagr),
+            ),
+            (
+                4 * 365,
+                &mut self.price_4y_ago,
+                &mut self._4y_price_returns,
+                Some(&mut self._4y_cagr),
+            ),
+            (
+                5 * 365,
+                &mut self.price_5y_ago,
+                &mut self._5y_price_returns,
+                Some(&mut self._5y_cagr),
+            ),
+            (
+                6 * 365,
+                &mut self.price_6y_ago,
+                &mut self._6y_price_returns,
+                Some(&mut self._6y_cagr),
+            ),
+            (
+                8 * 365,
+                &mut self.price_8y_ago,
+                &mut self._8y_price_returns,
+                Some(&mut self._8y_cagr),
+            ),
+            (
+                10 * 365,
+                &mut self.price_10y_ago,
+                &mut self._10y_price_returns,
+                Some(&mut self._10y_cagr),
+            ),
+        ]
+        .into_iter()
+        .try_for_each(|(days, ago, returns, cagr)| -> Result<()> {
+            ago.compute_all(starting_indexes, exit, |v| {
+                v.compute_previous_value(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_close.dateindex.u(),
+                    days,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+            returns.compute_all(starting_indexes, exit, |v| {
+                v.compute_percentage_change(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_close.dateindex.u(),
+                    days,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+            if let Some(cagr) = cagr {
+                cagr.compute_all(starting_indexes, exit, |v| {
+                    v.compute_cagr(
+                        starting_indexes.dateindex,
+                        returns.dateindex.u(),
+                        days,
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+            }
+
+            Ok(())
+        })?;
+
+        [
+            (
+                7,
+                &mut self._1w_dca_stack,
+                &mut self._1w_dca_avg_price,
+                &mut self._1w_dca_returns,
+                None,
+            ),
+            (
+                30,
+                &mut self._1m_dca_stack,
+                &mut self._1m_dca_avg_price,
+                &mut self._1m_dca_returns,
+                None,
+            ),
+            (
+                3 * 30,
+                &mut self._3m_dca_stack,
+                &mut self._3m_dca_avg_price,
+                &mut self._3m_dca_returns,
+                None,
+            ),
+            (
+                6 * 30,
+                &mut self._6m_dca_stack,
+                &mut self._6m_dca_avg_price,
+                &mut self._6m_dca_returns,
+                None,
+            ),
+            (
+                365,
+                &mut self._1y_dca_stack,
+                &mut self._1y_dca_avg_price,
+                &mut self._1y_dca_returns,
+                None,
+            ),
+            (
+                2 * 365,
+                &mut self._2y_dca_stack,
+                &mut self._2y_dca_avg_price,
+                &mut self._2y_dca_returns,
+                Some(&mut self._2y_dca_cagr),
+            ),
+            (
+                3 * 365,
+                &mut self._3y_dca_stack,
+                &mut self._3y_dca_avg_price,
+                &mut self._3y_dca_returns,
+                Some(&mut self._3y_dca_cagr),
+            ),
+            (
+                4 * 365,
+                &mut self._4y_dca_stack,
+                &mut self._4y_dca_avg_price,
+                &mut self._4y_dca_returns,
+                Some(&mut self._4y_dca_cagr),
+            ),
+            (
+                5 * 365,
+                &mut self._5y_dca_stack,
+                &mut self._5y_dca_avg_price,
+                &mut self._5y_dca_returns,
+                Some(&mut self._5y_dca_cagr),
+            ),
+            (
+                6 * 365,
+                &mut self._6y_dca_stack,
+                &mut self._6y_dca_avg_price,
+                &mut self._6y_dca_returns,
+                Some(&mut self._6y_dca_cagr),
+            ),
+            (
+                8 * 365,
+                &mut self._8y_dca_stack,
+                &mut self._8y_dca_avg_price,
+                &mut self._8y_dca_returns,
+                Some(&mut self._8y_dca_cagr),
+            ),
+            (
+                10 * 365,
+                &mut self._10y_dca_stack,
+                &mut self._10y_dca_avg_price,
+                &mut self._10y_dca_returns,
+                Some(&mut self._10y_dca_cagr),
+            ),
+        ]
+        .into_iter()
+        .try_for_each(
+            |(days, dca_stack, dca_avg_price, dca_returns, dca_cagr)| -> Result<()> {
+                dca_stack.compute_all(starting_indexes, exit, |v| {
+                    v.compute_dca_stack_via_len(
+                        starting_indexes.dateindex,
+                        price.timeindexes_to_price_close.dateindex.u(),
+                        days,
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+                dca_avg_price.compute_all(starting_indexes, exit, |v| {
+                    v.compute_dca_avg_price_via_len(
+                        starting_indexes.dateindex,
+                        dca_stack.dateindex.u(),
+                        days,
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+                dca_returns.compute_all(starting_indexes, exit, |v| {
+                    v.compute_percentage_difference(
+                        starting_indexes.dateindex,
+                        price.timeindexes_to_price_close.dateindex.u(),
+                        dca_avg_price.dateindex.u(),
+                        exit,
+                    )?;
+                    Ok(())
+                })?;
+
+                if let Some(dca_cagr) = dca_cagr {
+                    dca_cagr.compute_all(starting_indexes, exit, |v| {
+                        v.compute_cagr(
+                            starting_indexes.dateindex,
+                            dca_returns.dateindex.u(),
+                            days,
+                            exit,
+                        )?;
+                        Ok(())
+                    })?;
+                }
+
+                Ok(())
+            },
+        )?;
+
+        [
+            (
+                2015,
+                &mut self.dca_class_2015_avg_price,
+                &mut self.dca_class_2015_returns,
+                &mut self.dca_class_2015_stack,
+            ),
+            (
+                2016,
+                &mut self.dca_class_2016_avg_price,
+                &mut self.dca_class_2016_returns,
+                &mut self.dca_class_2016_stack,
+            ),
+            (
+                2017,
+                &mut self.dca_class_2017_avg_price,
+                &mut self.dca_class_2017_returns,
+                &mut self.dca_class_2017_stack,
+            ),
+            (
+                2018,
+                &mut self.dca_class_2018_avg_price,
+                &mut self.dca_class_2018_returns,
+                &mut self.dca_class_2018_stack,
+            ),
+            (
+                2019,
+                &mut self.dca_class_2019_avg_price,
+                &mut self.dca_class_2019_returns,
+                &mut self.dca_class_2019_stack,
+            ),
+            (
+                2020,
+                &mut self.dca_class_2020_avg_price,
+                &mut self.dca_class_2020_returns,
+                &mut self.dca_class_2020_stack,
+            ),
+            (
+                2021,
+                &mut self.dca_class_2021_avg_price,
+                &mut self.dca_class_2021_returns,
+                &mut self.dca_class_2021_stack,
+            ),
+            (
+                2022,
+                &mut self.dca_class_2022_avg_price,
+                &mut self.dca_class_2022_returns,
+                &mut self.dca_class_2022_stack,
+            ),
+            (
+                2023,
+                &mut self.dca_class_2023_avg_price,
+                &mut self.dca_class_2023_returns,
+                &mut self.dca_class_2023_stack,
+            ),
+            (
+                2024,
+                &mut self.dca_class_2024_avg_price,
+                &mut self.dca_class_2024_returns,
+                &mut self.dca_class_2024_stack,
+            ),
+            (
+                2025,
+                &mut self.dca_class_2025_avg_price,
+                &mut self.dca_class_2025_returns,
+                &mut self.dca_class_2025_stack,
+            ),
+        ]
+        .into_iter()
+        .try_for_each(|(year, avg_price, returns, stack)| -> Result<()> {
+            let dateindex = DateIndex::try_from(Date::new(year, 1, 1)).unwrap();
+
+            stack.compute_all(starting_indexes, exit, |v| {
+                v.compute_dca_stack_via_from(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_close.dateindex.u(),
+                    dateindex,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+            avg_price.compute_all(starting_indexes, exit, |v| {
+                v.compute_dca_avg_price_via_from(
+                    starting_indexes.dateindex,
+                    stack.dateindex.u(),
+                    dateindex,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+            returns.compute_all(starting_indexes, exit, |v| {
+                v.compute_percentage_difference(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_close.dateindex.u(),
+                    avg_price.dateindex.u(),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+            Ok(())
+        })?;
+
+        thread::scope(|s| -> Result<()> {
+            [
+                (
+                    &mut self.indexes_to_price_1w_sma,
+                    &mut self.indexes_to_price_1w_ema,
+                    7,
+                ),
+                (
+                    &mut self.indexes_to_price_8d_sma,
+                    &mut self.indexes_to_price_8d_ema,
+                    8,
+                ),
+                (
+                    &mut self.indexes_to_price_13d_sma,
+                    &mut self.indexes_to_price_13d_ema,
+                    13,
+                ),
+                (
+                    &mut self.indexes_to_price_21d_sma,
+                    &mut self.indexes_to_price_21d_ema,
+                    21,
+                ),
+                (
+                    &mut self.indexes_to_price_1m_sma,
+                    &mut self.indexes_to_price_1m_ema,
+                    30,
+                ),
+                (
+                    &mut self.indexes_to_price_34d_sma,
+                    &mut self.indexes_to_price_34d_ema,
+                    34,
+                ),
+                (
+                    &mut self.indexes_to_price_55d_sma,
+                    &mut self.indexes_to_price_55d_ema,
+                    55,
+                ),
+                (
+                    &mut self.indexes_to_price_89d_sma,
+                    &mut self.indexes_to_price_89d_ema,
+                    89,
+                ),
+                (
+                    &mut self.indexes_to_price_144d_sma,
+                    &mut self.indexes_to_price_144d_ema,
+                    144,
+                ),
+                (
+                    &mut self.indexes_to_price_200d_sma,
+                    &mut self.indexes_to_price_200d_ema,
+                    200,
+                ),
+                (
+                    &mut self.indexes_to_price_1y_sma,
+                    &mut self.indexes_to_price_1y_ema,
+                    365,
+                ),
+                (
+                    &mut self.indexes_to_price_2y_sma,
+                    &mut self.indexes_to_price_2y_ema,
+                    2 * 365,
+                ),
+                (
+                    &mut self.indexes_to_price_200w_sma,
+                    &mut self.indexes_to_price_200w_ema,
+                    200 * 7,
+                ),
+                (
+                    &mut self.indexes_to_price_4y_sma,
+                    &mut self.indexes_to_price_4y_ema,
+                    4 * 365,
+                ),
+            ]
+            .into_iter()
+            .for_each(|(sma, ema, days)| {
+                s.spawn(move || -> Result<()> {
+                    sma.compute_all(price, starting_indexes, exit, |v| {
+                        v.compute_sma(
+                            starting_indexes.dateindex,
+                            price.timeindexes_to_price_close.dateindex.u(),
+                            days,
+                            exit,
+                        )?;
+                        Ok(())
+                    })?;
+
+                    ema.compute_all(price, starting_indexes, exit, |v| {
+                        v.compute_ema(
+                            starting_indexes.dateindex,
+                            price.timeindexes_to_price_close.dateindex.u(),
+                            days,
+                            exit,
+                        )?;
+                        Ok(())
+                    })
+                });
+            });
+            Ok(())
+        })?;
+
+        self.indexes_to_price_200d_sma_x0_8
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_price_200d_sma
+                        .price
+                        .as_ref()
+                        .unwrap()
+                        .dateindex
+                        .as_ref()
+                        .unwrap(),
+                    |(i, v, ..)| (i, v * 0.8),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_200d_sma_x2_4
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_price_200d_sma
+                        .price
+                        .as_ref()
+                        .unwrap()
+                        .dateindex
+                        .as_ref()
+                        .unwrap(),
+                    |(i, v, ..)| (i, v * 2.4),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_1d_returns_1w_sd.compute_all(
+            starting_indexes,
+            exit,
+            self._1d_price_returns.dateindex.u(),
+            None as Option<&EagerVec<PcoVec<DateIndex, Dollars>>>,
+        )?;
+        self.indexes_to_1d_returns_1m_sd.compute_all(
+            starting_indexes,
+            exit,
+            self._1d_price_returns.dateindex.u(),
+            None as Option<&EagerVec<PcoVec<DateIndex, Dollars>>>,
+        )?;
+        self.indexes_to_1d_returns_1y_sd.compute_all(
+            starting_indexes,
+            exit,
+            self._1d_price_returns.dateindex.u(),
+            None as Option<&EagerVec<PcoVec<DateIndex, Dollars>>>,
+        )?;
+
+        self.indexes_to_price_1w_volatility
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_1d_returns_1w_sd
+                        .sd
+                        .dateindex
+                        .as_ref()
+                        .unwrap(),
+                    |(i, v, ..)| (i, (*v * 7.0_f32.sqrt()).into()),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        self.indexes_to_price_1m_volatility
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_1d_returns_1m_sd
+                        .sd
+                        .dateindex
+                        .as_ref()
+                        .unwrap(),
+                    |(i, v, ..)| (i, (*v * 30.0_f32.sqrt()).into()),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+        self.indexes_to_price_1y_volatility
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_transform(
+                    starting_indexes.dateindex,
+                    self.indexes_to_1d_returns_1y_sd
+                        .sd
+                        .dateindex
+                        .as_ref()
+                        .unwrap(),
+                    |(i, v, ..)| (i, (*v * 365.0_f32.sqrt()).into()),
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.dateindex_to_price_true_range.compute_transform3(
+            starting_indexes.dateindex,
+            price.timeindexes_to_price_open.dateindex.u(),
+            price.timeindexes_to_price_high.dateindex.u(),
+            price.timeindexes_to_price_low.dateindex.u(),
+            |(i, open, high, low, ..)| {
+                let high_min_low = **high - **low;
+                let high_min_open = (**high - **open).abs();
+                let low_min_open = (**low - **open).abs();
+                (i, high_min_low.max(high_min_open).max(low_min_open).into())
+            },
+            exit,
+        )?;
+
+        self.dateindex_to_price_true_range_2w_sum.compute_sum(
+            starting_indexes.dateindex,
+            &self.dateindex_to_price_true_range,
+            14,
+            exit,
+        )?;
+
+        self.indexes_to_price_1w_max
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_max(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_high.dateindex.u(),
+                    7,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_1w_min
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_min(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_low.dateindex.u(),
+                    7,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_2w_max
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_max(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_high.dateindex.u(),
+                    14,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_2w_min
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_min(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_low.dateindex.u(),
+                    14,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_1m_max
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_max(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_high.dateindex.u(),
+                    30,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_1m_min
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_min(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_low.dateindex.u(),
+                    30,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_1y_max
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_max(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_high.dateindex.u(),
+                    365,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_1y_min
+            .compute_all(starting_indexes, exit, |v| {
+                v.compute_min(
+                    starting_indexes.dateindex,
+                    price.timeindexes_to_price_low.dateindex.u(),
+                    365,
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        self.indexes_to_price_2w_choppiness_index
+            .compute_all(starting_indexes, exit, |v| {
+                let n = 14;
+                let log10n = (n as f32).log10();
+                v.compute_transform3(
+                    starting_indexes.dateindex,
+                    &self.dateindex_to_price_true_range_2w_sum,
+                    self.indexes_to_price_2w_max.dateindex.u(),
+                    self.indexes_to_price_2w_min.dateindex.u(),
+                    |(i, tr_sum, max, min, ..)| {
+                        (
+                            i,
+                            StoredF32::from(
+                                100.0 * (*tr_sum / (*max - *min) as f32).log10() / log10n,
+                            ),
+                        )
+                    },
+                    exit,
+                )?;
+                Ok(())
+            })?;
+
+        Ok(())
+    }
+}