brk/crates/brk_binder/DESIGN.md

brk_binder Design Document

Goal

Generate typed API clients for Rust, JavaScript, and Python with:

• Discoverability: Full IDE autocomplete for 20k+ metrics
• Ease of use: Fluent API with .fetch() on each metric node

Current State

What's Working ✅

1. JS + JSDoc generator: Generates client.js with full JSDoc type annotations
2. Python generator: Generates client.py with type hints and httpx
3. Rust generator: Generates client.rs with strong typing and reqwest
4. schemars integration: JSON schemas embedded in MetricLeafWithSchema for type info
5. Tree navigation: client.tree.blocks.difficulty.fetch() pattern
6. OpenAPI integration: All GET endpoints generate typed methods
7. Server integration: brk_server calls brk_binder on startup (when clients/ dir exists)

Generated Output

When crates/brk_binder/clients/ directory exists, running the server generates:

crates/brk_binder/clients/
├── javascript/
│   └── client.js      # JS + JSDoc with tree + API methods
├── python/
│   └── client.py      # Python with type hints + httpx
└── rust/
    └── client.rs      # Rust with reqwest + strong typing

Target Architecture

Input Sources

┌─────────────────────────────────────────────────────────────┐
│                      Input Sources                          │
├─────────────────────────────────────────────────────────────┤
│  1. OpenAPI spec (from aide) - endpoint definitions         │
│  2. brk_query catalog - metric tree structure               │
│  3. brk_types - Rust types for responses (Rust client only) │
└─────────────────────────────────────────────────────────────┘

Output: Fluent Client

// JavaScript (with JSDoc for IDE support)
const client = new BrkClient("http://localhost:3000");
const data = await client.tree.supply.active.by_date.fetch();
//                        ^^^^ autocomplete all the way down

# Python
client = BrkClient("http://localhost:3000")
data = client.tree.supply.active.by_date.fetch()

// Rust
let client = BrkClient::new("http://localhost:3000")?;
let data = client.tree().supply.active.by_date.fetch()?;

Implementation Details

Smart Metric Nodes

Each tree leaf becomes a "smart node" holding a client reference:

// JavaScript + JSDoc
/**
 * Metric node with fetch capability
 * @template T
 */
class MetricNode {
  constructor(client, path) {
    this._client = client;
    this._path = path;
  }

  async fetch() {
    return this._client.get(this._path);
  }
}

# Python
class MetricNode(Generic[T]):
    def __init__(self, client: BrkClientBase, path: str):
        self._client = client
        self._path = path

    def fetch(self) -> T:
        return self._client.get(self._path)

// Rust
pub struct MetricNode<'a, T> {
    client: &'a BrkClientBase,
    path: &'static str,
    _marker: PhantomData<T>,
}

impl<'a, T: DeserializeOwned> MetricNode<'a, T> {
    pub fn fetch(&self) -> Result<T> {
        self.client.get(self.path)
    }
}

Pattern Reuse

To avoid 20k+ individual types, reuse structural patterns:

// Shared pattern for metrics with same index groupings
struct ByDateHeightMonth<T> {
    by_date: MetricNode<T>,
    by_height: MetricNode<T>,
    by_month: MetricNode<T>,
}

// Composed into full tree
struct Supply {
    active: ByDateHeightMonth<Vec<f64>>,
    total: ByDateHeightMonth<Vec<f64>>,
}

Type Discovery Solution ✅ IMPLEMENTED

The Problem

Type information was erased at runtime because metrics are stored as &dyn AnyExportableVec trait objects.

The Solution

Use std::any::type_name::<T>() with caching to extract short type names.

Implementation (vecdb)

Added short_type_name<T>() helper and value_type_to_string() to AnyVec trait.

Result

brk_query now exposes:

for (metric_name, index_to_vec) in &vecs.metric_to_index_to_vec {
    for (index, vec) in index_to_vec {
        println!("{} @ {} -> {}",
            metric_name,                    // "difficulty"
            vec.index_type_to_string(),     // "Height"
            vec.value_type_to_string(),     // "StoredF64"
        );
    }
}

TreeNode Enhancement ✅ IMPLEMENTED

Changed TreeNode::Leaf(String) to TreeNode::Leaf(MetricLeafWithSchema) where:

#[derive(Debug, Clone, Serialize, JsonSchema)]
pub struct MetricLeafWithSchema {
    #[serde(flatten)]
    pub leaf: MetricLeaf,
    #[serde(skip)]
    pub schema: serde_json::Value,  // JSON Schema from schemars
}

OpenAPI Integration ✅ IMPLEMENTED

Flow

1. brk_server creates OpenAPI spec via aide
2. On startup, serializes spec to JSON string
3. Passes JSON to brk_binder::generate_clients()
4. brk_binder parses with oas3 crate (supports OpenAPI 3.1)
5. Generates typed methods for all GET endpoints

Why oas3?

aide generates OpenAPI 3.1 specs. The openapiv3 crate only supports 3.0.x. The oas3 crate supports OpenAPI 3.1.x parsing.

Tasks

Phase 0: Type Infrastructure ✅ COMPLETE

• vecdb: Add short_type_name<T>() and value_type_to_string()
• vecdb: Add optional schemars feature with AnySchemaVec trait
• brk_types: Enhance TreeNode::Leaf to include MetricLeafWithSchema
• brk_traversable: Update all to_tree_node() with schemars integration
• brk_binder: Set up generator module structure

Phase 1: JavaScript Client ✅ COMPLETE

• Define MetricNode class with JSDoc generics
• Define BrkClient with base HTTP functionality
• Generate client.js with full JSDoc type annotations
• Tree navigation: client.tree.category.metric.fetch()
• API methods from OpenAPI endpoints

Phase 2: OpenAPI Integration ✅ COMPLETE

• Add oas3 crate dependency (OpenAPI 3.1 support)
• brk_server passes OpenAPI JSON to brk_binder on startup
• Parse OpenAPI spec and extract endpoint definitions
• Generate typed methods for each GET endpoint

Phase 3: Python Client ✅ COMPLETE

• Define MetricNode class with type hints
• Define BrkClient with httpx
• Generate typed methods from OpenAPI
• Generate tree navigation

Phase 4: Rust Client ✅ COMPLETE

• Define MetricNode<T> struct with lifetimes
• Define BrkClient with reqwest (blocking)
• Generate tree navigation with proper lifetimes
• Generate typed methods from OpenAPI

Phase 5: Polish

• Switch from openapiv3 to oas3 crate
• Error types per language
• Documentation generation
• Tests
• Example usage in each language
• Async Rust client variant

File Structure

crates/brk_binder/
├── src/
│   ├── lib.rs
│   ├── js.rs              # JS constants generation (existing)
│   └── generator/
│       ├── mod.rs         # generate_clients() entry point
│       ├── types.rs       # ClientMetadata, MetricInfo, IndexPattern
│       ├── openapi.rs     # OpenAPI 3.1 spec parsing (oas3)
│       ├── javascript.rs  # JavaScript + JSDoc client ✅
│       ├── python.rs      # Python client ✅
│       └── rust.rs        # Rust client ✅
├── clients/               # Generated output (gitignored)
│   ├── javascript/
│   ├── python/
│   └── rust/
├── Cargo.toml
├── README.md
└── DESIGN.md

crates/brk_server/
└── src/
    ├── lib.rs             # Calls brk_binder::generate_clients() on startup
    └── api/
        └── openapi.rs     # create_openapi() for aide

Dependencies

[dependencies]
brk_query = { workspace = true }
brk_types = { workspace = true }
oas3 = "0.20"                # OpenAPI 3.1 spec parsing
schemars = { workspace = true }
serde_json = { workspace = true }

Usage

To generate clients:

# Create the output directory
mkdir -p crates/brk_binder/clients

# Run the server (generates clients on startup)
cargo run -p brk_server