brk/crates/brk_binder/DESIGN.md

brk_binder Design Document

Goal

Generate typed API clients for Rust, TypeScript, and Python with:

• Discoverability: Full IDE autocomplete for 20k+ metrics
• Ease of use: Fluent API with .fetch() on each metric node

Current State

What Exists

1. js.rs: Generates compressed metric catalogs for JS (constants only, no HTTP client)
2. tree.rs: (kept for reference, not compiled) Brainstorming output for pattern extraction
3. generator/: Module structure for client generation
   • types.rs: Intermediate representation (ClientMetadata, MetricInfo, IndexPattern)
   • rust.rs: Rust client generation (stub)
   • typescript.rs: TypeScript client generation (stub)
   • python.rs: Python client generation (stub)

What's Missing

• HTTP client integration (.fetch() methods)
• OpenAPI as input source
• Rust client using brk_types instead of generating types
• Typed response types per metric

Target Architecture

Input Sources

┌─────────────────────────────────────────────────────────────┐
│                      Input Sources                          │
├─────────────────────────────────────────────────────────────┤
│  1. OpenAPI spec (from aide) - endpoint definitions         │
│  2. brk_query catalog - metric tree structure               │
│  3. brk_types - Rust types for responses (Rust client only) │
└─────────────────────────────────────────────────────────────┘

Output: Fluent Client (Option B)

// TypeScript
const client = new BrkClient("http://localhost:3000");
const data = await client.tree.supply.active.by_date.fetch();
//                        ^^^^ autocomplete all the way down

# Python
client = BrkClient("http://localhost:3000")
data = await client.tree.supply.active.by_date.fetch()

// Rust
let client = BrkClient::new("http://localhost:3000");
let data = client.tree.supply.active.by_date.fetch().await?;

Implementation Details

Smart Metric Nodes

Each tree leaf becomes a "smart node" holding a client reference:

// TypeScript
class MetricNode<T = unknown> {
  constructor(private client: BrkClient, private path: string) {}
  async fetch(): Promise<T> {
    return this.client.get<T>(this.path);
  }
}

# Python
class MetricNode(Generic[T]):
    def __init__(self, client: BrkClient, path: str):
        self._client = client
        self._path = path

    async def fetch(self) -> T:
        return await self._client.get(self._path)

// Rust
pub struct MetricNode<T> {
    client: Arc<BrkClient>,
    path: &'static str,
    _phantom: PhantomData<T>,
}

impl<T: DeserializeOwned> MetricNode<T> {
    pub async fn fetch(&self) -> Result<T, BrkError> {
        self.client.get(&self.path).await
    }
}

Pattern Reuse (from tree.rs)

To avoid 20k+ individual types, reuse structural patterns:

// Shared pattern for metrics with same index groupings
struct ByDateHeightMonth<T> {
    by_date: MetricNode<T>,
    by_height: MetricNode<T>,
    by_month: MetricNode<T>,
}

// Composed into full tree
struct Supply {
    active: ByDateHeightMonth<Vec<f64>>,
    total: ByDateHeightMonth<Vec<f64>>,
}

Rust Client: Using brk_types

The Rust client should import brk_types rather than generating duplicate types:

use brk_types::{Height, Sats, DateIndex, ...};

// Response types come from brk_types
pub struct MetricNode<T: brk_types::Metric> { ... }

Type Discovery Solution ✅ IMPLEMENTED

The Problem

Type information was erased at runtime because metrics are stored as &dyn AnyExportableVec trait objects.

The Solution

Use std::any::type_name::<T>() with caching to extract short type names.

Note

: Unlike PrintableIndex which needs to_possible_strings() for parsing from multiple string representations, for values we only need output, so type_name suffices.

Implementation (vecdb)

Added short_type_name<T>() helper in traits/printable.rs:

pub fn short_type_name<T: 'static>() -> &'static str {
    static CACHE: OnceLock<Mutex<HashMap<&'static str, &'static str>>> = OnceLock::new();

    let full: &'static str = std::any::type_name::<T>();
    // ... caching logic, extracts "Sats" from "brk_types::sats::Sats"
}

Added value_type_to_string() to AnyVec trait in traits/any.rs:

pub trait AnyVec: Send + Sync {
    // ... existing methods
    fn value_type_to_string(&self) -> &'static str;
}

Implemented in all vec variants:

• variants/eager/mod.rs
• variants/lazy/from1/mod.rs, from2/mod.rs, from3/mod.rs
• variants/raw/inner/mod.rs
• variants/compressed/inner/mod.rs
• variants/macros.rs (for wrapper types)

fn value_type_to_string(&self) -> &'static str {
    short_type_name::<V::T>()
}

No changes needed to brk_types - works automatically for all types.

Result

brk_query now exposes:

for (metric_name, index_to_vec) in &vecs.metric_to_index_to_vec {
    for (index, vec) in index_to_vec {
        println!("{} @ {} -> {}",
            metric_name,                    // "difficulty"
            vec.index_type_to_string(),     // "Height"
            vec.value_type_to_string(),     // "StoredF64"
        );
    }
}

This enables fully typed client generation.

TreeNode Enhancement ✅ IMPLEMENTED

The Problem

TreeNode::Leaf originally held just a String (the metric name), losing type and index information.

The Solution

Changed TreeNode::Leaf(String) to TreeNode::Leaf(MetricLeaf) where:

#[derive(Debug, Clone, Serialize, PartialEq, Eq, JsonSchema)]
pub struct MetricLeaf {
    pub name: String,
    pub value_type: String,
    pub indexes: BTreeSet<Index>,
}

Implementation

brk_types/src/treenode.rs:

• Added MetricLeaf struct with name, value_type, and indexes
• Added merge_indexes() method to union indexes when flattening tree
• Updated TreeNode enum to use Leaf(MetricLeaf)
• Updated merge logic to handle index merging

brk_traversable/src/lib.rs:

• Added make_leaf<I, V>() helper that creates MetricLeaf with proper fields
• Updated all Traversable::to_tree_node() implementations

Result

The catalog tree now includes full type information at each leaf:

TreeNode::Leaf(MetricLeaf {
    name: "difficulty".to_string(),
    value_type: "StoredF64".to_string(),
    indexes: btreeset![Index::Height, Index::Date],
})

When trees are merged/simplified, indexes are unioned together.

2. Async Runtime

• TypeScript: Native Promise
• Python: asyncio or sync variant?
• Rust: tokio assumed, or feature-flag for other runtimes?

3. Error Handling

• HTTP errors (4xx, 5xx)
• Deserialization errors
• Network errors
• Should errors be typed per language?

4. Additional Client Features

• Request timeout configuration
• Retry logic
• Rate limiting
• Caching
• Batch requests (fetch multiple metrics at once)

Tasks

Phase 0: Type Infrastructure ✅ COMPLETE

• vecdb: Add short_type_name<T>() helper in traits/printable.rs
• vecdb: Add value_type_to_string() to AnyVec trait
• vecdb: Implement in all vec variants (eager, lazy, raw, compressed, macros)
• brk_types: Enhance TreeNode::Leaf to include MetricLeaf with name, value_type, indexes
• brk_traversable: Update all to_tree_node() implementations to populate MetricLeaf
• brk_query: Export Vecs publicly for client generation
• brk_binder: Set up generator module structure (types, rust, typescript, python stubs)
• brk: Verify compilation

Phase 1: Client Foundation

• Define MetricNode<T> struct/class for each language
• Define BrkClient with base HTTP functionality
• Implement ClientMetadata::from_vecs() to extract metadata from brk_query::Vecs
• Client holds reference, nodes borrow it

Phase 2: Type-Aware Generation

• Create type mapping: value_type_string → Rust type / TS type / Python type
• Generate typed MetricNode<T> with correct T per metric
• For Rust: import from brk_types instead of generating

Phase 3: OpenAPI Integration

• Parse OpenAPI spec with openapiv3 crate
• Extract non-metric endpoint definitions
• Generate methods for health, info, catalog, etc.

Phase 4: Polish

• Error types per language
• Documentation generation
• Tests
• Example usage in each language

File Structure

crates/brk_binder/
├── src/
│   ├── lib.rs
│   ├── js.rs           # JS constants generation (existing)
│   ├── tree.rs         # Pattern extraction (reference only, not compiled)
│   └── generator/
│       ├── mod.rs
│       ├── types.rs    # ClientMetadata, MetricInfo, IndexPattern
│       ├── rust.rs     # Rust client generation
│       ├── typescript.rs
│       └── python.rs
├── Cargo.toml
├── README.md
└── DESIGN.md           # This file

Dependencies (Proposed)

[dependencies]
openapiv3 = "2"           # OpenAPI parsing
serde = { version = "1", features = ["derive"] }
serde_json = "1"
serde_yaml = "0.9"        # If parsing YAML specs
tera = "1"                # Optional: templating