Mirror/brk
1
0
Fork 0
mirror of https://github.com/bitcoinresearchkit/brk.git synced 2026-07-27 18:58:10 -07:00
Code Issues Packages Projects Releases Wiki Activity

global: snapshot

Browse Source
This commit is contained in:
nym21
2026-01-13 22:32:29 +01:00
parent 0c442b4a71
commit e77993fb76
61 changed files with 5047 additions and 5404 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/brk_bindgen/.gitignore
+4
View File
@@ -1 +1,5 @@
clients/
/*.json
/*.js
/*.rs
/*.py
crates/brk_bindgen/DESIGN.md
-296
View File
@@ -1,296 +0,0 @@
# brk_bindgen Design Document
## Goal
Generate typed API clients for **Rust, JavaScript, and Python** with:
- **Discoverability**: Full IDE autocomplete for 20k+ metrics
- **Ease of use**: Fluent API with `.fetch()` on each metric node
## Current State
### What's Working ✅
1. **JS + JSDoc generator**: Generates `client.js` with full JSDoc type annotations
2. **Python generator**: Generates `client.py` with type hints and httpx
3. **Rust generator**: Generates `client.rs` with strong typing and reqwest
4. **schemars integration**: JSON schemas embedded in `MetricLeafWithSchema` for type info
5. **Tree navigation**: `client.tree.blocks.difficulty.fetch()` pattern
6. **OpenAPI integration**: All GET endpoints generate typed methods
7. **Server integration**: brk_server calls brk_bindgen on startup (when clients/ dir exists)
### Generated Output
When `crates/brk_bindgen/clients/` directory exists, running the server generates:
```
crates/brk_bindgen/clients/
├── javascript/
│ └── client.js # JS + JSDoc with tree + API methods
├── python/
│ └── client.py # Python with type hints + httpx
└── rust/
└── client.rs # Rust with reqwest + strong typing
```
## Target Architecture
### Input Sources
```
┌─────────────────────────────────────────────────────────────┐
│ Input Sources │
├─────────────────────────────────────────────────────────────┤
│ 1. OpenAPI spec (from aide) - endpoint definitions │
│ 2. brk_query catalog - metric tree structure │
│ 3. brk_types - Rust types for responses (Rust client only) │
└─────────────────────────────────────────────────────────────┘
```
### Output: Fluent Client
```javascript
// JavaScript (with JSDoc for IDE support)
const client = new BrkClient("http://localhost:3000");
const data = await client.tree.supply.active.by_date.fetch();
// ^^^^ autocomplete all the way down
```
```python
# Python
client = BrkClient("http://localhost:3000")
data = client.tree.supply.active.by_date.fetch()
```
```rust
// Rust
let client = BrkClient::new("http://localhost:3000")?;
let data = client.tree().supply.active.by_date.fetch()?;
```
## Implementation Details
### Smart Metric Nodes
Each tree leaf becomes a "smart node" holding a client reference:
```javascript
// JavaScript + JSDoc
/**
* Metric node with fetch capability
* @template T
*/
class MetricNode {
constructor(client, path) {
this._client = client;
this._path = path;
}
async fetch() {
return this._client.get(this._path);
}
}
```
```python
# Python
class MetricNode(Generic[T]):
def __init__(self, client: BrkClientBase, path: str):
self._client = client
self._path = path
def fetch(self) -> T:
return self._client.get(self._path)
```
```rust
// Rust
pub struct MetricNode<'a, T> {
client: &'a BrkClientBase,
path: &'static str,
_marker: PhantomData<T>,
}
impl<'a, T: DeserializeOwned> MetricNode<'a, T> {
pub fn fetch(&self) -> Result<T> {
self.client.get(self.path)
}
}
```
### Pattern Reuse
To avoid 20k+ individual types, reuse structural patterns:
```rust
// Shared pattern for metrics with same index groupings
struct ByDateHeightMonth<T> {
by_date: MetricNode<T>,
by_height: MetricNode<T>,
by_month: MetricNode<T>,
}
// Composed into full tree
struct Supply {
active: ByDateHeightMonth<Vec<f64>>,
total: ByDateHeightMonth<Vec<f64>>,
}
```
## Type Discovery Solution ✅ IMPLEMENTED
### The Problem
Type information was erased at runtime because metrics are stored as `&dyn AnyExportableVec` trait objects.
### The Solution
Use `std::any::type_name::<T>()` with caching to extract short type names.
#### Implementation (vecdb)
Added `short_type_name<T>()` helper and `value_type_to_string()` to `AnyVec` trait.
### Result
`brk_query` now exposes:
```rust
for (metric_name, index_to_vec) in &vecs.metric_to_index_to_vec {
for (index, vec) in index_to_vec {
println!("{} @ {} -> {}",
metric_name, // "difficulty"
vec.index_type_to_string(), // "Height"
vec.value_type_to_string(), // "StoredF64"
);
}
}
```
## TreeNode Enhancement ✅ IMPLEMENTED
Changed `TreeNode::Leaf(String)` to `TreeNode::Leaf(MetricLeafWithSchema)` where:
```rust
#[derive(Debug, Clone, Serialize, JsonSchema)]
pub struct MetricLeafWithSchema {
#[serde(flatten)]
pub leaf: MetricLeaf,
#[serde(skip)]
pub schema: serde_json::Value, // JSON Schema from schemars
}
```
## OpenAPI Integration ✅ IMPLEMENTED
### Flow
1. brk_server creates OpenAPI spec via aide
2. On startup, serializes spec to JSON string
3. Passes JSON to `brk_bindgen::generate_clients()`
4. brk_bindgen parses with `oas3` crate (supports OpenAPI 3.1)
5. Generates typed methods for all GET endpoints
### Why oas3?
aide generates OpenAPI 3.1 specs. The `openapiv3` crate only supports 3.0.x.
The `oas3` crate supports OpenAPI 3.1.x parsing.
## Tasks
### Phase 0: Type Infrastructure ✅ COMPLETE
- [x] vecdb: Add `short_type_name<T>()` and `value_type_to_string()`
- [x] vecdb: Add optional `schemars` feature with `AnySchemaVec` trait
- [x] brk_types: Enhance `TreeNode::Leaf` to include `MetricLeafWithSchema`
- [x] brk_traversable: Update all `to_tree_node()` with schemars integration
- [x] brk_bindgen: Set up generator module structure
### Phase 1: JavaScript Client ✅ COMPLETE
- [x] Define `MetricNode` class with JSDoc generics
- [x] Define `BrkClient` with base HTTP functionality
- [x] Generate `client.js` with full JSDoc type annotations
- [x] Tree navigation: `client.tree.category.metric.fetch()`
- [x] API methods from OpenAPI endpoints
### Phase 2: OpenAPI Integration ✅ COMPLETE
- [x] Add `oas3` crate dependency (OpenAPI 3.1 support)
- [x] brk_server passes OpenAPI JSON to brk_bindgen on startup
- [x] Parse OpenAPI spec and extract endpoint definitions
- [x] Generate typed methods for each GET endpoint
### Phase 3: Python Client ✅ COMPLETE
- [x] Define `MetricNode` class with type hints
- [x] Define `BrkClient` with httpx
- [x] Generate typed methods from OpenAPI
- [x] Generate tree navigation
### Phase 4: Rust Client ✅ COMPLETE
- [x] Define `MetricNode<T>` struct with lifetimes
- [x] Define `BrkClient` with reqwest (blocking)
- [x] Generate tree navigation with proper lifetimes
- [x] Generate typed methods from OpenAPI
### Phase 5: Polish
- [x] Switch from `openapiv3` to `oas3` crate
- [ ] Error types per language
- [ ] Documentation generation
- [ ] Tests
- [ ] Example usage in each language
- [ ] Async Rust client variant
## File Structure
```
crates/brk_bindgen/
├── src/
│ ├── lib.rs
│ ├── js.rs # JS constants generation (existing)
│ └── generator/
│ ├── mod.rs # generate_clients() entry point
│ ├── types.rs # ClientMetadata, MetricInfo, IndexPattern
│ ├── openapi.rs # OpenAPI 3.1 spec parsing (oas3)
│ ├── javascript.rs # JavaScript + JSDoc client ✅
│ ├── python.rs # Python client ✅
│ └── rust.rs # Rust client ✅
├── clients/ # Generated output (gitignored)
│ ├── javascript/
│ ├── python/
│ └── rust/
├── Cargo.toml
├── README.md
└── DESIGN.md
crates/brk_server/
└── src/
├── lib.rs # Calls brk_bindgen::generate_clients() on startup
└── api/
└── openapi.rs # create_openapi() for aide
```
## Dependencies
```toml
[dependencies]
brk_query = { workspace = true }
brk_types = { workspace = true }
oas3 = "0.20" # OpenAPI 3.1 spec parsing
schemars = { workspace = true }
serde_json = { workspace = true }
```
## Usage
To generate clients:
```bash
# Create the output directory
mkdir -p crates/brk_bindgen/clients
# Run the server (generates clients on startup)
cargo run -p brk_server
```
crates/brk_bindgen/src/analysis/names.rs
+79 -386
View File
@@ -1,201 +1,17 @@
//! Vec name deconstruction and reconstruction logic.
//! Common prefix/suffix detection for metric names.
//!
//! This module analyzes vec names bottom-up to detect common denominators
//! (prefixes or suffixes) and field positions for pattern instances.
use std::collections::HashMap;
use crate::FieldNamePosition;
/// Common denominator found across children's effective names.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CommonDenominator {
/// Children share this prefix. Fields append their unique suffix.
/// Example: children are ["addrs_0sats", "addrs_1sats"], common = "addrs_"
Prefix(String),
/// Children share this suffix. Fields prepend their unique prefix.
/// Example: children are ["cumulative_supply", "net_supply"], common = "_supply"
Suffix(String),
/// No common part found. Fields use Identity (field = base).
None,
}
/// Result of analyzing a pattern level.
#[derive(Debug, Clone)]
pub struct PatternAnalysis {
/// The common prefix/suffix found across all children.
pub common: CommonDenominator,
/// What's left after stripping the common part (passed to parent).
pub base: String,
/// How each field modifies the accumulated name.
pub field_positions: HashMap<String, FieldNamePosition>,
}
/// Analyze a pattern level using child effective names.
///
/// This is the core algorithm that detects common prefix/suffix and
/// determines field positions for each child.
///
/// # Arguments
/// * `child_names` - Vec of (field_name, effective_name) pairs
/// where effective_name is either:
/// - For leaves: the leaf's vec name
/// - For branches: the base returned by analyzing that branch
pub fn analyze_pattern_level(child_names: &[(String, String)]) -> PatternAnalysis {
if child_names.is_empty() {
return PatternAnalysis {
common: CommonDenominator::None,
base: String::new(),
field_positions: HashMap::new(),
};
}
if child_names.len() == 1 {
let (field_name, effective) = &child_names[0];
let mut positions = HashMap::new();
// Try suffix match: effective ends with "_fieldname"
let suffix_pattern = format!("_{}", field_name);
if let Some(base) = effective.strip_suffix(&suffix_pattern) {
positions.insert(
field_name.clone(),
FieldNamePosition::Append(suffix_pattern),
);
return PatternAnalysis {
common: CommonDenominator::None,
base: base.to_string(),
field_positions: positions,
};
}
// Try prefix match: effective starts with "fieldname_"
let prefix_pattern = format!("{}_", field_name);
if let Some(base) = effective.strip_prefix(&prefix_pattern) {
positions.insert(
field_name.clone(),
FieldNamePosition::Prepend(prefix_pattern),
);
return PatternAnalysis {
common: CommonDenominator::None,
base: base.to_string(),
field_positions: positions,
};
}
// Field equals effective OR field doesn't appear → Identity
// Root-level instances where field == effective are handled by
// passing empty `acc` and conditional position expressions
positions.insert(field_name.clone(), FieldNamePosition::Identity);
return PatternAnalysis {
common: CommonDenominator::None,
base: effective.clone(),
field_positions: positions,
};
}
let effective_names: Vec<&str> = child_names.iter().map(|(_, n)| n.as_str()).collect();
// Try to find common prefix first
if let Some(prefix) = find_common_prefix(&effective_names)
&& !prefix.is_empty()
{
let base = prefix.trim_end_matches('_').to_string();
let mut positions = HashMap::new();
for (field_name, effective) in child_names {
// If effective equals the base (prefix without underscore), use Identity
if effective == &base {
positions.insert(field_name.clone(), FieldNamePosition::Identity);
} else if let Some(suffix) = effective.strip_prefix(&prefix) {
// Normal case: effective has the full prefix
let suffix_with_underscore = if suffix.starts_with('_') {
suffix.to_string()
} else {
format!("_{}", suffix)
};
positions.insert(
field_name.clone(),
FieldNamePosition::Append(suffix_with_underscore),
);
} else {
// Fallback: use Identity if strip_prefix fails unexpectedly
positions.insert(field_name.clone(), FieldNamePosition::Identity);
}
}
return PatternAnalysis {
common: CommonDenominator::Prefix(prefix),
base,
field_positions: positions,
};
}
// Try to find common suffix
if let Some(suffix) = find_common_suffix(&effective_names)
&& !suffix.is_empty()
{
let mut positions = HashMap::new();
for (field_name, effective) in child_names {
let prefix = effective
.strip_suffix(&suffix)
.unwrap_or(effective)
.to_string();
let prefix_with_underscore = if prefix.ends_with('_') {
prefix
} else {
format!("{}_", prefix)
};
positions.insert(
field_name.clone(),
FieldNamePosition::Prepend(prefix_with_underscore),
);
}
let base = suffix.trim_start_matches('_').to_string();
return PatternAnalysis {
common: CommonDenominator::Suffix(suffix),
base,
field_positions: positions,
};
}
// No common part - use Identity for all fields
let mut positions = HashMap::new();
for (field_name, _) in child_names {
positions.insert(field_name.clone(), FieldNamePosition::Identity);
}
// Check if all fields are "true Identity" (field_name == effective_name)
// In that case, the base should be empty since metrics are accessed directly by field name
let all_true_identity = child_names
.iter()
.all(|(field_name, effective)| field_name == effective);
let base = if all_true_identity {
String::new()
} else {
// Use the first name as base (they're all independent but have different names)
child_names
.first()
.map(|(_, n)| n.clone())
.unwrap_or_default()
};
PatternAnalysis {
common: CommonDenominator::None,
base,
field_positions: positions,
}
}
//! This module provides utilities to find common prefixes and suffixes
//! among metric names, which is used to detect pattern mode (suffix vs prefix).
/// Find the longest common prefix among all strings.
/// The prefix must end at an underscore boundary for semantic coherence.
fn find_common_prefix(names: &[&str]) -> Option<String> {
if names.is_empty() {
/// Returns the prefix WITH trailing underscore if found at word boundary.
/// Returns None if no common prefix exists.
pub fn find_common_prefix(names: &[&str]) -> Option<String> {
if names.is_empty() || names.iter().any(|n| n.is_empty()) {
return None;
}
let first = names[0];
if first.is_empty() {
return None;
}
// Find character-by-character common prefix
let mut prefix_len = 0;
@@ -213,48 +29,41 @@ fn find_common_prefix(names: &[&str]) -> Option<String> {
let raw_prefix = &first[..prefix_len];
// If raw_prefix exactly matches one of the names, it's a complete metric name.
// In this case, return it with trailing underscore to preserve the full name.
// Must end at underscore boundary for semantic coherence
if raw_prefix.ends_with('_') {
return Some(raw_prefix.to_string());
}
// If raw_prefix equals one of the full names (one name is a prefix of all others),
// return it with trailing underscore for proper base detection
if names.contains(&raw_prefix) {
return Some(format!("{}_", raw_prefix));
}
// Find the last underscore position to get a clean boundary
// Prefer ending at an underscore for semantic coherence
if let Some(last_underscore) = raw_prefix.rfind('_')
&& last_underscore > 0
{
// Find the last underscore position
if let Some(last_underscore) = raw_prefix.rfind('_') {
let clean_prefix = &first[..=last_underscore];
// Verify this still works for all names
if names.iter().all(|n| n.starts_with(clean_prefix)) {
return Some(clean_prefix.to_string());
}
}
// If no underscore boundary works, the full prefix must end at an underscore
if raw_prefix.ends_with('_') {
return Some(raw_prefix.to_string());
}
None
}
/// Find the longest common suffix among all strings.
/// The suffix must start at an underscore boundary for semantic coherence.
fn find_common_suffix(names: &[&str]) -> Option<String> {
if names.is_empty() {
/// Returns the suffix WITH leading underscore if found at word boundary.
/// Returns None if no common suffix exists.
pub fn find_common_suffix(names: &[&str]) -> Option<String> {
if names.is_empty() || names.iter().any(|n| n.is_empty()) {
return None;
}
let first = names[0];
if first.is_empty() {
return None;
}
let first_chars: Vec<char> = first.chars().collect();
// Find character-by-character common suffix (from the end)
let first_chars: Vec<char> = first.chars().collect();
let mut suffix_len = 0;
for i in 0..first_chars.len() {
let idx_from_end = first_chars.len() - 1 - i;
let ch = first_chars[idx_from_end];
@@ -280,22 +89,34 @@ fn find_common_suffix(names: &[&str]) -> Option<String> {
let raw_suffix = &first[first.len() - suffix_len..];
// Find the first underscore position to get a clean boundary
if let Some(first_underscore) = raw_suffix.find('_')
&& first_underscore < raw_suffix.len() - 1
{
// Must start at underscore boundary for semantic coherence
if raw_suffix.starts_with('_') {
return Some(raw_suffix.to_string());
}
// Check if preceded by underscore in all names (word boundary)
let at_word_boundary = names.iter().all(|n| {
if *n == raw_suffix {
true // Suffix is the whole string
} else if let Some(prefix) = n.strip_suffix(raw_suffix) {
prefix.ends_with('_')
} else {
false
}
});
if at_word_boundary {
return Some(format!("_{}", raw_suffix));
}
// Find the first underscore position in suffix
if let Some(first_underscore) = raw_suffix.find('_') {
let clean_suffix = &raw_suffix[first_underscore..];
// Verify this still works for all names
if names.iter().all(|n| n.ends_with(clean_suffix)) {
return Some(clean_suffix.to_string());
}
}
// If no underscore boundary works, the full suffix must start with underscore
if raw_suffix.starts_with('_') {
return Some(raw_suffix.to_string());
}
None
}
@@ -304,187 +125,59 @@ mod tests {
use super::*;
#[test]
fn test_common_prefix() {
fn test_common_prefix_basic() {
let names = vec!["addrs_0sats", "addrs_1sats", "addrs_2sats"];
assert_eq!(find_common_prefix(&names), Some("addrs_".to_string()));
}
#[test]
fn test_common_suffix() {
fn test_common_prefix_none() {
let names = vec!["foo", "bar", "baz"];
assert_eq!(find_common_prefix(&names), None);
}
#[test]
fn test_common_prefix_lth() {
let names = vec!["lth_cost_basis_max", "lth_cost_basis_min", "lth_cost_basis"];
assert_eq!(find_common_prefix(&names), Some("lth_cost_basis_".to_string()));
}
#[test]
fn test_common_suffix_basic() {
let names = vec!["cumulative_supply", "net_supply", "total_supply"];
assert_eq!(find_common_suffix(&names), Some("_supply".to_string()));
}
#[test]
fn test_no_common() {
fn test_common_prefix_cost_basis() {
// With suffix naming convention, cost_basis variants share a common prefix
let names = vec!["cost_basis_max", "cost_basis_min", "cost_basis"];
assert_eq!(find_common_prefix(&names), Some("cost_basis_".to_string()));
}
#[test]
fn test_common_suffix_none() {
let names = vec!["foo", "bar", "baz"];
assert_eq!(find_common_prefix(&names), None);
assert_eq!(find_common_suffix(&names), None);
}
#[test]
fn test_analyze_pattern_level_prefix() {
let children = vec![
("_0sats".to_string(), "addrs_0sats".to_string()),
("_1sats".to_string(), "addrs_1sats".to_string()),
fn test_common_prefix_one_is_prefix_of_other() {
// When one name is a prefix of another (block_count vs block_count_cumulative)
let names = vec!["block_count_cumulative", "block_count"];
assert_eq!(find_common_prefix(&names), Some("block_count_".to_string()));
}
#[test]
fn test_common_suffix_realized_loss() {
let names = vec![
"cumulative_realized_loss",
"net_realized_loss",
"realized_loss",
];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::Prefix(_)));
assert_eq!(analysis.base, "addrs");
assert!(matches!(
analysis.field_positions.get("_0sats"),
Some(FieldNamePosition::Append(_))
));
}
#[test]
fn test_analyze_pattern_level_suffix() {
let children = vec![
("cumulative".to_string(), "cumulative_supply".to_string()),
("net".to_string(), "net_supply".to_string()),
];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::Suffix(_)));
assert_eq!(analysis.base, "supply");
assert!(matches!(
analysis.field_positions.get("cumulative"),
Some(FieldNamePosition::Prepend(_))
));
}
#[test]
fn test_single_child_suffix() {
// Field "count" appears as suffix "_count" in "activity_count"
let children = vec![("count".to_string(), "activity_count".to_string())];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::None));
assert_eq!(analysis.base, "activity");
assert_eq!(
analysis.field_positions.get("count"),
Some(&FieldNamePosition::Append("_count".to_string()))
);
}
#[test]
fn test_single_child_prefix() {
// Field "cumulative" appears as prefix "cumulative_" in "cumulative_supply"
let children = vec![("cumulative".to_string(), "cumulative_supply".to_string())];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::None));
assert_eq!(analysis.base, "supply");
assert_eq!(
analysis.field_positions.get("cumulative"),
Some(&FieldNamePosition::Prepend("cumulative_".to_string()))
);
}
#[test]
fn test_single_child_identity_equal() {
// Field "supply" equals effective "supply" → Identity
// (root-level handling is done via empty acc and conditional expressions)
let children = vec![("supply".to_string(), "supply".to_string())];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::None));
assert_eq!(analysis.base, "supply");
assert_eq!(
analysis.field_positions.get("supply"),
Some(&FieldNamePosition::Identity)
);
}
#[test]
fn test_single_child_identity_structural() {
// Field "x" doesn't appear in "a_b" - it's structural grouping
let children = vec![("x".to_string(), "a_b".to_string())];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::None));
assert_eq!(analysis.base, "a_b"); // passes through unchanged
assert_eq!(
analysis.field_positions.get("x"),
Some(&FieldNamePosition::Identity)
);
}
#[test]
fn test_common_prefix_exact_match() {
// When one name exactly matches the common prefix, preserve the full name
// This fixes the realized_loss vs realized_count bug
let names = vec!["realized_loss", "realized_loss_cumulative"];
assert_eq!(
find_common_prefix(&names),
Some("realized_loss_".to_string())
);
}
#[test]
fn test_common_prefix_exact_match_multiple() {
// Multiple children with same base name
let names = vec!["realized_loss", "realized_loss", "realized_loss_cumulative"];
assert_eq!(
find_common_prefix(&names),
Some("realized_loss_".to_string())
);
}
#[test]
fn test_analyze_pattern_level_full_base() {
// When names are like [realized_loss, realized_loss_cumulative],
// base should be "realized_loss" not "realized"
let children = vec![
("sum".to_string(), "realized_loss".to_string()),
(
"cumulative".to_string(),
"realized_loss_cumulative".to_string(),
),
];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::Prefix(_)));
assert_eq!(analysis.base, "realized_loss");
// sum effective equals base, so position is Identity
assert_eq!(
analysis.field_positions.get("sum"),
Some(&FieldNamePosition::Identity)
);
// cumulative has suffix "_cumulative" after the base
assert_eq!(
analysis.field_positions.get("cumulative"),
Some(&FieldNamePosition::Append("_cumulative".to_string()))
);
}
#[test]
fn test_analyze_pattern_level_no_base_field() {
// When there's no base field (like block_weight which has no block_weight metric),
// only suffixed metrics like block_weight_average, block_weight_sum, etc.
// Base should still be "block_weight"
let children = vec![
("average".to_string(), "block_weight_average".to_string()),
("sum".to_string(), "block_weight_sum".to_string()),
(
"cumulative".to_string(),
"block_weight_cumulative".to_string(),
),
("max".to_string(), "block_weight_max".to_string()),
("min".to_string(), "block_weight_min".to_string()),
];
let analysis = analyze_pattern_level(&children);
assert!(matches!(analysis.common, CommonDenominator::Prefix(_)));
assert_eq!(analysis.base, "block_weight");
assert_eq!(
analysis.field_positions.get("average"),
Some(&FieldNamePosition::Append("_average".to_string()))
);
assert_eq!(
analysis.field_positions.get("sum"),
Some(&FieldNamePosition::Append("_sum".to_string()))
find_common_suffix(&names),
Some("_realized_loss".to_string())
);
}
}
crates/brk_bindgen/src/analysis/patterns.rs
+10 -6
View File
@@ -7,7 +7,7 @@ use std::collections::{BTreeSet, HashMap};
use brk_types::{TreeNode, extract_json_type};
use super::analyze_all_field_positions;
use super::analyze_pattern_modes;
use crate::{PatternField, StructuralPattern, to_pascal_case};
/// Context for pattern detection, holding all intermediate state.
@@ -39,6 +39,7 @@ impl PatternContext {
/// Detect structural patterns in the tree using a bottom-up approach.
///
/// Returns (patterns, concrete_to_pattern, concrete_to_type_param).
/// Each pattern has its `mode` set based on analysis of all instances.
pub fn detect_structural_patterns(
tree: &TreeNode,
) -> (
@@ -52,7 +53,9 @@ pub fn detect_structural_patterns(
let (generic_patterns, generic_mappings, type_mappings) =
detect_generic_patterns(&ctx.signature_to_pattern);
let mut patterns: Vec<StructuralPattern> = ctx.signature_to_pattern
// Only include patterns that appear 2+ times for the patterns list
let mut patterns: Vec<StructuralPattern> = ctx
.signature_to_pattern
.iter()
.filter(|(sig, _)| {
ctx.signature_counts.get(*sig).copied().unwrap_or(0) >= 2
@@ -76,7 +79,7 @@ pub fn detect_structural_patterns(
StructuralPattern {
name: name.clone(),
fields: fields_with_type_params,
field_positions: HashMap::new(),
mode: None, // Will be determined by analyze_pattern_modes
is_generic: false,
}
})
@@ -84,6 +87,7 @@ pub fn detect_structural_patterns(
patterns.extend(generic_patterns);
// Build pattern lookup for mode analysis (patterns appearing 2+ times)
let mut pattern_lookup: HashMap<Vec<PatternField>, String> = HashMap::new();
for (sig, name) in &ctx.signature_to_pattern {
if ctx.signature_counts.get(sig).copied().unwrap_or(0) >= 2 {
@@ -94,8 +98,8 @@ pub fn detect_structural_patterns(
let concrete_to_pattern = pattern_lookup.clone();
// Use the new bottom-up field position analysis
analyze_all_field_positions(tree, &mut patterns, &pattern_lookup);
// Analyze pattern modes (suffix vs prefix) from all instances
analyze_pattern_modes(tree, &mut patterns, &pattern_lookup);
patterns.sort_by(|a, b| b.fields.len().cmp(&a.fields.len()));
(patterns, concrete_to_pattern, type_mappings)
@@ -137,7 +141,7 @@ fn detect_generic_patterns(
patterns.push(StructuralPattern {
name: generic_name,
fields: normalized_fields,
field_positions: HashMap::new(),
mode: None, // Will be determined by analyze_pattern_modes
is_generic: true,
});
}
crates/brk_bindgen/src/analysis/positions.rs
+381 -90
View File
@@ -1,149 +1,440 @@
//! Field position detection for pattern instances.
//! Pattern mode detection and field part extraction.
//!
//! This module bridges the name analysis with pattern field positions,
//! processing patterns bottom-up to determine how each field modifies
//! the accumulated metric name.
//! This module analyzes pattern instances to detect whether they use
//! suffix mode (fields append to acc) or prefix mode (fields prepend to acc),
//! and extracts the field parts (relatives or prefixes) for code generation.
use std::collections::HashMap;
use brk_types::TreeNode;
use super::{analyze_pattern_level, get_node_fields};
use crate::{FieldNamePosition, PatternField, StructuralPattern};
use super::{find_common_prefix, find_common_suffix, get_node_fields};
use crate::{PatternField, PatternMode, StructuralPattern};
/// Analyze field positions for all patterns using bottom-up tree traversal.
/// Result of analyzing a single pattern instance.
#[derive(Debug, Clone)]
struct InstanceAnalysis {
/// The base to return to parent (used for nesting)
base: String,
/// For suffix mode: field -> relative name
/// For prefix mode: field -> prefix
field_parts: HashMap<String, String>,
/// Whether this instance appears to be suffix mode
is_suffix_mode: bool,
}
/// Analyze all pattern instances and determine their modes.
///
/// This is the main entry point for field position detection. It processes
/// the tree bottom-up, analyzing each pattern instance and aggregating
/// the positions across all instances.
pub fn analyze_all_field_positions(
/// This is the main entry point for mode detection. It processes
/// the tree bottom-up, collecting analysis for each pattern instance,
/// then determines the consistent mode for each pattern.
pub fn analyze_pattern_modes(
tree: &TreeNode,
patterns: &mut [StructuralPattern],
pattern_lookup: &HashMap<Vec<PatternField>, String>,
) {
let mut all_positions: HashMap<String, HashMap<String, Vec<FieldNamePosition>>> =
HashMap::new();
// Collect analyses from all instances, keyed by pattern name
let mut all_analyses: HashMap<String, Vec<InstanceAnalysis>> = HashMap::new();
// Collect positions from all instances bottom-up
collect_positions_bottom_up(tree, pattern_lookup, &mut all_positions);
// Bottom-up traversal
collect_instance_analyses(tree, pattern_lookup, &mut all_analyses);
// Merge positions into patterns
// For each pattern, determine mode from collected instances
for pattern in patterns.iter_mut() {
if let Some(field_positions) = all_positions.get(&pattern.name) {
pattern.field_positions = merge_field_positions(field_positions);
if let Some(analyses) = all_analyses.get(&pattern.name) {
pattern.mode = determine_pattern_mode(analyses, &pattern.fields);
}
}
}
/// Recursively collect field positions bottom-up.
/// Returns the effective base for this node (used by parent level).
fn collect_positions_bottom_up(
/// Recursively collect instance analyses bottom-up.
/// Returns the "base" for this node (used by parent for its analysis).
fn collect_instance_analyses(
node: &TreeNode,
pattern_lookup: &HashMap<Vec<PatternField>, String>,
all_positions: &mut HashMap<String, HashMap<String, Vec<FieldNamePosition>>>,
all_analyses: &mut HashMap<String, Vec<InstanceAnalysis>>,
) -> Option<String> {
match node {
TreeNode::Leaf(leaf) => {
// Leaves return their vec name as the effective base
// Leaves return their metric name as the base
Some(leaf.name().to_string())
}
TreeNode::Branch(children) => {
// First, process all children recursively (bottom-up)
let mut child_bases: HashMap<String, String> = HashMap::new();
for (field_name, child_node) in children {
if let Some(base) = collect_positions_bottom_up(child_node, pattern_lookup, all_positions) {
if let Some(base) =
collect_instance_analyses(child_node, pattern_lookup, all_analyses)
{
child_bases.insert(field_name.clone(), base);
}
}
// Build child names for this level's analysis
let child_names: Vec<(String, String)> = children
.keys()
.filter_map(|field_name| {
child_bases
.get(field_name)
.map(|base| (field_name.clone(), base.clone()))
})
.collect();
if child_names.is_empty() {
if child_bases.is_empty() {
return None;
}
// Analyze this level
let analysis = analyze_pattern_level(&child_names);
// Analyze this instance
let analysis = analyze_instance(&child_bases);
// Get the pattern name for this node (if any)
let fields = get_node_fields(children, pattern_lookup);
if let Some(pattern_name) = pattern_lookup.get(&fields) {
// Record field positions for this pattern instance
for (field_name, position) in &analysis.field_positions {
all_positions
.entry(pattern_name.clone())
.or_default()
.entry(field_name.clone())
.or_default()
.push(position.clone());
}
all_analyses
.entry(pattern_name.clone())
.or_default()
.push(analysis.clone());
}
// Return our base for the parent level
// Return the base for parent
Some(analysis.base)
}
}
}
/// Check if a list of positions contains incompatible values.
///
/// Positions are incompatible if there are multiple different non-Identity positions,
/// meaning different pattern instances use different naming conventions.
fn has_incompatible_positions(positions: &[FieldNamePosition]) -> bool {
let non_identity: Vec<_> = positions
.iter()
.filter(|p| !matches!(p, FieldNamePosition::Identity))
.collect();
/// Analyze a single pattern instance from its child bases.
fn analyze_instance(child_bases: &HashMap<String, String>) -> InstanceAnalysis {
let bases: Vec<&str> = child_bases.values().map(|s| s.as_str()).collect();
if non_identity.len() <= 1 {
return false;
// Try suffix mode first: look for common prefix among children
if let Some(common_prefix) = find_common_prefix(&bases) {
let base = common_prefix.trim_end_matches('_').to_string();
let mut field_parts = HashMap::new();
for (field_name, child_base) in child_bases {
// Relative = child_base with common prefix stripped
// If child_base equals base, relative is empty (identity field)
let relative = if child_base == &base {
String::new()
} else {
child_base
.strip_prefix(&common_prefix)
.unwrap_or(child_base)
.to_string()
};
field_parts.insert(field_name.clone(), relative);
}
return InstanceAnalysis {
base,
field_parts,
is_suffix_mode: true,
};
}
// Check if all non-identity positions are the same
let first = &non_identity[0];
non_identity.iter().skip(1).any(|p| p != first)
// Try prefix mode: look for common suffix among children
if let Some(common_suffix) = find_common_suffix(&bases) {
let base = common_suffix.trim_start_matches('_').to_string();
let mut field_parts = HashMap::new();
for (field_name, child_base) in child_bases {
// Prefix = child_base with common suffix stripped
let prefix = child_base
.strip_suffix(&common_suffix)
.map(|s| {
// Ensure prefix ends with underscore if non-empty
if s.is_empty() {
String::new()
} else if s.ends_with('_') {
s.to_string()
} else {
format!("{}_", s)
}
})
.unwrap_or_default();
field_parts.insert(field_name.clone(), prefix);
}
return InstanceAnalysis {
base,
field_parts,
is_suffix_mode: false,
};
}
// No common prefix or suffix - use first child's base and treat as suffix mode
// with full metric names as relatives
let base = child_bases.values().next().cloned().unwrap_or_default();
let field_parts = child_bases
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
InstanceAnalysis {
base,
field_parts,
is_suffix_mode: true,
}
}
/// Merge multiple observed positions for each field into a single position.
///
/// Returns an empty map if any field has incompatible positions across instances,
/// which will cause `is_parameterizable()` to return false for the pattern.
fn merge_field_positions(
field_positions: &HashMap<String, Vec<FieldNamePosition>>,
) -> HashMap<String, FieldNamePosition> {
// First check for incompatible positions
for positions in field_positions.values() {
if has_incompatible_positions(positions) {
// Incompatible positions found - pattern cannot be parameterized
return HashMap::new();
/// Determine the consistent mode for a pattern from all its instances.
/// Uses majority voting: if most instances agree on mode and field_parts,
/// use those. Minority instances will be inlined at usage sites.
fn determine_pattern_mode(
analyses: &[InstanceAnalysis],
fields: &[PatternField],
) -> Option<PatternMode> {
if analyses.is_empty() {
return None;
}
// Group instances by (mode, field_parts) signature
let suffix_instances: Vec<_> = analyses.iter().filter(|a| a.is_suffix_mode).collect();
let prefix_instances: Vec<_> = analyses.iter().filter(|a| !a.is_suffix_mode).collect();
// Pick the majority mode group
let (majority_instances, is_suffix) = if suffix_instances.len() >= prefix_instances.len() {
(suffix_instances, true)
} else {
(prefix_instances, false)
};
if majority_instances.is_empty() {
return None;
}
// Find the most common field_parts within the majority group
// Convert to sorted Vec for comparison since HashMap isn't hashable
let mut parts_counts: HashMap<Vec<(String, String)>, usize> = HashMap::new();
for analysis in &majority_instances {
let mut sorted: Vec<_> = analysis.field_parts.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
sorted.sort();
*parts_counts.entry(sorted).or_insert(0) += 1;
}
let (best_parts_vec, _count) = parts_counts.into_iter().max_by_key(|(_, count)| *count)?;
let best_parts: HashMap<String, String> = best_parts_vec.into_iter().collect();
// Verify all required fields have parts
for field in fields {
if !best_parts.contains_key(&field.name) {
return None;
}
}
// All positions are compatible, proceed with merge
field_positions
.iter()
.filter_map(|(field_name, positions)| {
if positions.is_empty() {
return None;
}
let field_parts = best_parts;
// Prefer Append/Prepend over Identity, as Identity at root-level
// is handled by empty acc and conditional position expressions
let preferred = positions
.iter()
.find(|p| !matches!(p, FieldNamePosition::Identity))
.cloned()
.unwrap_or_else(|| positions[0].clone());
Some((field_name.clone(), preferred))
if is_suffix {
Some(PatternMode::Suffix {
relatives: field_parts,
})
.collect()
} else {
Some(PatternMode::Prefix {
prefixes: field_parts,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_analyze_instance_suffix_mode() {
let mut child_bases = HashMap::new();
child_bases.insert("max".to_string(), "lth_cost_basis_max".to_string());
child_bases.insert("min".to_string(), "lth_cost_basis_min".to_string());
child_bases.insert("percentiles".to_string(), "lth_cost_basis".to_string());
let analysis = analyze_instance(&child_bases);
assert!(analysis.is_suffix_mode);
assert_eq!(analysis.base, "lth_cost_basis");
assert_eq!(analysis.field_parts.get("max"), Some(&"max".to_string()));
assert_eq!(analysis.field_parts.get("min"), Some(&"min".to_string()));
assert_eq!(analysis.field_parts.get("percentiles"), Some(&"".to_string()));
}
#[test]
fn test_analyze_instance_prefix_mode() {
// Period-prefixed metrics like "1y_lump_sum_stack", "1m_lump_sum_stack"
// share a common suffix "_lump_sum_stack" with different period prefixes
let mut child_bases = HashMap::new();
child_bases.insert("_1y".to_string(), "1y_lump_sum_stack".to_string());
child_bases.insert("_1m".to_string(), "1m_lump_sum_stack".to_string());
child_bases.insert("_1w".to_string(), "1w_lump_sum_stack".to_string());
let analysis = analyze_instance(&child_bases);
assert!(!analysis.is_suffix_mode);
assert_eq!(analysis.base, "lump_sum_stack");
assert_eq!(analysis.field_parts.get("_1y"), Some(&"1y_".to_string()));
assert_eq!(analysis.field_parts.get("_1m"), Some(&"1m_".to_string()));
assert_eq!(analysis.field_parts.get("_1w"), Some(&"1w_".to_string()));
}
#[test]
fn test_analyze_instance_root_suffix() {
// At root level with suffix naming convention
let mut child_bases = HashMap::new();
child_bases.insert("max".to_string(), "cost_basis_max".to_string());
child_bases.insert("min".to_string(), "cost_basis_min".to_string());
child_bases.insert("percentiles".to_string(), "cost_basis".to_string());
let analysis = analyze_instance(&child_bases);
// With suffix naming, common prefix is "cost_basis_" (since cost_basis is one of the names)
assert!(analysis.is_suffix_mode);
assert_eq!(analysis.base, "cost_basis");
assert_eq!(analysis.field_parts.get("max"), Some(&"max".to_string()));
assert_eq!(analysis.field_parts.get("min"), Some(&"min".to_string()));
assert_eq!(analysis.field_parts.get("percentiles"), Some(&"".to_string()));
}
#[test]
fn test_determine_pattern_mode_majority_voting() {
// Test that majority voting works when instances have mixed modes.
// This simulates CostBasisPattern2: most instances use suffix mode,
// but root-level uses prefix mode (max_cost_basis, min_cost_basis, cost_basis).
use std::collections::BTreeSet;
let fields = vec![
PatternField {
name: "max".to_string(),
rust_type: "TestType".to_string(),
json_type: "number".to_string(),
indexes: BTreeSet::new(),
type_param: None,
},
PatternField {
name: "min".to_string(),
rust_type: "TestType".to_string(),
json_type: "number".to_string(),
indexes: BTreeSet::new(),
type_param: None,
},
PatternField {
name: "percentiles".to_string(),
rust_type: "TestType".to_string(),
json_type: "number".to_string(),
indexes: BTreeSet::new(),
type_param: None,
},
];
// 3 suffix mode instances (majority)
let suffix1 = InstanceAnalysis {
base: "lth_cost_basis".to_string(),
field_parts: [
("max".to_string(), "max".to_string()),
("min".to_string(), "min".to_string()),
("percentiles".to_string(), "".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: true,
};
let suffix2 = InstanceAnalysis {
base: "sth_cost_basis".to_string(),
field_parts: [
("max".to_string(), "max".to_string()),
("min".to_string(), "min".to_string()),
("percentiles".to_string(), "".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: true,
};
let suffix3 = InstanceAnalysis {
base: "utxo_cost_basis".to_string(),
field_parts: [
("max".to_string(), "max".to_string()),
("min".to_string(), "min".to_string()),
("percentiles".to_string(), "".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: true,
};
// 1 prefix mode instance (minority - root level)
let prefix1 = InstanceAnalysis {
base: "cost_basis".to_string(),
field_parts: [
("max".to_string(), "max_".to_string()),
("min".to_string(), "min_".to_string()),
("percentiles".to_string(), "".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: false,
};
let analyses = vec![suffix1, suffix2, suffix3, prefix1];
let mode = determine_pattern_mode(&analyses, &fields);
// Should pick suffix mode (majority) with the common field_parts
assert!(mode.is_some());
match mode.unwrap() {
PatternMode::Suffix { relatives } => {
assert_eq!(relatives.get("max"), Some(&"max".to_string()));
assert_eq!(relatives.get("min"), Some(&"min".to_string()));
assert_eq!(relatives.get("percentiles"), Some(&"".to_string()));
}
PatternMode::Prefix { .. } => {
panic!("Expected suffix mode, got prefix mode");
}
}
}
#[test]
fn test_determine_pattern_mode_all_same() {
// Test when all instances agree on mode and field_parts
use std::collections::BTreeSet;
let fields = vec![
PatternField {
name: "max".to_string(),
rust_type: "TestType".to_string(),
json_type: "number".to_string(),
indexes: BTreeSet::new(),
type_param: None,
},
PatternField {
name: "min".to_string(),
rust_type: "TestType".to_string(),
json_type: "number".to_string(),
indexes: BTreeSet::new(),
type_param: None,
},
];
let instance1 = InstanceAnalysis {
base: "metric_a".to_string(),
field_parts: [
("max".to_string(), "max".to_string()),
("min".to_string(), "min".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: true,
};
let instance2 = InstanceAnalysis {
base: "metric_b".to_string(),
field_parts: [
("max".to_string(), "max".to_string()),
("min".to_string(), "min".to_string()),
]
.into_iter()
.collect(),
is_suffix_mode: true,
};
let analyses = vec![instance1, instance2];
let mode = determine_pattern_mode(&analyses, &fields);
assert!(mode.is_some());
match mode.unwrap() {
PatternMode::Suffix { relatives } => {
assert_eq!(relatives.get("max"), Some(&"max".to_string()));
assert_eq!(relatives.get("min"), Some(&"min".to_string()));
}
PatternMode::Prefix { .. } => {
panic!("Expected suffix mode");
}
}
}
}
crates/brk_bindgen/src/analysis/tree.rs
+80 -17
View File
@@ -7,7 +7,9 @@ use std::collections::{BTreeMap, BTreeSet, HashMap};
use brk_types::{Index, TreeNode, extract_json_type};
use crate::{IndexSetPattern, PatternField, analysis::names::{analyze_pattern_level, CommonDenominator}, child_type_name};
use crate::{IndexSetPattern, PatternField, child_type_name};
use super::{find_common_prefix, find_common_suffix};
/// Get the first leaf name from a tree node.
pub fn get_first_leaf_name(node: &TreeNode) -> Option<String> {
@@ -147,8 +149,7 @@ impl PatternBaseResult {
/// Get the metric base for a pattern instance by analyzing direct children.
///
/// Uses field names and first leaf names from direct children to determine
/// the common base via `analyze_pattern_level`.
/// Uses the shortest leaf names from direct children to find common prefix/suffix.
///
/// If the initial analysis fails to find a common pattern, it tries excluding
/// each child one at a time to detect outliers (e.g., a mismatched "base" field
@@ -164,18 +165,12 @@ pub fn get_pattern_instance_base(node: &TreeNode) -> PatternBaseResult {
};
}
let analysis = analyze_pattern_level(&child_names);
// If we found a common pattern, use it
if !matches!(analysis.common, CommonDenominator::None) {
return PatternBaseResult {
base: analysis.base,
has_outlier: false,
};
// Try to find common base from leaf names
if let Some((base, has_outlier)) = try_find_base(&child_names, false) {
return PatternBaseResult { base, has_outlier };
}
// If no common pattern found, try excluding each child one at a time
// to detect if there's a single outlier breaking the pattern.
// If no common pattern found and we have enough children, try excluding outliers
if child_names.len() > 2 {
for i in 0..child_names.len() {
let filtered: Vec<_> = child_names
@@ -185,22 +180,43 @@ pub fn get_pattern_instance_base(node: &TreeNode) -> PatternBaseResult {
.map(|(_, v)| v.clone())
.collect();
let filtered_analysis = analyze_pattern_level(&filtered);
if !matches!(filtered_analysis.common, CommonDenominator::None) {
if let Some((base, _)) = try_find_base(&filtered, true) {
return PatternBaseResult {
base: filtered_analysis.base,
base,
has_outlier: true,
};
}
}
}
// Fallback: no common prefix/suffix found - this is a root-level pattern
// Return empty base so metric names are used directly
PatternBaseResult {
base: analysis.base,
base: String::new(),
has_outlier: false,
}
}
/// Try to find a common base from child names using prefix/suffix detection.
/// Returns Some((base, has_outlier)) if found.
fn try_find_base(child_names: &[(String, String)], is_outlier_attempt: bool) -> Option<(String, bool)> {
let leaf_names: Vec<&str> = child_names.iter().map(|(_, n)| n.as_str()).collect();
// Try common prefix first (suffix mode)
if let Some(prefix) = find_common_prefix(&leaf_names) {
let base = prefix.trim_end_matches('_').to_string();
return Some((base, is_outlier_attempt));
}
// Try common suffix (prefix mode)
if let Some(suffix) = find_common_suffix(&leaf_names) {
let base = suffix.trim_start_matches('_').to_string();
return Some((base, is_outlier_attempt));
}
None
}
/// Get (field_name, shortest_leaf_name) pairs for direct children of a branch node.
///
/// Uses the shortest leaf name from each child subtree to find the "base" case
@@ -371,4 +387,51 @@ mod tests {
assert_eq!(result.base, "block_weight");
assert!(result.has_outlier); // Pattern factory should NOT be used
}
#[test]
fn test_get_pattern_instance_base_root_level_no_common_pattern() {
// Simulates root-level pattern with metrics that have no common prefix/suffix.
// These names have no shared prefix or suffix, even when excluding any one.
// In this case, we should return empty base so metric names are used directly.
let tree = make_branch(vec![
("alpha", make_leaf("foo_metric")),
("beta", make_leaf("bar_value")),
("gamma", make_leaf("baz_count")),
]);
let result = get_pattern_instance_base(&tree);
// No common prefix or suffix - return empty base
assert_eq!(result.base, "");
assert!(!result.has_outlier);
}
#[test]
fn test_get_pattern_instance_base_two_children_no_pattern() {
// Two children with no common pattern - should still return empty base
let tree = make_branch(vec![
("foo", make_leaf("alpha")),
("bar", make_leaf("beta")),
]);
let result = get_pattern_instance_base(&tree);
assert_eq!(result.base, "");
assert!(!result.has_outlier);
}
#[test]
fn test_get_pattern_instance_base_with_outlier_excluded() {
// Simulates the realized pattern: adjusted_sopr, sopr, asopr.
// When "asopr" is excluded as outlier, "adjusted_sopr" and "sopr" share suffix "_sopr".
// The outlier detection should find base="sopr" with has_outlier=true.
let tree = make_branch(vec![
("adjustedSopr", make_leaf("adjusted_sopr")),
("sopr", make_leaf("sopr")),
("asopr", make_leaf("asopr")),
]);
let result = get_pattern_instance_base(&tree);
// Outlier detected - pattern base found by excluding "asopr"
assert_eq!(result.base, "sopr");
assert!(result.has_outlier); // Pattern factory should NOT be used (inline instead)
}
}
crates/brk_bindgen/src/backends/javascript.rs
+21 -27
View File
@@ -1,6 +1,6 @@
//! JavaScript language syntax implementation.
use crate::{FieldNamePosition, GenericSyntax, LanguageSyntax, to_camel_case, to_pascal_case};
use crate::{GenericSyntax, LanguageSyntax, to_camel_case, to_pascal_case};
/// JavaScript-specific code generation syntax.
pub struct JavaScriptSyntax;
@@ -16,32 +16,26 @@ impl LanguageSyntax for JavaScriptSyntax {
format!("`${{{}}}{}`", var_name, suffix)
}
fn position_expr(&self, pos: &FieldNamePosition, base_var: &str) -> String {
// Convert base_var to camelCase for JavaScript
let var_name = to_camel_case(base_var);
match pos {
FieldNamePosition::Append(s) => {
// Use helper _m(acc, suffix) to build metric name
// e.g., _m(acc, "cap") produces: acc ? `${acc}_cap` : 'cap'
if let Some(suffix) = s.strip_prefix('_') {
format!("_m({}, '{}')", var_name, suffix)
} else {
format!("`${{{}}}{}`", var_name, s)
}
}
FieldNamePosition::Prepend(s) => {
// Handle empty acc case for prepend
if let Some(prefix) = s.strip_suffix('_') {
format!(
"({} ? `{}${{{}}}` : '{}')",
var_name, s, var_name, prefix
)
} else {
format!("`{}${{{}}}`", s, var_name)
}
}
FieldNamePosition::Identity => var_name,
FieldNamePosition::SetBase(s) => format!("'{}'", s),
fn suffix_expr(&self, acc_var: &str, relative: &str) -> String {
let var_name = to_camel_case(acc_var);
if relative.is_empty() {
// Identity: just return acc
var_name
} else {
// _m(acc, relative) -> acc ? `${acc}_relative` : 'relative'
format!("_m({}, '{}')", var_name, relative)
}
}
fn prefix_expr(&self, prefix: &str, acc_var: &str) -> String {
let var_name = to_camel_case(acc_var);
if prefix.is_empty() {
// Identity: just return acc
var_name
} else {
// _p(prefix, acc) -> acc ? `${prefix}${acc}` : 'prefix_without_underscore'
let prefix_base = prefix.trim_end_matches('_');
format!("_p('{}', {})", prefix_base, var_name)
}
}
crates/brk_bindgen/src/backends/python.rs
+19 -25
View File
@@ -1,6 +1,6 @@
//! Python language syntax implementation.
use crate::{FieldNamePosition, GenericSyntax, LanguageSyntax, escape_python_keyword, to_snake_case};
use crate::{GenericSyntax, LanguageSyntax, escape_python_keyword, to_snake_case};
/// Python-specific code generation syntax.
pub struct PythonSyntax;
@@ -14,30 +14,24 @@ impl LanguageSyntax for PythonSyntax {
format!("f'{{{}}}{}'", base_var, suffix)
}
fn position_expr(&self, pos: &FieldNamePosition, base_var: &str) -> String {
match pos {
FieldNamePosition::Append(s) => {
// Use helper _m(acc, suffix) to build metric name
if let Some(suffix) = s.strip_prefix('_') {
format!("_m({}, '{}')", base_var, suffix)
} else {
format!("f'{{{}}}{}'", base_var, s)
}
}
FieldNamePosition::Prepend(s) => {
// Handle empty acc case for prepend
// Want to produce: (f'prefix_{acc}' if acc else 'prefix')
if let Some(prefix) = s.strip_suffix('_') {
format!(
"(f'{}{{{}}}' if {} else '{}')",
s, base_var, base_var, prefix
)
} else {
format!("f'{}{{{}}}'" , s, base_var)
}
}
FieldNamePosition::Identity => base_var.to_string(),
FieldNamePosition::SetBase(s) => format!("'{}'", s),
fn suffix_expr(&self, acc_var: &str, relative: &str) -> String {
if relative.is_empty() {
// Identity: just return acc
acc_var.to_string()
} else {
// _m(acc, relative) -> f'{acc}_{relative}' if acc else 'relative'
format!("_m({}, '{}')", acc_var, relative)
}
}
fn prefix_expr(&self, prefix: &str, acc_var: &str) -> String {
if prefix.is_empty() {
// Identity: just return acc
acc_var.to_string()
} else {
// _p(prefix, acc) -> f'{prefix}{acc}' if acc else 'prefix_base'
let prefix_base = prefix.trim_end_matches('_');
format!("_p('{}', {})", prefix_base, acc_var)
}
}
crates/brk_bindgen/src/backends/rust.rs
+19 -25
View File
@@ -1,6 +1,6 @@
//! Rust language syntax implementation.
use crate::{FieldNamePosition, GenericSyntax, LanguageSyntax, to_snake_case};
use crate::{GenericSyntax, LanguageSyntax, to_snake_case};
/// Rust-specific code generation syntax.
pub struct RustSyntax;
@@ -14,30 +14,24 @@ impl LanguageSyntax for RustSyntax {
format!("format!(\"{{{}}}{}\")", base_var, suffix)
}
fn position_expr(&self, pos: &FieldNamePosition, _base_var: &str) -> String {
match pos {
FieldNamePosition::Append(s) => {
// Use helper _m(&acc, suffix) to build metric name
if let Some(suffix) = s.strip_prefix('_') {
format!("_m(&acc, \"{}\")", suffix)
} else {
format!("format!(\"{{acc}}{}\")", s)
}
}
FieldNamePosition::Prepend(s) => {
// Handle empty acc case for prepend
if let Some(prefix) = s.strip_suffix('_') {
format!(
"if acc.is_empty() {{ \"{prefix}\".to_string() }} else {{ format!(\"{s}{{acc}}\") }}",
prefix = prefix,
s = s
)
} else {
format!("format!(\"{}{{acc}}\")", s)
}
}
FieldNamePosition::Identity => "acc.clone()".to_string(),
FieldNamePosition::SetBase(base) => format!("\"{}\".to_string()", base),
fn suffix_expr(&self, acc_var: &str, relative: &str) -> String {
if relative.is_empty() {
// Identity: just return acc
format!("{}.clone()", acc_var)
} else {
// _m(&acc, relative) -> if acc.is_empty() { relative } else { format!("{acc}_{relative}") }
format!("_m(&{}, \"{}\")", acc_var, relative)
}
}
fn prefix_expr(&self, prefix: &str, acc_var: &str) -> String {
if prefix.is_empty() {
// Identity: just return acc
format!("{}.clone()", acc_var)
} else {
// _p(prefix, &acc) -> if acc.is_empty() { prefix_base } else { format!("{prefix}{acc}") }
let prefix_base = prefix.trim_end_matches('_');
format!("_p(\"{}\", &{})", prefix_base, acc_var)
}
}
crates/brk_bindgen/src/generate/fields.rs
+45 -35
View File
@@ -20,6 +20,46 @@ fn path_suffix(name: &str) -> String {
}
}
/// Compute path expression from pattern mode and field part.
fn compute_path_expr<S: LanguageSyntax>(
syntax: &S,
pattern: &StructuralPattern,
field: &PatternField,
base_var: &str,
) -> String {
match pattern.get_field_part(&field.name) {
Some(part) => {
if pattern.is_suffix_mode() {
syntax.suffix_expr(base_var, part)
} else {
syntax.prefix_expr(part, base_var)
}
}
None => syntax.path_expr(base_var, &path_suffix(&field.name)),
}
}
/// Compute field value from path expression.
fn compute_field_value<S: LanguageSyntax>(
syntax: &S,
field: &PatternField,
metadata: &ClientMetadata,
path_expr: &str,
) -> String {
if metadata.is_pattern_type(&field.rust_type) {
syntax.constructor(&field.rust_type, path_expr)
} else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
syntax.constructor(&accessor.name, path_expr)
} else if field.is_branch() {
syntax.constructor(&field.rust_type, path_expr)
} else {
panic!(
"Field '{}' has no matching pattern or index accessor. All metrics must be indexed.",
field.name
)
}
}
/// Generate a parameterized field using the language syntax.
///
/// This is used for pattern instances where fields use an accumulated
@@ -34,26 +74,8 @@ pub fn generate_parameterized_field<S: LanguageSyntax>(
) {
let field_name = syntax.field_name(&field.name);
let type_ann = metadata.field_type_annotation(field, pattern.is_generic, None, syntax.generic_syntax());
// Compute path expression from field position
let path_expr = pattern
.get_field_position(&field.name)
.map(|pos| syntax.position_expr(pos, "acc"))
.unwrap_or_else(|| syntax.path_expr("acc", &path_suffix(&field.name)));
let value = if metadata.is_pattern_type(&field.rust_type) {
syntax.constructor(&field.rust_type, &path_expr)
} else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
syntax.constructor(&accessor.name, &path_expr)
} else if field.is_branch() {
// Non-pattern branch - instantiate the nested struct
syntax.constructor(&field.rust_type, &path_expr)
} else {
panic!(
"Field '{}' has no matching pattern or index accessor. All metrics must be indexed.",
field.name
)
};
let path_expr = compute_path_expr(syntax, pattern, field, "acc");
let value = compute_field_value(syntax, field, metadata, &path_expr);
writeln!(output, "{}", syntax.field_init(indent, &field_name, &type_ann, &value)).unwrap();
}
@@ -66,26 +88,14 @@ pub fn generate_tree_path_field<S: LanguageSyntax>(
output: &mut String,
syntax: &S,
field: &PatternField,
pattern: &StructuralPattern,
metadata: &ClientMetadata,
indent: &str,
) {
let field_name = syntax.field_name(&field.name);
let type_ann = metadata.field_type_annotation(field, false, None, syntax.generic_syntax());
let path_expr = syntax.path_expr("base_path", &path_suffix(&field.name));
let value = if metadata.is_pattern_type(&field.rust_type) {
syntax.constructor(&field.rust_type, &path_expr)
} else if let Some(accessor) = metadata.find_index_set_pattern(&field.indexes) {
syntax.constructor(&accessor.name, &path_expr)
} else if field.is_branch() {
// Non-pattern branch - instantiate the nested struct
syntax.constructor(&field.rust_type, &path_expr)
} else {
panic!(
"Field '{}' has no matching pattern or index accessor. All metrics must be indexed.",
field.name
)
};
let path_expr = compute_path_expr(syntax, pattern, field, "base_path");
let value = compute_field_value(syntax, field, metadata, &path_expr);
writeln!(output, "{}", syntax.field_init(indent, &field_name, &type_ann, &value)).unwrap();
}
crates/brk_bindgen/src/generate/tree.rs
+15 -3
View File
@@ -23,10 +23,12 @@ pub struct ChildContext<'a> {
pub base_result: PatternBaseResult,
/// Whether this is a leaf node.
pub is_leaf: bool,
/// Whether to use an inline type instead of a pattern factory (only meaningful for branches).
/// Whether to use an inline type instead of a pattern type (only meaningful for branches).
pub should_inline: bool,
/// The type name to use for inline branches.
pub inline_type_name: String,
/// Whether the pattern is parameterizable (has ::new() constructor).
pub is_parameterizable: bool,
}
/// Context for generating a tree node, returned by `prepare_tree_node`.
@@ -78,11 +80,20 @@ pub fn prepare_tree_node<'a>(
.map(|((child_name, child_node), (field, child_fields))| {
let is_leaf = matches!(child_node, TreeNode::Leaf(_));
let base_result = get_pattern_instance_base(child_node);
// For type annotations: use pattern type if ANY pattern matches
let matches_any_pattern = child_fields
.as_ref()
.is_some_and(|cf| metadata.matches_pattern(cf));
// For constructors: only use ::new() if parameterizable
let is_parameterizable = child_fields
.as_ref()
.is_some_and(|cf| metadata.is_parameterizable_fields(cf));
// should_inline is only meaningful for branches
let should_inline = !is_leaf && base_result.should_inline(is_parameterizable);
// should_inline determines if we generate an inline struct type
// We inline only if it's a branch AND doesn't match any pattern
let should_inline = !is_leaf && !matches_any_pattern;
// Inline type name (only used when should_inline is true)
let inline_type_name = if should_inline {
@@ -100,6 +111,7 @@ pub fn prepare_tree_node<'a>(
is_leaf,
should_inline,
inline_type_name,
is_parameterizable,
}
})
.collect();
crates/brk_bindgen/src/generators/javascript/client.rs
+18 -28
View File
@@ -12,7 +12,7 @@ use serde_json::Value;
use crate::{
ClientMetadata, GenericSyntax, IndexSetPattern, JavaScriptSyntax, StructuralPattern, VERSION,
generate_parameterized_field, generate_tree_path_field, to_camel_case,
generate_parameterized_field, to_camel_case,
};
/// Generate the base BrkClient class with HTTP functionality.
@@ -186,7 +186,7 @@ function _endpoint(client, name, index) {{
get(index) {{ return singleItemBuilder(index); }},
slice(start, end) {{ return rangeBuilder(start, end); }},
first(n) {{ return rangeBuilder(undefined, n); }},
last(n) {{ return rangeBuilder(-n, undefined); }},
last(n) {{ return n === 0 ? rangeBuilder(undefined, 0) : rangeBuilder(-n, undefined); }},
skip(n) {{ return skippedBuilder(n); }},
fetch(onUpdate) {{ return client.getJson(buildPath(), onUpdate); }},
fetchCsv() {{ return client.getText(buildPath(undefined, undefined, 'csv')); }},
@@ -220,7 +220,7 @@ class BrkClientBase {{
const base = this.baseUrl.endsWith('/') ? this.baseUrl.slice(0, -1) : this.baseUrl;
const url = `${{base}}${{path}}`;
const res = await fetch(url, {{ signal: AbortSignal.timeout(this.timeout) }});
if (!res.ok) throw new BrkError(`HTTP ${{res.status}}`, res.status);
if (!res.ok) throw new BrkError(`HTTP ${{res.status}}: ${{url}}`, res.status);
return res;
}}
@@ -271,12 +271,20 @@ class BrkClientBase {{
}}
/**
* Build metric name with optional prefix.
* Build metric name with suffix.
* @param {{string}} acc - Accumulated prefix
* @param {{string}} s - Metric suffix
* @returns {{string}}
*/
const _m = (acc, s) => acc ? `${{acc}}_${{s}}` : s;
const _m = (acc, s) => s ? (acc ? `${{acc}}_${{s}}` : s) : acc;
/**
* Build metric name with prefix.
* @param {{string}} prefix - Prefix to prepend
* @param {{string}} acc - Accumulated name
* @returns {{string}}
*/
const _p = (prefix, acc) => acc ? `${{prefix}}_${{acc}}` : prefix;
"#
)
@@ -470,8 +478,7 @@ pub fn generate_structural_patterns(
writeln!(output, "// Reusable structural pattern factories\n").unwrap();
for pattern in patterns {
let is_parameterizable = pattern.is_parameterizable();
// Generate typedef
writeln!(output, "/**").unwrap();
if pattern.is_generic {
writeln!(output, " * @template T").unwrap();
@@ -494,17 +501,14 @@ pub fn generate_structural_patterns(
}
writeln!(output, " */\n").unwrap();
// Generate factory function for ALL patterns
writeln!(output, "/**").unwrap();
writeln!(output, " * Create a {} pattern node", pattern.name).unwrap();
if pattern.is_generic {
writeln!(output, " * @template T").unwrap();
}
writeln!(output, " * @param {{BrkClientBase}} client").unwrap();
if is_parameterizable {
writeln!(output, " * @param {{string}} acc - Accumulated metric name").unwrap();
} else {
writeln!(output, " * @param {{string}} basePath").unwrap();
}
writeln!(output, " * @param {{string}} acc - Accumulated metric name").unwrap();
let return_type = if pattern.is_generic {
format!("{}<T>", pattern.name)
} else {
@@ -513,26 +517,12 @@ pub fn generate_structural_patterns(
writeln!(output, " * @returns {{{}}}", return_type).unwrap();
writeln!(output, " */").unwrap();
let param_name = if is_parameterizable {
"acc"
} else {
"basePath"
};
writeln!(
output,
"function create{}(client, {}) {{",
pattern.name, param_name
)
.unwrap();
writeln!(output, "function create{}(client, acc) {{", pattern.name).unwrap();
writeln!(output, " return {{").unwrap();
let syntax = JavaScriptSyntax;
for field in &pattern.fields {
if is_parameterizable {
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
} else {
generate_tree_path_field(output, &syntax, field, metadata, " ");
}
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
}
writeln!(output, " }};").unwrap();
crates/brk_bindgen/src/generators/javascript/mod.rs
+3 -3
View File
@@ -3,9 +3,9 @@
//! This module generates a JavaScript + JSDoc client for the BRK API.
mod api;
mod client;
mod tree;
mod types;
pub mod client;
pub mod tree;
pub mod types;
use std::{fmt::Write, fs, io, path::Path};
crates/brk_bindgen/src/generators/javascript/tree.rs
+2 -4
View File
@@ -175,10 +175,8 @@ fn generate_tree_initializer(
TreeNode::Branch(grandchildren) => {
let field_name = to_camel_case(child_name);
let child_fields = get_node_fields(grandchildren, pattern_lookup);
// Only use pattern factory if pattern is parameterizable
let pattern_name = pattern_lookup
.get(&child_fields)
.filter(|name| metadata.is_parameterizable(name));
// Use pattern factory if ANY pattern matches (not just parameterizable)
let pattern_name = pattern_lookup.get(&child_fields);
let base_result = get_pattern_instance_base(child_node);
crates/brk_bindgen/src/generators/python/client.rs
+22 -30
View File
@@ -11,7 +11,7 @@ use serde::Serialize;
use crate::{
ClientMetadata, IndexSetPattern, PythonSyntax, StructuralPattern, VERSION,
generate_parameterized_field, generate_tree_path_field, index_to_field_name,
generate_parameterized_field, index_to_field_name,
};
/// Generate class-level constants for the BrkClient class.
@@ -132,9 +132,15 @@ class BrkClientBase:
def _m(acc: str, s: str) -> str:
"""Build metric name with optional prefix."""
"""Build metric name with suffix."""
if not s: return acc
return f"{{acc}}_{{s}}" if acc else s
def _p(prefix: str, acc: str) -> str:
"""Build metric name with prefix."""
return f"{{prefix}}_{{acc}}" if acc else prefix
"#
)
.unwrap();
@@ -309,9 +315,10 @@ class MetricEndpointBuilder(Generic[T]):
def tail(self, n: int = 10) -> RangeBuilder[T]:
"""Get the last n items (pandas-style)."""
start, end = (None, 0) if n == 0 else (-n, None)
return RangeBuilder(_EndpointConfig(
self._config.client, self._config.name, self._config.index,
-n, None
start, end
))
def skip(self, n: int) -> SkippedBuilder[T]:
@@ -467,9 +474,7 @@ pub fn generate_structural_patterns(
writeln!(output, "# Reusable structural pattern classes\n").unwrap();
for pattern in patterns {
let is_parameterizable = pattern.is_parameterizable();
// For generic patterns, inherit from Generic[T]
// Generate class
if pattern.is_generic {
writeln!(output, "class {}(Generic[T]):", pattern.name).unwrap();
} else {
@@ -481,33 +486,20 @@ pub fn generate_structural_patterns(
)
.unwrap();
writeln!(output, " ").unwrap();
if is_parameterizable {
writeln!(
output,
" def __init__(self, client: BrkClientBase, acc: str):"
)
.unwrap();
writeln!(
output,
" \"\"\"Create pattern node with accumulated metric name.\"\"\""
)
.unwrap();
} else {
writeln!(
output,
" def __init__(self, client: BrkClientBase, base_path: str):"
)
.unwrap();
}
writeln!(
output,
" def __init__(self, client: BrkClientBase, acc: str):"
)
.unwrap();
writeln!(
output,
" \"\"\"Create pattern node with accumulated metric name.\"\"\""
)
.unwrap();
let syntax = PythonSyntax;
for field in &pattern.fields {
if is_parameterizable {
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
} else {
generate_tree_path_field(output, &syntax, field, metadata, " ");
}
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
}
writeln!(output).unwrap();
crates/brk_bindgen/src/generators/python/mod.rs
+4 -4
View File
@@ -2,10 +2,10 @@
//!
//! This module generates a Python client with type hints for the BRK API.
mod api;
mod client;
mod tree;
mod types;
pub mod api;
pub mod client;
pub mod tree;
pub mod types;
use std::{fmt::Write, fs, io, path::Path};
crates/brk_bindgen/src/generators/rust/client.rs
+30 -36
View File
@@ -4,7 +4,7 @@ use std::fmt::Write;
use crate::{
ClientMetadata, GenericSyntax, IndexSetPattern, RustSyntax, StructuralPattern,
generate_parameterized_field, generate_tree_path_field, index_to_field_name, to_snake_case,
generate_parameterized_field, index_to_field_name, to_snake_case,
};
/// Generate import statements.
@@ -116,10 +116,18 @@ impl BrkClientBase {{
}}
}}
/// Build metric name with optional prefix.
/// Build metric name with suffix.
#[inline]
fn _m(acc: &str, s: &str) -> String {{
if acc.is_empty() {{ s.to_string() }} else {{ format!("{{acc}}_{{s}}") }}
if s.is_empty() {{ acc.to_string() }}
else if acc.is_empty() {{ s.to_string() }}
else {{ format!("{{acc}}_{{s}}") }}
}}
/// Build metric name with prefix.
#[inline]
fn _p(prefix: &str, acc: &str) -> String {{
if acc.is_empty() {{ prefix.to_string() }} else {{ format!("{{prefix}}_{{acc}}") }}
}}
"#
@@ -265,7 +273,11 @@ impl<T: DeserializeOwned> MetricEndpointBuilder<T> {{
/// Take the last n items.
pub fn last(mut self, n: usize) -> RangeBuilder<T> {{
self.config.start = Some(-(n as i64));
if n == 0 {{
self.config.end = Some(0);
}} else {{
self.config.start = Some(-(n as i64));
}}
RangeBuilder {{ config: self.config, _marker: std::marker::PhantomData }}
}}
@@ -399,7 +411,6 @@ pub fn generate_index_accessors(output: &mut String, patterns: &[IndexSetPattern
)
.unwrap();
writeln!(output, "pub struct {}<T> {{", pattern.name).unwrap();
writeln!(output, " client: Arc<BrkClientBase>,").unwrap();
writeln!(output, " name: Arc<str>,").unwrap();
writeln!(output, " pub by: {}<T>,", by_name).unwrap();
writeln!(output, "}}\n").unwrap();
@@ -413,13 +424,8 @@ pub fn generate_index_accessors(output: &mut String, patterns: &[IndexSetPattern
.unwrap();
writeln!(output, " let name: Arc<str> = name.into();").unwrap();
writeln!(output, " Self {{").unwrap();
writeln!(output, " client: client.clone(),").unwrap();
writeln!(output, " name: name.clone(),").unwrap();
writeln!(output, " by: {} {{", by_name).unwrap();
writeln!(output, " client,").unwrap();
writeln!(output, " name,").unwrap();
writeln!(output, " _marker: std::marker::PhantomData,").unwrap();
writeln!(output, " }}").unwrap();
writeln!(output, " by: {} {{ client, name, _marker: std::marker::PhantomData }}", by_name).unwrap();
writeln!(output, " }}").unwrap();
writeln!(output, " }}").unwrap();
writeln!(output).unwrap();
@@ -472,9 +478,9 @@ pub fn generate_pattern_structs(
writeln!(output, "// Reusable pattern structs\n").unwrap();
for pattern in patterns {
let is_parameterizable = pattern.is_parameterizable();
let generic_params = if pattern.is_generic { "<T>" } else { "" };
// Generate struct definition
writeln!(output, "/// Pattern struct for repeated tree structure.").unwrap();
writeln!(output, "pub struct {}{} {{", pattern.name, generic_params).unwrap();
@@ -487,7 +493,7 @@ pub fn generate_pattern_structs(
writeln!(output, "}}\n").unwrap();
// Generate impl block with constructor
// Generate impl block with constructor for ALL patterns
let impl_generic = if pattern.is_generic {
"<T: DeserializeOwned>"
} else {
@@ -500,33 +506,21 @@ pub fn generate_pattern_structs(
)
.unwrap();
if is_parameterizable {
writeln!(
output,
" /// Create a new pattern node with accumulated metric name."
)
.unwrap();
writeln!(
output,
" pub fn new(client: Arc<BrkClientBase>, acc: String) -> Self {{"
)
.unwrap();
} else {
writeln!(
output,
" pub fn new(client: Arc<BrkClientBase>, base_path: String) -> Self {{"
)
.unwrap();
}
writeln!(
output,
" /// Create a new pattern node with accumulated metric name."
)
.unwrap();
writeln!(
output,
" pub fn new(client: Arc<BrkClientBase>, acc: String) -> Self {{"
)
.unwrap();
writeln!(output, " Self {{").unwrap();
let syntax = RustSyntax;
for field in &pattern.fields {
if is_parameterizable {
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
} else {
generate_tree_path_field(output, &syntax, field, metadata, " ");
}
generate_parameterized_field(output, &syntax, field, pattern, metadata, " ");
}
writeln!(output, " }}").unwrap();
crates/brk_bindgen/src/generators/rust/mod.rs
+3 -3
View File
@@ -2,9 +2,9 @@
//!
//! This module generates a Rust client with full type safety for the BRK API.
mod api;
mod client;
mod tree;
pub mod api;
pub mod client;
pub mod tree;
mod types;
use std::{fmt::Write, fs, io, path::Path};
crates/brk_bindgen/src/generators/rust/tree.rs
+4 -2
View File
@@ -86,7 +86,7 @@ fn generate_tree_node(
);
}
} else if child.should_inline {
// Inline struct
// Inline struct type - only for nodes that don't match any pattern
let path_expr = syntax.path_expr("base_path", &format!("_{}", child.name));
writeln!(
output,
@@ -95,7 +95,9 @@ fn generate_tree_node(
)
.unwrap();
} else {
// Use pattern constructor
// Pattern type - use ::new() constructor
// All patterns have ::new(), parameterizable ones use detected mode,
// non-parameterizable ones use field name fallback
generate_tree_node_field(
output,
&syntax,
crates/brk_bindgen/src/lib.rs
+1 -1
View File
@@ -58,7 +58,7 @@ mod types;
pub use analysis::*;
pub use backends::*;
pub use generate::*;
pub use generators::{generate_javascript_client, generate_python_client, generate_rust_client};
pub use generators::*;
pub use openapi::*;
pub use syntax::*;
pub use types::*;
crates/brk_bindgen/src/syntax.rs
+21 -5
View File
@@ -4,7 +4,7 @@
//! language-specific code generation patterns, allowing shared generation
//! logic to work across Python, JavaScript, and Rust backends.
use crate::{FieldNamePosition, GenericSyntax};
use crate::GenericSyntax;
/// Language-specific syntax for code generation.
///
@@ -30,11 +30,27 @@ pub trait LanguageSyntax {
/// - Rust: `format!("{acc}_suffix")`
fn path_expr(&self, base_var: &str, suffix: &str) -> String;
/// Format a `FieldNamePosition` as a path expression.
/// Format a suffix mode expression: `_m(acc, relative)`.
///
/// This handles the different name transformation patterns (append, prepend,
/// identity, set_base) in a language-specific way.
fn position_expr(&self, pos: &FieldNamePosition, base_var: &str) -> String;
/// Suffix mode appends the relative name to the accumulator.
/// - If relative is empty, returns just acc (identity)
/// - Otherwise: `{acc}_{relative}` or `{relative}` if acc is empty
///
/// # Arguments
/// * `acc_var` - The accumulator variable name (e.g., "acc")
/// * `relative` - The relative name to append (e.g., "max_cost_basis")
fn suffix_expr(&self, acc_var: &str, relative: &str) -> String;
/// Format a prefix mode expression: `_p(prefix, acc)`.
///
/// Prefix mode prepends the prefix to the accumulator.
/// - If prefix is empty, returns just acc (identity)
/// - Otherwise: `{prefix}{acc}` (prefix includes trailing underscore)
///
/// # Arguments
/// * `prefix` - The prefix to prepend (e.g., "cumulative_")
/// * `acc_var` - The accumulator variable name (e.g., "acc")
fn prefix_expr(&self, prefix: &str, acc_var: &str) -> String;
/// Generate a constructor call for patterns and accessors.
///
crates/brk_bindgen/src/types/metadata.rs
+35 -6
View File
@@ -28,7 +28,11 @@ pub struct ClientMetadata {
impl ClientMetadata {
/// Extract metadata from brk_query::Vecs.
pub fn from_vecs(vecs: &Vecs) -> Self {
let catalog = vecs.catalog().clone();
Self::from_catalog(vecs.catalog().clone())
}
/// Extract metadata from a catalog TreeNode directly.
pub fn from_catalog(catalog: brk_types::TreeNode) -> Self {
let (structural_patterns, concrete_to_pattern, concrete_to_type_param) =
analysis::detect_structural_patterns(&catalog);
let (used_indexes, index_set_patterns) = analysis::detect_index_patterns(&catalog);
@@ -65,9 +69,33 @@ impl ClientMetadata {
self.find_pattern(name).is_some_and(|p| p.is_generic)
}
/// Check if a pattern by name is parameterizable.
/// Check if a pattern by name is fully parameterizable.
/// A pattern is parameterizable if it has a mode AND all its branch fields
/// are also parameterizable (or not patterns at all).
pub fn is_parameterizable(&self, name: &str) -> bool {
self.find_pattern(name).is_some_and(|p| p.is_parameterizable())
self.find_pattern(name).is_some_and(|p| {
if !p.is_parameterizable() {
return false;
}
// Check all branch fields have parameterizable types (or are not patterns)
p.fields.iter().all(|f| {
if f.is_branch() {
self.structural_patterns
.iter()
.find(|pat| pat.name == f.rust_type)
.is_none_or(|pat| pat.is_parameterizable())
} else {
true
}
})
})
}
/// Check if child fields match ANY pattern (parameterizable or not).
/// Used for type annotations - we want to reuse pattern types for all patterns.
pub fn matches_pattern(&self, fields: &[PatternField]) -> bool {
self.concrete_to_pattern.contains_key(fields)
|| self.structural_patterns.iter().any(|p| p.fields == fields)
}
/// Check if child fields match a parameterizable pattern.
@@ -84,8 +112,8 @@ impl ClientMetadata {
.is_some_and(|name| self.is_parameterizable(name))
}
/// Resolve the type name for a tree field, considering parameterizability.
/// If the field matches a parameterizable pattern, returns type annotation.
/// Resolve the type name for a tree field.
/// If the field matches ANY pattern (parameterizable or not), returns pattern type.
/// Otherwise returns the inline type name (parent_child format).
pub fn resolve_tree_field_type(
&self,
@@ -96,7 +124,8 @@ impl ClientMetadata {
syntax: GenericSyntax,
) -> String {
match child_fields {
Some(cf) if self.is_parameterizable_fields(cf) => {
// Use pattern type for ANY matching pattern (parameterizable or not)
Some(cf) if self.matches_pattern(cf) => {
let generic_value_type = self.get_type_param(cf).map(String::as_str);
self.field_type_annotation(field, false, generic_value_type, syntax)
}
crates/brk_bindgen/src/types/positions.rs
+23 -11
View File
@@ -1,14 +1,26 @@
//! Field name position types for metric name reconstruction.
//! Pattern mode and field parts for metric name reconstruction.
//!
//! Patterns are either suffix mode or prefix mode:
//! - Suffix mode: `_m(acc, relative)` → `acc_relative` or just `relative` if acc empty
//! - Prefix mode: `_p(prefix, acc)` → `prefix_acc` or just `acc` if prefix empty
/// How a field modifies the accumulated metric name.
use std::collections::HashMap;
/// How a pattern constructs metric names from the accumulator.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum FieldNamePosition {
/// Field prepends a prefix: leaf.name() = prefix + accumulated
Prepend(String),
/// Field appends a suffix: leaf.name() = accumulated + suffix
Append(String),
/// Field IS the accumulated name (no modification)
Identity,
/// Field sets a new base name (used at pattern entry points)
SetBase(String),
pub enum PatternMode {
/// Fields append their relative name to acc.
/// Formula: `_m(acc, relative)` → `{acc}_{relative}` or `{relative}` if acc empty
/// Example: `_m("lth", "max_cost_basis")` → `"lth_max_cost_basis"`
Suffix {
/// Maps field name to its relative name (full metric name when acc = "")
relatives: HashMap<String, String>,
},
/// Fields prepend their prefix to acc.
/// Formula: `_p(prefix, acc)` → `{prefix}_{acc}` or `{acc}` if prefix empty
/// Example: `_p("cumulative", "lth_realized_loss")` → `"cumulative_lth_realized_loss"`
Prefix {
/// Maps field name to its prefix (empty string for identity)
prefixes: HashMap<String, String>,
},
}
crates/brk_bindgen/src/types/structs.rs
+23 -13
View File
@@ -1,10 +1,10 @@
//! Structural pattern and field types.
use std::collections::{BTreeSet, HashMap};
use std::collections::BTreeSet;
use brk_types::Index;
use super::FieldNamePosition;
use super::PatternMode;
/// A pattern of indexes that appear together on multiple metrics.
#[derive(Debug, Clone)]
@@ -22,8 +22,8 @@ pub struct StructuralPattern {
pub name: String,
/// Ordered list of child fields
pub fields: Vec<PatternField>,
/// How each field modifies the accumulated name
pub field_positions: HashMap<String, FieldNamePosition>,
/// How fields construct metric names from acc (None = not parameterizable)
pub mode: Option<PatternMode>,
/// If true, all leaf fields use a type parameter T
pub is_generic: bool,
}
@@ -34,18 +34,28 @@ impl StructuralPattern {
self.fields.iter().any(|f| f.is_leaf())
}
/// Returns true if all leaf fields have consistent name transformations.
/// Returns true if this pattern can be parameterized with an accumulator.
pub fn is_parameterizable(&self) -> bool {
!self.field_positions.is_empty()
&& self
.fields
.iter()
.all(|f| f.is_branch() || self.field_positions.contains_key(&f.name))
self.mode.is_some()
}
/// Get the field position for a given field name.
pub fn get_field_position(&self, field_name: &str) -> Option<&FieldNamePosition> {
self.field_positions.get(field_name)
/// Get the field part (relative name or prefix) for a given field.
pub fn get_field_part(&self, field_name: &str) -> Option<&str> {
match &self.mode {
Some(PatternMode::Suffix { relatives }) => relatives.get(field_name).map(|s| s.as_str()),
Some(PatternMode::Prefix { prefixes }) => prefixes.get(field_name).map(|s| s.as_str()),
None => None,
}
}
/// Returns true if this pattern is in suffix mode.
pub fn is_suffix_mode(&self) -> bool {
matches!(&self.mode, Some(PatternMode::Suffix { .. }))
}
/// Returns true if this pattern is in prefix mode.
pub fn is_prefix_mode(&self) -> bool {
matches!(&self.mode, Some(PatternMode::Prefix { .. }))
}
}
crates/brk_bindgen/tests/catalog_test.rs
+822
View File
@@ -0,0 +1,822 @@
//! Tests that verify pattern analysis using the real catalog.
use std::collections::HashSet;
use std::fmt::Write;
use brk_bindgen::ClientMetadata;
use brk_types::TreeNode;
/// Load the catalog from the JSON file.
fn load_catalog() -> TreeNode {
let path = concat!(env!("CARGO_MANIFEST_DIR"), "/catalog.json");
let catalog_json = std::fs::read_to_string(path).expect("Failed to read catalog.json");
serde_json::from_str(&catalog_json).expect("Failed to parse catalog.json")
}
/// Load OpenAPI spec from api.json.
fn load_openapi_json() -> String {
let path = concat!(env!("CARGO_MANIFEST_DIR"), "/api.json");
std::fs::read_to_string(path).expect("Failed to read api.json")
}
/// Load metadata from the catalog.
#[allow(unused)]
fn load_metadata() -> ClientMetadata {
ClientMetadata::from_catalog(load_catalog())
}
/// Collect all leaf metric names from a tree.
fn collect_leaf_names(node: &TreeNode, names: &mut HashSet<String>) {
match node {
TreeNode::Leaf(leaf) => {
names.insert(leaf.name().to_string());
}
TreeNode::Branch(children) => {
for child in children.values() {
collect_leaf_names(child, names);
}
}
}
}
#[test]
fn test_catalog_loads() {
let catalog = load_catalog();
// Should be a branch with top-level categories
let TreeNode::Branch(categories) = &catalog else {
panic!("Expected catalog to be a branch");
};
// Check some expected top-level categories exist
assert!(
categories.contains_key("addresses"),
"Missing addresses category"
);
assert!(categories.contains_key("blocks"), "Missing blocks category");
assert!(categories.contains_key("market"), "Missing market category");
assert!(categories.contains_key("supply"), "Missing supply category");
println!("Catalog has {} top-level categories", categories.len());
}
#[test]
fn test_all_leaves_have_names() {
let catalog = load_catalog();
let mut names = HashSet::new();
collect_leaf_names(&catalog, &mut names);
println!("Catalog has {} unique metric names", names.len());
assert!(!names.is_empty(), "Should have at least some metrics");
// All names should be non-empty
for name in &names {
assert!(!name.is_empty(), "Found empty metric name");
}
}
#[test]
fn test_pattern_detection() {
let catalog = load_catalog();
let (patterns, concrete_to_pattern, concrete_to_type_param) =
brk_bindgen::detect_structural_patterns(&catalog);
println!("Detected {} structural patterns", patterns.len());
println!(
"Concrete to pattern mappings: {}",
concrete_to_pattern.len()
);
println!("Type parameter mappings: {}", concrete_to_type_param.len());
// Print pattern details
for pattern in &patterns {
let mode_str = match &pattern.mode {
Some(brk_bindgen::PatternMode::Suffix { relatives }) => {
format!("Suffix({})", relatives.len())
}
Some(brk_bindgen::PatternMode::Prefix { prefixes }) => {
format!("Prefix({})", prefixes.len())
}
None => "None".to_string(),
};
println!(
" {} (fields: {}, generic: {}, mode: {})",
pattern.name,
pattern.fields.len(),
pattern.is_generic,
mode_str
);
}
// Should have detected some patterns
assert!(!patterns.is_empty(), "Should detect at least some patterns");
// Check that parameterizable patterns have valid modes
for pattern in &patterns {
if pattern.is_parameterizable() {
let mode = pattern.mode.as_ref().unwrap();
match mode {
brk_bindgen::PatternMode::Suffix { relatives } => {
assert_eq!(
relatives.len(),
pattern.fields.len(),
"Pattern {} should have relative for each field",
pattern.name
);
}
brk_bindgen::PatternMode::Prefix { prefixes } => {
assert_eq!(
prefixes.len(),
pattern.fields.len(),
"Pattern {} should have prefix for each field",
pattern.name
);
}
}
}
}
}
#[test]
fn test_cost_basis_pattern() {
let catalog = load_catalog();
let (patterns, _, _) = brk_bindgen::detect_structural_patterns(&catalog);
// Find CostBasisPattern2 and inspect it
let cost_basis = patterns
.iter()
.find(|p| p.name == "CostBasisPattern2")
.expect("CostBasisPattern2 should exist");
println!("CostBasisPattern2:");
println!(
" Fields: {:?}",
cost_basis
.fields
.iter()
.map(|f| &f.name)
.collect::<Vec<_>>()
);
println!(" Mode: {:?}", cost_basis.mode);
println!(" Is generic: {}", cost_basis.is_generic);
// With suffix naming convention (cost_basis_max, cost_basis_min, cost_basis):
//
// At root level: common prefix is "cost_basis_" -> suffix mode
// max -> "max"
// min -> "min"
// percentiles -> "" (identity)
//
// At lth_ level: common prefix is "lth_cost_basis_" -> suffix mode
// max -> "max"
// min -> "min"
// percentiles -> "" (identity)
//
// Both use suffix mode with same relatives, so pattern IS parameterizable!
assert!(
cost_basis.is_parameterizable(),
"CostBasisPattern2 should be parameterizable with consistent suffix mode"
);
}
#[test]
fn test_realized_pattern3_fields() {
let catalog = load_catalog();
let metadata = ClientMetadata::from_catalog(catalog);
let pattern = metadata
.find_pattern("RealizedPattern3")
.expect("RealizedPattern3 should exist");
println!("RealizedPattern3 fields:");
for field in &pattern.fields {
let is_branch = field.is_branch();
let is_pattern = metadata.find_pattern(&field.rust_type).is_some();
let is_param = metadata.is_parameterizable(&field.rust_type);
println!(
" {} -> {} (branch={}, pattern={}, param={})",
field.name, field.rust_type, is_branch, is_pattern, is_param
);
}
// Check if RealizedPattern3 is considered parameterizable
println!(
"\nRealizedPattern3 is_parameterizable (metadata): {}",
metadata.is_parameterizable("RealizedPattern3")
);
}
#[test]
fn test_parameterizable_patterns_have_mode() {
let catalog = load_catalog();
let (patterns, _, _) = brk_bindgen::detect_structural_patterns(&catalog);
// All patterns that appear 2+ times should either:
// 1. Be parameterizable (have a mode)
// 2. Or have inconsistent instances (mode = None)
//
// Patterns with mode = None should be inlined, not generate factories
let parameterizable: Vec<_> = patterns.iter().filter(|p| p.is_parameterizable()).collect();
let non_parameterizable: Vec<_> = patterns
.iter()
.filter(|p| !p.is_parameterizable())
.collect();
println!("\nParameterizable patterns ({}):", parameterizable.len());
for p in &parameterizable {
let mode = p.mode.as_ref().unwrap();
let mode_type = match mode {
brk_bindgen::PatternMode::Suffix { .. } => "Suffix",
brk_bindgen::PatternMode::Prefix { .. } => "Prefix",
};
println!(" {} ({} fields, {})", p.name, p.fields.len(), mode_type);
}
println!(
"\nNon-parameterizable patterns ({}):",
non_parameterizable.len()
);
for p in &non_parameterizable {
println!(" {} ({} fields)", p.name, p.fields.len());
}
// Verify all parameterizable patterns have valid modes with all fields
for pattern in &parameterizable {
let mode = pattern.mode.as_ref().unwrap();
let field_names: HashSet<_> = pattern.fields.iter().map(|f| f.name.clone()).collect();
match mode {
brk_bindgen::PatternMode::Suffix { relatives } => {
let mode_fields: HashSet<_> = relatives.keys().cloned().collect();
assert_eq!(
field_names, mode_fields,
"Pattern {} suffix mode should have all fields",
pattern.name
);
}
brk_bindgen::PatternMode::Prefix { prefixes } => {
let mode_fields: HashSet<_> = prefixes.keys().cloned().collect();
assert_eq!(
field_names, mode_fields,
"Pattern {} prefix mode should have all fields",
pattern.name
);
}
}
}
}
#[test]
fn test_index_patterns() {
let catalog = load_catalog();
let (used_indexes, index_patterns) = brk_bindgen::detect_index_patterns(&catalog);
println!("Used indexes: {:?}", used_indexes);
println!("Index set patterns: {}", index_patterns.len());
for pattern in &index_patterns {
println!(" {} -> {:?}", pattern.name, pattern.indexes);
}
// Should have detected some index patterns
assert!(!index_patterns.is_empty(), "Should detect index patterns");
}
#[test]
fn test_generated_rust_output() {
let catalog = load_catalog();
let metadata = ClientMetadata::from_catalog(catalog.clone());
// Collect all metric names from the catalog
let mut all_metrics = HashSet::new();
collect_leaf_names(&catalog, &mut all_metrics);
// Generate Rust client output
let mut rust_output = String::new();
brk_bindgen::rust::client::generate_imports(&mut rust_output);
brk_bindgen::rust::client::generate_base_client(&mut rust_output);
brk_bindgen::rust::client::generate_metric_pattern_trait(&mut rust_output);
brk_bindgen::rust::client::generate_endpoint(&mut rust_output);
brk_bindgen::rust::client::generate_index_accessors(
&mut rust_output,
&metadata.index_set_patterns,
);
brk_bindgen::rust::client::generate_pattern_structs(
&mut rust_output,
&metadata.structural_patterns,
&metadata,
);
brk_bindgen::rust::tree::generate_tree(&mut rust_output, &metadata.catalog, &metadata);
brk_bindgen::rust::api::generate_main_client(&mut rust_output, &[]);
// Count metrics that appear as direct string literals
let mut direct_metrics = 0;
for metric in &all_metrics {
if rust_output.contains(&format!("\"{}\"", metric)) {
direct_metrics += 1;
}
}
println!("\nGenerated Rust output stats:");
println!(" Total metrics in catalog: {}", all_metrics.len());
println!(" Direct string literals: {}", direct_metrics);
println!(
" Via pattern factories: {}",
all_metrics.len() - direct_metrics
);
println!(" Output size: {} bytes", rust_output.len());
// Write output to actual client location
let output_path = concat!(env!("CARGO_MANIFEST_DIR"), "/../brk_client/src/lib.rs");
std::fs::write(output_path, &rust_output).expect("Failed to write client output");
println!(" Wrote output to: {}", output_path);
// Verify the output contains the key components
assert!(rust_output.contains("fn _m("), "Should define _m helper");
assert!(
rust_output.contains("pub struct MetricsTree"),
"Should have MetricsTree"
);
assert!(
rust_output.contains("impl MetricsTree"),
"Should have MetricsTree impl"
);
// Count parameterizable patterns (these use _m for dynamic metric names)
// Use metadata.is_parameterizable() for full recursive check
let parameterizable_count = metadata
.structural_patterns
.iter()
.filter(|p| metadata.is_parameterizable(&p.name))
.count();
println!(" Parameterizable patterns: {}", parameterizable_count);
// Verify all pattern structs are generated (parameterizable and non)
for pattern in &metadata.structural_patterns {
assert!(
rust_output.contains(&format!("pub struct {}", pattern.name)),
"Missing pattern struct: {}",
pattern.name
);
}
println!("\nGenerated Rust client is complete!");
}
#[test]
fn test_generated_javascript_output() {
let catalog = load_catalog();
let metadata = ClientMetadata::from_catalog(catalog.clone());
// Collect all metric names from the catalog
let mut all_metrics = HashSet::new();
collect_leaf_names(&catalog, &mut all_metrics);
// Load schemas from OpenAPI spec only (catalog schemas require runtime data)
let openapi_json = load_openapi_json();
let schemas = brk_bindgen::extract_schemas(&openapi_json);
// Generate JavaScript client output
let mut js_output = String::new();
writeln!(js_output, "// Auto-generated BRK JavaScript client").unwrap();
writeln!(js_output, "// Do not edit manually\n").unwrap();
brk_bindgen::javascript::types::generate_type_definitions(&mut js_output, &schemas);
brk_bindgen::javascript::client::generate_base_client(&mut js_output);
brk_bindgen::javascript::client::generate_index_accessors(
&mut js_output,
&metadata.index_set_patterns,
);
brk_bindgen::javascript::client::generate_structural_patterns(
&mut js_output,
&metadata.structural_patterns,
&metadata,
);
brk_bindgen::javascript::tree::generate_tree_typedefs(
&mut js_output,
&metadata.catalog,
&metadata,
);
brk_bindgen::javascript::tree::generate_main_client(
&mut js_output,
&metadata.catalog,
&metadata,
&[],
);
// Count metrics that appear as direct string literals
let mut direct_metrics = 0;
for metric in &all_metrics {
if js_output.contains(&format!("'{}'", metric))
|| js_output.contains(&format!("\"{}\"", metric))
{
direct_metrics += 1;
}
}
println!("\nGenerated JavaScript output stats:");
println!(" Total metrics in catalog: {}", all_metrics.len());
println!(" Direct string literals: {}", direct_metrics);
println!(
" Via pattern factories: {}",
all_metrics.len() - direct_metrics
);
println!(" Output size: {} bytes", js_output.len());
println!(" Output lines: {}", js_output.lines().count());
// Write output to actual client location
let output_path = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../modules/brk-client/index.js"
);
std::fs::write(output_path, &js_output).expect("Failed to write JS client output");
println!(" Wrote output to: {}", output_path);
// Verify the output contains key components
assert!(js_output.contains("const _m ="), "Should define _m helper");
assert!(js_output.contains("const _p ="), "Should define _p helper");
assert!(
js_output.contains("@typedef {Object} MetricsTree"),
"Should have MetricsTree typedef"
);
assert!(
js_output.contains("class BrkClient"),
"Should have BrkClient class"
);
// Verify all pattern factories are generated
for pattern in &metadata.structural_patterns {
assert!(
js_output.contains(&format!("function create{}(", pattern.name)),
"Missing pattern factory: {}",
pattern.name
);
}
println!("\nGenerated JavaScript client is complete!");
}
#[test]
fn test_generated_python_output() {
let catalog = load_catalog();
let metadata = ClientMetadata::from_catalog(catalog.clone());
// Collect all metric names from the catalog
let mut all_metrics = HashSet::new();
collect_leaf_names(&catalog, &mut all_metrics);
// Load schemas from OpenAPI spec only (catalog schemas require runtime data)
let openapi_json = load_openapi_json();
let schemas = brk_bindgen::extract_schemas(&openapi_json);
// Generate Python client output
let mut py_output = String::new();
writeln!(py_output, "# Auto-generated BRK Python client").unwrap();
writeln!(py_output, "# Do not edit manually\n").unwrap();
writeln!(py_output, "from typing import TypeVar, Generic, Any, Optional, List, Literal, TypedDict, Union, Protocol, overload").unwrap();
writeln!(
py_output,
"from http.client import HTTPSConnection, HTTPConnection"
)
.unwrap();
writeln!(py_output, "from urllib.parse import urlparse").unwrap();
writeln!(py_output, "import json\n").unwrap();
writeln!(py_output, "T = TypeVar('T')\n").unwrap();
brk_bindgen::python::types::generate_type_definitions(&mut py_output, &schemas);
brk_bindgen::python::client::generate_base_client(&mut py_output);
brk_bindgen::python::client::generate_endpoint_class(&mut py_output);
brk_bindgen::python::client::generate_index_accessors(
&mut py_output,
&metadata.index_set_patterns,
);
brk_bindgen::python::client::generate_structural_patterns(
&mut py_output,
&metadata.structural_patterns,
&metadata,
);
brk_bindgen::python::tree::generate_tree_classes(&mut py_output, &metadata.catalog, &metadata);
brk_bindgen::python::api::generate_main_client(&mut py_output, &[]);
// Count metrics that appear as direct string literals
let mut direct_metrics = 0;
for metric in &all_metrics {
if py_output.contains(&format!("'{}'", metric))
|| py_output.contains(&format!("\"{}\"", metric))
{
direct_metrics += 1;
}
}
println!("\nGenerated Python output stats:");
println!(" Total metrics in catalog: {}", all_metrics.len());
println!(" Direct string literals: {}", direct_metrics);
println!(
" Via pattern factories: {}",
all_metrics.len() - direct_metrics
);
println!(" Output size: {} bytes", py_output.len());
println!(" Output lines: {}", py_output.lines().count());
// Write output to actual client location
let output_path = concat!(
env!("CARGO_MANIFEST_DIR"),
"/../../packages/brk_client/brk_client/__init__.py"
);
std::fs::write(output_path, &py_output).expect("Failed to write Python client output");
println!(" Wrote output to: {}", output_path);
// Verify the output contains key components
assert!(py_output.contains("def _m("), "Should define _m helper");
assert!(py_output.contains("def _p("), "Should define _p helper");
assert!(
py_output.contains("class MetricsTree:"),
"Should have MetricsTree class"
);
assert!(
py_output.contains("class BrkClient"),
"Should have BrkClient class"
);
// Verify all pattern classes have constructors
for pattern in &metadata.structural_patterns {
assert!(
py_output.contains(&format!("class {}:", pattern.name))
|| py_output.contains(&format!("class {}(", pattern.name)),
"Missing pattern class: {}",
pattern.name
);
}
println!("\nGenerated Python client is complete!");
}
#[test]
fn test_cost_basis_relatives() {
let catalog = load_catalog();
// Find cost_basis branches that have 3 direct children (max, min, percentiles)
fn find_cost_basis_with_percentiles(
node: &TreeNode,
path: &str,
) -> Vec<(String, Vec<(String, String)>)> {
let mut results = Vec::new();
if let TreeNode::Branch(children) = node {
for (name, child) in children {
let child_path = if path.is_empty() {
name.clone()
} else {
format!("{}.{}", path, name)
};
if name == "cost_basis"
&& let TreeNode::Branch(cb_children) = child
&& cb_children.contains_key("percentiles")
{
// Found a cost_basis with percentiles
let mut metrics = Vec::new();
for (field_name, field_node) in cb_children {
match field_node {
TreeNode::Leaf(leaf) => {
metrics.push((field_name.clone(), leaf.name().to_string()));
}
TreeNode::Branch(pct_children) => {
// Get first percentile as example
if let Some((_, TreeNode::Leaf(first))) = pct_children.iter().next()
{
metrics.push((
format!("{}.first", field_name),
first.name().to_string(),
));
}
}
}
}
results.push((child_path.clone(), metrics));
}
results.extend(find_cost_basis_with_percentiles(child, &child_path));
}
}
results
}
let instances = find_cost_basis_with_percentiles(&catalog, "");
println!("\nCostBasisPattern2 instances (with percentiles):");
for (path, metrics) in instances.iter().take(10) {
println!(" {}:", path);
for (field, metric) in metrics {
println!(" {} -> {}", field, metric);
}
}
// Now compute what relatives the pattern detection would see
// The key is: percentiles returns its BASE (common prefix of pct05, pct10, etc.)
// not the individual percentile metrics
use brk_bindgen::find_common_prefix;
println!("\nComputing relatives (simulating branch base returns):");
for (path, metrics) in instances.iter().take(5) {
println!(" Instance: {}", path);
// For leaves (max, min), the base is the metric name
// For branches (percentiles), the base is the common prefix of its children
let mut child_bases: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
for (field, metric) in metrics {
if field.starts_with("percentiles.") {
// This is a percentile metric - compute what the percentiles branch would return
// The base is the metric name with the pct suffix stripped
let base = metric
.strip_suffix("_pct05")
.or_else(|| metric.strip_suffix("_pct10"))
.unwrap_or(metric)
.to_string();
child_bases.insert("percentiles".to_string(), base);
} else {
child_bases.insert(field.clone(), metric.clone());
}
}
let bases: Vec<&str> = child_bases.values().map(|s| s.as_str()).collect();
println!(" Child bases:");
for (field, base) in &child_bases {
println!(" {} -> {}", field, base);
}
if let Some(prefix) = find_common_prefix(&bases) {
println!(" Common prefix: '{}'", prefix);
for (field, base) in &child_bases {
let relative = base.strip_prefix(&prefix).unwrap_or(base);
println!(" {} -> relative '{}'", field, relative);
}
} else {
println!(" No common prefix found!");
}
}
}
#[test]
fn test_debug_cost_basis_pattern2_mode() {
// Debug why CostBasisPattern2 has mode=None
let catalog = load_catalog();
let metadata = brk_bindgen::ClientMetadata::from_catalog(catalog.clone());
let pattern_lookup = metadata.pattern_lookup();
let pattern = metadata
.find_pattern("CostBasisPattern2")
.expect("CostBasisPattern2 should exist");
println!("\nCostBasisPattern2 fields:");
for field in &pattern.fields {
println!(" {} (type: {})", field.name, field.rust_type);
}
println!("Mode: {:?}", pattern.mode);
// Now debug the instance collection
#[derive(Debug, Clone)]
struct DebugInstanceAnalysis {
base: String,
field_parts: std::collections::HashMap<String, String>,
is_suffix_mode: bool,
}
fn collect_debug(
node: &TreeNode,
pattern_lookup: &std::collections::HashMap<Vec<brk_bindgen::PatternField>, String>,
all_analyses: &mut std::collections::HashMap<String, Vec<DebugInstanceAnalysis>>,
) -> Option<String> {
match node {
TreeNode::Leaf(leaf) => Some(leaf.name().to_string()),
TreeNode::Branch(children) => {
let mut child_bases: std::collections::HashMap<String, String> =
std::collections::HashMap::new();
for (field_name, child_node) in children {
if let Some(base) = collect_debug(child_node, pattern_lookup, all_analyses) {
child_bases.insert(field_name.clone(), base);
}
}
if child_bases.is_empty() {
return None;
}
// Analyze this instance
let bases: Vec<&str> = child_bases.values().map(|s| s.as_str()).collect();
let (base, field_parts, is_suffix_mode) =
if let Some(common_prefix) = brk_bindgen::find_common_prefix(&bases) {
let base = common_prefix.trim_end_matches('_').to_string();
let mut parts = std::collections::HashMap::new();
for (field_name, child_base) in &child_bases {
let relative = if *child_base == base {
String::new()
} else {
child_base
.strip_prefix(&common_prefix)
.unwrap_or(child_base)
.to_string()
};
parts.insert(field_name.clone(), relative);
}
(base, parts, true)
} else {
let base = child_bases.values().next().cloned().unwrap_or_default();
let parts = child_bases
.iter()
.map(|(k, v)| (k.clone(), v.clone()))
.collect();
(base, parts, true)
};
let analysis = DebugInstanceAnalysis {
base: base.clone(),
field_parts,
is_suffix_mode,
};
// Get the pattern name for this node
let fields = brk_bindgen::get_node_fields(children, pattern_lookup);
if let Some(pattern_name) = pattern_lookup.get(&fields) {
all_analyses
.entry(pattern_name.clone())
.or_default()
.push(analysis);
}
Some(base)
}
}
}
let mut all_analyses: std::collections::HashMap<String, Vec<DebugInstanceAnalysis>> =
std::collections::HashMap::new();
collect_debug(&catalog, &pattern_lookup, &mut all_analyses);
if let Some(analyses) = all_analyses.get("CostBasisPattern2") {
println!(
"\nCollected {} instances of CostBasisPattern2:",
analyses.len()
);
for (i, a) in analyses.iter().enumerate() {
println!(" Instance {}:", i);
println!(" base: {}", a.base);
println!(" is_suffix: {}", a.is_suffix_mode);
println!(" field_parts:");
for (f, p) in &a.field_parts {
println!(" {} -> '{}'", f, p);
}
}
// Check consistency
if analyses.len() >= 2 {
let first = &analyses[0];
for (i, a) in analyses.iter().enumerate().skip(1) {
if a.is_suffix_mode != first.is_suffix_mode {
println!(" INCONSISTENT: Instance {} has different mode", i);
}
for (field, part) in &a.field_parts {
if first.field_parts.get(field) != Some(part) {
println!(
" INCONSISTENT: Instance {} field '{}' has part '{}' vs '{}'",
i,
field,
part,
first
.field_parts
.get(field)
.unwrap_or(&"<missing>".to_string())
);
}
}
}
}
} else {
println!("\nNo instances collected for CostBasisPattern2!");
}
}
#[test]
fn test_root_cost_basis_prefix() {
use brk_bindgen::find_common_prefix;
// Root-level cost_basis has:
// max -> "max_cost_basis"
// min -> "min_cost_basis"
// percentiles -> "cost_basis" (base of pct05, pct10, etc.)
let bases = vec!["max_cost_basis", "min_cost_basis", "cost_basis"];
let prefix = find_common_prefix(&bases);
println!("Root cost_basis prefix: {:?}", prefix);
// Compare with nested cost_basis
let nested_bases = vec![
"utxos_at_least_15y_old_max_cost_basis",
"utxos_at_least_15y_old_min_cost_basis",
"utxos_at_least_15y_old_cost_basis",
];
let nested_prefix = find_common_prefix(&nested_bases);
println!("Nested cost_basis prefix: {:?}", nested_prefix);
}