diff --git a/.claude/prompts/changelog.md b/.claude/prompts/changelog.md new file mode 100644 index 000000000..df8088e14 --- /dev/null +++ b/.claude/prompts/changelog.md @@ -0,0 +1,137 @@ +# Changelog Generation Prompt + +Update docs/CHANGELOG.md for ALL releases starting from v0.0.103 onwards. Use ONLY git commands - no other sources. + +## MANDATORY PROCESS - FOLLOW EXACTLY - NO EXCEPTIONS: +1. Run `git tag --list --sort=version:refname` to get releases in order +2. Process EXACTLY ONE release at a time starting from v0.0.103 +3. For EACH SINGLE release: run `git diff [previous-tag]..[current-tag]` +4. **MANDATORY ANALYSIS STEP**: Before writing ANY changelog entry, you MUST: + - Analyze each file change and explain what the code is doing + - Identify the purpose and impact of each modification + - Group related changes together logically + - State clearly what functionality is being added, removed, or modified + - If you cannot understand what a change does from the diff, explicitly say so +5. Only AFTER completing the analysis, write the detailed changelog entry +6. Update the CHANGELOG.md file with that ONE entry +7. STOP. Ask me if you should continue to the next release. + +## CRITICAL CONSTRAINTS: +- **NEVER EVER** process multiple releases in one go, even if there are many +- **NEVER** say "let me continue more efficiently by processing multiple releases" +- **NEVER** batch releases together for any reason +- If you feel tempted to process multiple releases, **STOP** and process only one +- Context window concerns do **NOT** justify batching - process one release only + +## ABSOLUTE REQUIREMENTS: +- **NEVER** read commit messages, PR descriptions, existing changelog, or any text documentation +- Use **ONLY** the actual code changes shown in git diff output +- Process releases **ONE BY ONE** - I don't care if there are 100 releases +- **MANDATORY**: Before writing changelog entries, demonstrate understanding by analyzing what each code change accomplishes +- Be **HIGHLY DESCRIPTIVE** about what each code change does and why it matters +- Don't be conservative - write detailed explanations of the impact and purpose of changes +- **If you don't understand a change from the code diff alone, DO NOT GUESS - say so explicitly** + +## SOURCE OF TRUTH: +- `git diff` output is the **ONLY** source of truth +- If you can't determine what a change does from the code diff alone, say so explicitly +- Ignore **ALL** text/documentation - focus purely on code additions, deletions, and modifications + +## CHANGELOG FILE REQUIREMENTS: +- Add a header at the top of the CHANGELOG.md file: `` +- Ensure this header appears before any changelog entries + +## CHANGELOG WRITING RULES: + +### RELEASE TITLE FORMAT: +**MUST** use this exact format: `## [vX.Y.Z](https://github.com/bitcoinresearchkit/brk/releases/tag/vX.Y.Z) - YYYY-MM-DD` +Use the actual release date from git tag information + +### ABSOLUTELY FORBIDDEN PATTERNS: +- **NEVER** mention line counts (e.g., "with 138 lines", "1,290 lines removed") +- **NEVER** use vague action words: "Enhanced", "Improved", "Updated", "Expanded", "Restructured", "Refactored", "Modified", "Adjusted" +- **NEVER** write sections about Cargo.lock or dependency updates unless they represent major functional changes +- **NEVER** use the format "Action: File with vague description" +- **NEVER** mention version bumps of local crates (e.g., "Updated all crate versions from 0.0.61 to 0.0.62") - this is implied by the release version +- **NEVER** mention dependency version changes in external crates unless they enable new functionality visible in the code +- **NEVER** write entries like "Updated dependencies" or "Cargo.lock maintenance" + +### REQUIRED WRITING STYLE: +- Write what the code **actually DOES**, not that it was "enhanced" or "improved" +- Be **specific about functionality**: "Added transaction validation logic", "Implemented caching for API responses" +- Focus on **business/functional impact**: "Enables users to...", "Fixes issue where...", "Adds support for..." +- **Mandatory structure**: Group by: Breaking Changes, New Features, Bug Fixes, Internal Changes +- Include GitHub file links for major changes (max 5 per entry) +- **Skip entirely**: minor dependency bumps, Cargo.lock changes, and local crate version bumps + +### MANDATORY ANALYSIS WORKFLOW: +**BEFORE writing any changelog entry, you MUST:** + +1. **Code Comprehension Check**: Go through each modified file and explain: + - What specific functionality is being added/removed/changed + - What the new/modified functions/structs/methods do + - How the changes affect the overall system behavior + +2. **Impact Assessment**: For each change, determine: + - Is this a new feature, bug fix, breaking change, or internal improvement? + - What user-facing or system behavior changes result from this code? + - What problem does this change solve? + +3. **Logical Grouping**: Organize related changes together: + - Group files that work together to implement a single feature + - Separate breaking changes from additions + - Distinguish user-facing changes from internal refactoring + +4. **Understanding Verification**: Before writing changelog text, state: + - "I understand this change does X because the code shows Y" + - If unclear: "I cannot determine the purpose of this change from the diff alone" + +**ONLY AFTER completing this analysis should you write the changelog entry.** + +### WHAT TO FOCUS ON (IN ORDER OF PRIORITY): +1. **New functionality** - What can users now do that they couldn't before? +2. **Breaking changes** - What existing functionality changed or was removed? +3. **Bug fixes** - What specific problems were resolved? +4. **Internal changes** - New modules, significant refactoring, architecture changes +5. **Skip completely** - Dependency updates, version bumps, Cargo.lock changes + +### VERBOSITY REQUIREMENTS: +- **Minimum 3-4 bullet points per section** when changes exist +- **Each bullet point should be 1-2 sentences** explaining both what changed and why it matters +- **For new features**: Explain what the feature does and what problem it solves +- **For bug fixes**: Describe the problem that was fixed (inferred from the code changes) +- **For internal changes**: Explain the architectural or structural improvement + +### EXAMPLES OF GOOD vs BAD: + +#### ❌ BAD EXAMPLES: +- "Enhanced: Chain analysis with sophisticated blockchain processing capabilities" +- "Updated: brk_rolldown from 0.0.1 to 0.1.0 with comprehensive bundling improvements" +- "Version Bump: Updated all crate versions from 0.0.61 to 0.0.62" +- "Improved error handling" +- "Refactored codebase" +- "Updated dependencies" + +#### ✅ GOOD EXAMPLES WITH ANALYSIS: + +**Analysis**: "Looking at the diff, I see a new `TransactionAnalyzer` struct was added with methods `calculate_fee()` and `is_coinbase()`. The struct takes transaction data and provides analysis methods. This enables users to programmatically analyze transaction properties." + +**Changelog**: "Added new `TransactionAnalyzer` struct that provides methods for computing transaction fees and detecting coinbase transactions" + +**Analysis**: "The diff shows error handling was added around block parsing where previously there was an unwrap(). Now it returns a Result and handles the empty block case explicitly. This prevents panics when processing malformed blocks." + +**Changelog**: "Fixed panic when processing blocks with zero transactions by adding explicit empty block handling and proper error propagation" + +**Analysis**: "I see a new caching layer was implemented with a HashMap storing block hashes as keys and block data as values. The API endpoints now check this cache before making network requests. This should improve performance for repeated queries." + +**Changelog**: "Implemented new caching layer for blockchain queries, reducing API response time by storing frequently accessed block data in memory" + +#### ❌ BAD EXAMPLES (NO UNDERSTANDING): +- "Enhanced error handling" (What specific errors? How were they enhanced?) +- "Improved performance" (What was improved? How?) +- "Updated transaction logic" (What specific logic? What changed?) + +## FINAL REMINDER: +**PROCESS ONLY ONE RELEASE. THEN STOP AND WAIT FOR MY CONFIRMATION.** + +You must be thorough and verbose - if there are code changes, there should be substantial changelog content explaining what those changes accomplish. diff --git a/.claude/prompts/readmes.md b/.claude/prompts/readmes.md new file mode 100644 index 000000000..62f28ed3e --- /dev/null +++ b/.claude/prompts/readmes.md @@ -0,0 +1,141 @@ +# README Generation Prompt + +Generate a professional, comprehensive README.md for each crate based SOLELY on code analysis. Use NO external documentation, commit messages, or existing READMEs. + +## MANDATORY PROCESS - FOLLOW EXACTLY: +1. Analyze each crate's source code thoroughly using file system exploration +2. **MANDATORY CODE ANALYSIS**: Before writing ANY README content, you MUST: + - Examine all Rust files in src/ directory + - Identify the main structs, enums, traits, and functions + - Understand the crate's architecture and data flow + - Determine the crate's purpose from its implementation + - Map dependencies to understand external integrations +3. Generate one complete README.md per crate +4. Focus on one crate at a time for thorough analysis + +## ABSOLUTE REQUIREMENTS: +- **SOURCE OF TRUTH**: Use ONLY the actual Rust code - no external docs, comments may provide hints but focus on implementation +- **PROFESSIONAL GRADE**: Write as if this will be published on crates.io for other developers +- **PROGRAMMER FOCUSED**: Assume audience knows Rust and relevant domain concepts +- **IMPLEMENTATION-BASED**: Describe what the code actually does, not what comments claim it should do +- **If you cannot determine functionality from code alone, state this explicitly** + +## README STRUCTURE (MANDATORY): + +### 1. CRATE HEADER +```markdown +# Crate Name + +Brief one-line description of what this crate does (max 80 chars). + +[![Crates.io](https://img.shields.io/crates/v/CRATE_NAME.svg)](https://crates.io/crates/CRATE_NAME) +[![Documentation](https://docs.rs/CRATE_NAME/badge.svg)](https://docs.rs/CRATE_NAME) +``` + +### 2. OVERVIEW SECTION +- **Purpose**: What problem does this crate solve? +- **Key Features**: 3-5 bullet points of main capabilities (derived from code analysis) +- **Target Use Cases**: Who would use this and for what? + +### 3. INSTALLATION +```toml +[dependencies] +crate_name = "X.Y.Z" +``` + +### 4. QUICK START / USAGE +- **Minimal working example** showing the primary API +- **Common patterns** observed in the code +- **Key structs/traits** that users will interact with + +### 5. API OVERVIEW +- **Core Types**: Main structs, enums, traits with brief descriptions +- **Key Methods**: Most important public functions +- **Module Structure**: Brief overview of how code is organized + +### 6. FEATURES (if applicable) +- Cargo features and what they enable +- Optional dependencies and their purpose + +### 7. EXAMPLES +- 2-3 practical code examples showing different use cases +- Based on public API analysis, not existing examples + +## WRITING REQUIREMENTS: + +### TONE AND STYLE: +- **Concise but comprehensive**: Every sentence must add value +- **Technical precision**: Use exact terminology, avoid marketing speak +- **Active voice**: "Provides X" not "X is provided" +- **Present tense**: "The crate handles..." not "The crate will handle..." + +### FORBIDDEN PATTERNS: +- **NEVER** use vague terms: "powerful", "flexible", "robust", "comprehensive", "advanced" +- **NEVER** write marketing copy: "cutting-edge", "state-of-the-art", "enterprise-grade" +- **NEVER** make claims you can't verify from code: "blazingly fast", "memory efficient" +- **NEVER** copy-paste from existing documentation or comments + +### REQUIRED SPECIFICITY: +- **Data structures**: Mention specific types (HashMap, Vec, etc.) +- **Algorithms**: Reference actual implementations found in code +- **Integration points**: Specific traits implemented, dependencies used +- **Error handling**: How errors are represented and handled +- **Async/sync**: Clearly state if operations are blocking or async + +### CODE ANALYSIS DEPTH: +**You MUST analyze and understand:** +1. **Public API surface**: All pub structs, functions, traits, modules +2. **Core abstractions**: Main data types and their relationships +3. **Error types**: Custom errors, Result patterns, panic conditions +4. **Dependencies**: How external crates are integrated +5. **Feature flags**: Conditional compilation and optional functionality +6. **Async patterns**: Use of futures, tokio, async-std, etc. +7. **Serialization**: Serde implementations, custom serialization +8. **Performance characteristics**: Algorithm complexity where obvious + +### EXAMPLE STRUCTURE ANALYSIS OUTPUT: +```markdown +## Code Analysis Summary +**Main Types**: `BlockProcessor`, `Transaction`, `ValidationError` +**Core Trait**: `Validator` - implemented by `BasicValidator` and `StrictValidator` +**Async Support**: All processing methods return `impl Future` +**Error Handling**: Custom `ValidationError` enum with specific error types +**Dependencies**: Uses `tokio` for async runtime, `serde` for serialization +**Architecture**: Pipeline pattern with configurable validation stages +``` + +## EXAMPLES OF QUALITY: + +### ❌ BAD (VAGUE): +```markdown +# My Crate +A powerful and flexible library for blockchain operations. + +## Features +- Fast processing +- Easy to use +- Robust error handling +``` + +### ✅ GOOD (SPECIFIC): +```markdown +# brk-chain-analyzer +Bitcoin blockchain analysis tools for transaction pattern detection. + +## Overview +Provides utilities for analyzing Bitcoin transaction data, detecting address clustering patterns, and computing blockchain statistics. Built around a streaming parser that processes block data without loading entire blocks into memory. + +## Key Types +- `TransactionAnalyzer`: Stateful analyzer for computing fees, detecting coinbase transactions +- `ClusterDetector`: Implements common input ownership heuristics for address clustering +- `BlockStream`: Async iterator over blockchain data with configurable batch sizes +``` + +## FINAL REQUIREMENTS: +- **One README per crate** - don't combine multiple crates +- **Minimum 200 words** - be thorough but concise +- **Maximum 800 words** - stay focused and relevant +- **Code examples must be syntactically correct** and compilable +- **All claims must be verifiable** from the source code + +**PROCESS ONE CRATE AT A TIME. ANALYZE THE CODE THOROUGHLY BEFORE WRITING.** diff --git a/.gitignore b/.gitignore index 86d579031..5a7b25df7 100644 --- a/.gitignore +++ b/.gitignore @@ -25,4 +25,4 @@ flamegraph.svg *.trace # AI -.claude +.claude/settings* diff --git a/docs/.gitignore b/docs/.gitignore index de7c0e451..2b89b9ddb 100644 --- a/docs/.gitignore +++ b/docs/.gitignore @@ -1,3 +1,2 @@ -CLAUDE*.md OPENSATS*.md DUMP.md diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 40a24e2ec..422b38a27 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,91 @@ +## [v0.0.107](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.107) - 2025-09-13 + +### New Features +- **Comprehensive Mining Analytics**: Added extensive Bitcoin mining dataset including hash rate calculations, difficulty analysis, target block metrics, and mining performance indicators, enabling detailed analysis of network security and mining ecosystem health ([crates/brk_computer/src/chain.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.107/crates/brk_computer/src/chain.rs)) +- **Enhanced Cointime Economics**: Expanded cointime analysis with advanced economic metrics, value flow calculations, and temporal Bitcoin analysis, providing deeper insights into HODLing behavior and economic activity patterns ([crates/brk_computer/src/cointime.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.107/crates/brk_computer/src/cointime.rs)) +- **Transaction Volume and Velocity Analysis**: Implemented annualized transaction volume calculations, transaction velocity metrics, and sent volume analysis, enabling comprehensive understanding of Bitcoin's economic activity and monetary flow ([crates/brk_computer/src/stateful/common.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.107/crates/brk_computer/src/stateful/common.rs)) +- **Advanced Chain Metrics**: Added block target analysis, hash rate per block calculations, and network difficulty tracking, providing granular insights into Bitcoin's proof-of-work security mechanism + +### Breaking Changes +- **Data Field Naming Standardization**: Renamed all currency denomination fields from verbose forms (`_in_usd`, `_in_btc`) to concise forms (`_usd`, `_btc`) across all datasets, affecting API field names and database schemas for improved consistency and reduced verbosity +- **Iterator-Based Processing Architecture**: Converted vector computation functions from callback-based `compute_all` patterns to iterator-based processing, fundamentally changing the internal API for stateful computations and improving memory efficiency + +### Performance Improvements +- **Computation Engine Refactoring**: Simplified complex computation functions with iterator-based patterns, reducing memory overhead and improving processing speed for large-scale Bitcoin dataset analysis +- **Vector System Optimization**: Removed redundant builder_computed module, streamlining the vector generation pipeline and reducing code complexity while maintaining functionality +- **Stateful Processing Enhancement**: Refactored stateful computation system for better scalability and performance when processing cohort-based analytics + +### Build System +- **Development Profile Optimization**: Added optimized development profile with thin LTO, level 3 optimization, and single codegen unit for faster development iterations while maintaining performance ([Cargo.toml](https://github.com/bitcoinresearchkit/brk/blob/v0.0.107/Cargo.toml)) +- **Clippy Integration Profile**: Created dedicated Clippy profile with optimized settings for fast linting, including no LTO, maximum codegen units, and minimal overhead for improved developer experience +- **Release Profile Enhancement**: Enhanced release profile with fat LTO, binary stripping, and panic abort for maximum production performance + +### Internal Changes +- **Mathematical Operations Expansion**: Added comprehensive mathematical operations to OHLC data structures and floating-point types, enabling more sophisticated financial calculations and analysis +- **Group Processing Enhancement**: Improved address and UTXO grouping capabilities with better filtering mechanisms and enhanced processing efficiency +- **Data Structure Consistency**: Standardized timestamp and date handling across all modules for better temporal data processing and analysis accuracy + +[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.106...v0.0.107) + +## [v0.0.106](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.106) - 2025-09-09 + +### New Features +- **Fibonacci Mathematical Constants**: Added constant_38_2 (38.2) and constant_61_8 (61.8) for Fibonacci retracement level calculations, commonly used in technical analysis and price level identification ([crates/brk_computer/src/constants.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.106/crates/brk_computer/src/constants.rs)) +- **Advanced Price Analysis Tools**: Implemented MinMax bands for 1-week, 2-week, 1-month, and 1-year timeframes, True Range indicator, and Choppiness Index with Fibonacci level markers for comprehensive price action analysis ([websites/bitview/scripts/options.js](https://github.com/bitcoinresearchkit/brk/blob/v0.0.106/websites/bitview/scripts/options.js)) +- **Enhanced OHLC Data Handling**: Added conversion methods for Open, High, and Low dollar values to base Dollars type, enabling more flexible price data manipulation and calculations ([crates/brk_structs/src/structs/dollars.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.106/crates/brk_structs/src/structs/dollars.rs)) + +### Breaking Changes +- **Percentile Naming Standardization**: Renamed all percentile fields from abbreviated forms (p10, p25, p75, p90) to full prefix forms (pct10, pct25, pct75, pct90) across statistical computation systems, affecting database schemas and API field names ([crates/brk_computer/src/grouped/builder_eager.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.106/crates/brk_computer/src/grouped/builder_eager.rs)) +- **Price Data Key Migration**: Updated website price data mapping from generic keys (ohlc_in_sats, close_in_sats) to prefixed keys (price_ohlc_in_sats, price_close_in_sats) for better data organization and namespace clarity + +### Build System +- **Rust Toolchain Standardization**: Added explicit Rust toolchain specification to version 1.89.0, ensuring consistent compilation environment across different development setups and CI/CD systems ([rust-toolchain.toml](https://github.com/bitcoinresearchkit/brk/blob/v0.0.106/rust-toolchain.toml)) + +### Internal Changes +- **Data Serialization Improvements**: Enhanced table data serialization with explicit null value handling, preventing serialization errors when processing incomplete datasets +- **Price Detection Logic**: Expanded price-related data detection to include true_range metrics and improved pattern matching for price analysis features + +[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.105...v0.0.106) + +## [v0.0.105](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.105) - 2025-09-08 + +### Internal Changes +- **Release Preparation**: Maintenance release preparing infrastructure for upcoming feature releases with codebase consistency improvements + +[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.104...v0.0.105) + +## [v0.0.104](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.104) - 2025-09-08 + +### Build System +- **Cargo Dist Upgrade**: Updated cargo-dist release tooling from v0.29.0 to v0.30.0, providing improved build automation, better cross-platform compilation support, and enhanced release artifact generation ([.github/workflows/release.yml](https://github.com/bitcoinresearchkit/brk/blob/v0.0.104/.github/workflows/release.yml)) +- **Cross-Platform Build Configuration**: Added comprehensive build configuration supporting multiple architectures including aarch64-apple-darwin, aarch64-unknown-linux-gnu, and x86_64-unknown-linux-gnu for broader platform compatibility ([Cargo.toml](https://github.com/bitcoinresearchkit/brk/blob/v0.0.104/Cargo.toml)) +- **CI/CD Enhancement**: Configured flexible CI builds with allow-dirty option for release workflows and standardized on Rust toolchain version 1.89 for consistent build environments + +[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.103...v0.0.104) + +## [v0.0.103](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.103) - 2025-09-08 + +### New Features +- **Enhanced Volatility Analytics**: Added comprehensive Bitcoin price volatility metrics with 1-week, 1-month, and 1-year volatility calculations based on daily returns standard deviations, providing insights into market stability across different time horizons ([crates/brk_computer/src/market.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.103/crates/brk_computer/src/market.rs)) +- **Configurable Standard Deviation System**: Implemented flexible `StandardDeviationVecsOptions` builder pattern allowing selective computation of z-scores, standard deviation bands, and price bands, optimizing memory usage and computation time by only generating requested metrics ([crates/brk_computer/src/grouped/sd_from_dateindex.rs](https://github.com/bitcoinresearchkit/brk/blob/v0.0.103/crates/brk_computer/src/grouped/sd_from_dateindex.rs)) +- **Volatility Visualization Dashboard**: Added dedicated volatility indicators section to the web interface featuring color-coded charts for 1-week (red), 1-month (orange), and 1-year (lime) volatility metrics with proper percentage formatting ([websites/bitview/scripts/options.js](https://github.com/bitcoinresearchkit/brk/blob/v0.0.103/websites/bitview/scripts/options.js)) + +### Breaking Changes +- **Standard Deviation Architecture Refactor**: Converted all standard deviation fields from required to optional types using `Option<>` wrappers, enabling selective computation and reducing memory footprint for unused statistical measures +- **Z-Score Field Repositioning**: Moved `zscore` field from end of struct to logical position after `sd` field, improving code organization and data structure coherence + +### Internal Changes +- **Statistical Computation Optimization**: Enhanced standard deviation computation infrastructure with dedicated 1-day returns analysis for multiple time periods (7, 30, 365 days), enabling more granular volatility measurements +- **Option-Based Vector Generation**: Refactored statistical vector initialization to use option-based conditional generation, preventing unnecessary computation and storage of unused statistical measures +- **Version Management**: Updated build dependencies with brk_rolldown advancement from 0.1.4 to 0.1.5, incorporating latest bundling improvements and optimizations + +[View changes](https://github.com/bitcoinresearchkit/brk/compare/v0.0.101...v0.0.103) + + + ## [v0.0.101](https://github.com/bitcoinresearchkit/brk/releases/tag/v0.0.101) - 2025-09-07 ### Breaking Changes