Compare
Sorry, something went wrong.
No results found
IBRL - Beacon & Policy VM for Hypercore Cluster (#20)
* feat: IBRL Phase 1 - Beacon Module Integration
Integrated IBRL (Incentivized Bandwidth Resource Layer) beacon functionality
into Hypercore to enable path-aware, economically-incentivized workload routing.
## Changes
### Protocol Buffer Extensions
- Added BeaconMetadata message (latency, jitter, packet loss, queue depth, price/GB, reputation)
- Added BeaconAttestation, PolicyContext, WorkloadProof messages for future phases
- Extended NodeStateResponse to include beacon metadata
- Added new cluster events: BEACON_ATTEST, POLICY_QUERY, PROOF_VERIFY
### New Beacon Package (pkg/beacon/)
- client.go: Beacon client with ed25519 cryptographic signing, metrics tracking
- registry.go: Node registry with metric-based filtering and stale node cleanup
- attestation.go: Cryptographic attestation generation and verification
### Cluster Integration
- Integrated beacon client into Agent struct in pkg/cluster/serf.go
- Added beacon metadata to NodeStateResponse broadcasts in monitorWorkloads()
- Added IBRL Prometheus metric: hypercore_ibrl_beacon_connected
### CLI Enhancement
- Added 'hypercore cluster metrics' command to display IBRL node metrics
- Shows latency, price, reputation, queue depth per node
## Architecture
- Zero-dependency standalone operation (no external beacon required)
- Thread-safe metric updates with mutex protection
- Ed25519 signatures for cryptographic attestations
- Backwards compatible with existing clusters
## Build Status
✅ Build successful
✅ Proto generation complete
✅ No breaking changes
Total: ~766 lines added across 3 new files + 5 modified files
See IBRL_PHASE1_SUMMARY.md for complete details.
* docs: add IBRL integration documentation
Added comprehensive documentation for IBRL integration planning and implementation:
- ARCHITECTURE_ANALYSIS.md (626 lines): Complete Hypercore codebase analysis
including cluster module architecture, Serf integration, workload scheduling,
and existing monitoring infrastructure
- IBRL_INTEGRATION_GUIDE.md (577 lines): Step-by-step integration guide with
proto extensions, package structure, integration points, testing strategy,
deployment checklist, and phased rollout plan
- QUICK_REFERENCE.md (337 lines): Quick reference for file locations, port
assignments, code entry points, and configuration defaults
- EXPLORATION_SUMMARY.txt: Summary of exploration findings and key discoveries
These documents provide essential context for IBRL development and serve as
reference material for Phase 2 (Policy VM) and Phase 3 (Proof & Settlement).
* feat: IBRL Phase 2 - Policy Engine Integration
Implemented intelligent, policy-based workload scheduling to replace first-fit
with multi-criteria node selection based on latency, price, reputation, and
queue depth metrics.
## Changes
### New Policy Engine Package (pkg/policy/)
- policy.go: JSON-based policy language with hard constraints and soft scoring
- engine.go: Policy evaluation engine with weighted node ranking
- Supports two modes: "enforce" (strict) and "permissive" (fallback)
- Thread-safe policy updates with mutex protection
### Policy Features
- Hard Constraints: max_latency_ms, max_price_per_gb, min_reputation_score,
max_queue_depth, max_packet_loss, max_jitter_ms
- Weighted Scoring: configurable weights for latency, price, reputation, queue
- Node Ranking: selects optimal node based on composite score
- Graceful Fallback: falls back to broadcast if policy selection fails
### Cluster Integration (pkg/cluster/serf.go)
- Added policyEngine to Agent struct
- Modified SpawnRequest() to use policy-based node selection
- SelectNodes() evaluates all cluster members against policy
- Tries nodes in priority order (highest score first)
- Logs policy decisions for observability
### CLI Enhancements
- Added --cluster-policy flag for cluster-wide default policy
- Added --policy flag for per-spawn policy override
- Modified config.go and flags.go to support policy file paths
### Example Policies (examples/policies/)
- low-latency.json: Prioritize low-latency nodes (60% latency weight)
- cost-optimized.json: Minimize cost (70% price weight)
- balanced.json: Equal weights across all metrics
- high-trust.json: Only use nodes with reputation >= 0.9
- permissive.json: Accept any available node (default)
### Documentation
- IBRL_E2E_TESTING_GUIDE.md (435 lines): Complete testing guide with
multi-node setup, policy testing scenarios, troubleshooting
- IBRL_PHASE2_SUMMARY.md: Architecture overview, scoring algorithm,
usage examples, performance characteristics
## Algorithm
For each node:
```
score = (latency_weight × latency_score) +
(price_weight × price_score) +
(reputation_weight × reputation_score) +
(queue_weight × queue_score)
```
Nodes ranked by score (descending), spawns attempted in priority order.
## Build Status
✅ Build successful
✅ No breaking changes
✅ Backward compatible (policies optional)
## Usage
Start cluster with policy:
```bash
sudo ./bin/hypercore cluster \
--cluster-policy examples/policies/balanced.json
```
Spawn with policy override:
```bash
./bin/hypercore cluster spawn \
--image-ref nginx:latest \
--policy examples/policies/low-latency.json
```
Total: ~774 lines added (2 new packages, 5 examples, 2 docs)
See IBRL_PHASE2_SUMMARY.md and IBRL_E2E_TESTING_GUIDE.md for details.
* fix beacon metadata propagates correctly, so the policy engine can evaluate all nodes and route intelligently. refactor: remove obsolete documentation files
Deleted tmp .md file
* feat: Enhance IBRL Beacon Integration and Metrics
- Added new configuration options for beacon endpoint, price, and reputation in Config struct.
- Updated ClusterCommand to handle containerd availability on Mac, providing user guidance for metrics and list commands.
- Modified Agent struct to include beacon parameters and updated NewAgent function to initialize the beacon client with these parameters.
- Implemented HTTP health check for the beacon endpoint, falling back to TCP connection if necessary.
- Enhanced workload monitoring to include latency and jitter metrics, improving decision-making for the policy engine.
These changes improve the robustness of the beacon integration and provide better observability for cluster performance.
* ci fix
---------
Co-authored-by: Claude <noreply@anthropic.com>