Skip to content

0.0.13

Choose a tag to compare

@github-actions github-actions released this 11 Nov 05:14
· 2 commits to main since this release
4adf7bb
IBRL - Beacon & Policy VM for Hypercore Cluster (#20)

* feat: IBRL Phase 1 - Beacon Module Integration

Integrated IBRL (Incentivized Bandwidth Resource Layer) beacon functionality
into Hypercore to enable path-aware, economically-incentivized workload routing.

## Changes

### Protocol Buffer Extensions
- Added BeaconMetadata message (latency, jitter, packet loss, queue depth, price/GB, reputation)
- Added BeaconAttestation, PolicyContext, WorkloadProof messages for future phases
- Extended NodeStateResponse to include beacon metadata
- Added new cluster events: BEACON_ATTEST, POLICY_QUERY, PROOF_VERIFY

### New Beacon Package (pkg/beacon/)
- client.go: Beacon client with ed25519 cryptographic signing, metrics tracking
- registry.go: Node registry with metric-based filtering and stale node cleanup
- attestation.go: Cryptographic attestation generation and verification

### Cluster Integration
- Integrated beacon client into Agent struct in pkg/cluster/serf.go
- Added beacon metadata to NodeStateResponse broadcasts in monitorWorkloads()
- Added IBRL Prometheus metric: hypercore_ibrl_beacon_connected

### CLI Enhancement
- Added 'hypercore cluster metrics' command to display IBRL node metrics
- Shows latency, price, reputation, queue depth per node

## Architecture
- Zero-dependency standalone operation (no external beacon required)
- Thread-safe metric updates with mutex protection
- Ed25519 signatures for cryptographic attestations
- Backwards compatible with existing clusters

## Build Status
✅ Build successful
✅ Proto generation complete
✅ No breaking changes

Total: ~766 lines added across 3 new files + 5 modified files

See IBRL_PHASE1_SUMMARY.md for complete details.

* docs: add IBRL integration documentation

Added comprehensive documentation for IBRL integration planning and implementation:

- ARCHITECTURE_ANALYSIS.md (626 lines): Complete Hypercore codebase analysis
  including cluster module architecture, Serf integration, workload scheduling,
  and existing monitoring infrastructure

- IBRL_INTEGRATION_GUIDE.md (577 lines): Step-by-step integration guide with
  proto extensions, package structure, integration points, testing strategy,
  deployment checklist, and phased rollout plan

- QUICK_REFERENCE.md (337 lines): Quick reference for file locations, port
  assignments, code entry points, and configuration defaults

- EXPLORATION_SUMMARY.txt: Summary of exploration findings and key discoveries

These documents provide essential context for IBRL development and serve as
reference material for Phase 2 (Policy VM) and Phase 3 (Proof & Settlement).

* feat: IBRL Phase 2 - Policy Engine Integration

Implemented intelligent, policy-based workload scheduling to replace first-fit
with multi-criteria node selection based on latency, price, reputation, and
queue depth metrics.

## Changes

### New Policy Engine Package (pkg/policy/)
- policy.go: JSON-based policy language with hard constraints and soft scoring
- engine.go: Policy evaluation engine with weighted node ranking
- Supports two modes: "enforce" (strict) and "permissive" (fallback)
- Thread-safe policy updates with mutex protection

### Policy Features
- Hard Constraints: max_latency_ms, max_price_per_gb, min_reputation_score,
  max_queue_depth, max_packet_loss, max_jitter_ms
- Weighted Scoring: configurable weights for latency, price, reputation, queue
- Node Ranking: selects optimal node based on composite score
- Graceful Fallback: falls back to broadcast if policy selection fails

### Cluster Integration (pkg/cluster/serf.go)
- Added policyEngine to Agent struct
- Modified SpawnRequest() to use policy-based node selection
- SelectNodes() evaluates all cluster members against policy
- Tries nodes in priority order (highest score first)
- Logs policy decisions for observability

### CLI Enhancements
- Added --cluster-policy flag for cluster-wide default policy
- Added --policy flag for per-spawn policy override
- Modified config.go and flags.go to support policy file paths

### Example Policies (examples/policies/)
- low-latency.json: Prioritize low-latency nodes (60% latency weight)
- cost-optimized.json: Minimize cost (70% price weight)
- balanced.json: Equal weights across all metrics
- high-trust.json: Only use nodes with reputation >= 0.9
- permissive.json: Accept any available node (default)

### Documentation
- IBRL_E2E_TESTING_GUIDE.md (435 lines): Complete testing guide with
  multi-node setup, policy testing scenarios, troubleshooting
- IBRL_PHASE2_SUMMARY.md: Architecture overview, scoring algorithm,
  usage examples, performance characteristics

## Algorithm

For each node:
```
score = (latency_weight × latency_score) +
        (price_weight × price_score) +
        (reputation_weight × reputation_score) +
        (queue_weight × queue_score)
```

Nodes ranked by score (descending), spawns attempted in priority order.

## Build Status
✅ Build successful
✅ No breaking changes
✅ Backward compatible (policies optional)

## Usage

Start cluster with policy:
```bash
sudo ./bin/hypercore cluster \
  --cluster-policy examples/policies/balanced.json
```

Spawn with policy override:
```bash
./bin/hypercore cluster spawn \
  --image-ref nginx:latest \
  --policy examples/policies/low-latency.json
```

Total: ~774 lines added (2 new packages, 5 examples, 2 docs)

See IBRL_PHASE2_SUMMARY.md and IBRL_E2E_TESTING_GUIDE.md for details.

* fix beacon metadata propagates correctly, so the policy engine can evaluate all nodes and route intelligently. refactor: remove obsolete documentation files

Deleted tmp .md file

* feat: Enhance IBRL Beacon Integration and Metrics

- Added new configuration options for beacon endpoint, price, and reputation in Config struct.
- Updated ClusterCommand to handle containerd availability on Mac, providing user guidance for metrics and list commands.
- Modified Agent struct to include beacon parameters and updated NewAgent function to initialize the beacon client with these parameters.
- Implemented HTTP health check for the beacon endpoint, falling back to TCP connection if necessary.
- Enhanced workload monitoring to include latency and jitter metrics, improving decision-making for the policy engine.

These changes improve the robustness of the beacon integration and provide better observability for cluster performance.

* ci fix

---------

Co-authored-by: Claude <noreply@anthropic.com>