Skip to content

Release 4.4.0

Choose a tag to compare

View all tags
@dylanuys dylanuys released this 31 Jan 06:30
· 95 commits to main since this release
v4.4.0
6e593ac
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Summary

  • Fix generator reward logic for sharper penalties when generative miners stop responding to queries
  • Improve media format detection with proper ftyp-based MP4 detection and support for additional formats
  • Fix file format handling - detect actual format from magic bytes instead of hardcoding extensions/content-types
  • Add dynamic escrow address fetching from the API with fallback to defaults
  • Add prompt modality tracking to ensure prompts are matched to their intended modality
  • Add prompt cache cleanup to delete prompts after use, preventing unbounded cache growth
  • Add detailed error responses for generative callback failures so miners can debug issues
  • Add webhook stats tracking for miners to monitor success/failure rates per validator
  • Add C2PA verification CLI for testing C2PA credentials on local files
  • Remove LocalService from registry (since it doesn't produce C2PA-signed content)

Changes

Generator Miner Rewards Fix

  • Fixed bug where include_all=True was double-counting already-rewarded media in verification stats
  • Changed reward eligibility from union to intersection: miners must now have both local verified submissions and benchmark results to receive rewards
  • Changed base reward default from 1e-4 to 0 for generators without verified submissions
  • Increased EMA alpha from 0.2 to 0.5 for more aggressive score decay on inactive miners
  • Added hard cutoff to zero out scores for generators not active within the liveness window

Media Format Detection (#311)

  • Fixed MP4 detection by checking for ftyp at offset 4 instead of hardcoded box sizes
  • Consolidated all ftyp-based format detection to fix unreachable M4A code path
  • Added support for new image formats: HEIC/HEIF, AVIF, GIF, BMP, TIFF
  • Added support for new video formats: MOV, 3GP, AVI
  • Added support for audio formats: MP3, WAV, FLAC, OGG, M4A
  • Added length checks to prevent index errors on short data
  • Improved JPEG detection with 3-byte signature

Dynamic Escrow Addresses (#322)

  • Added new get_escrow_addresses() function to fetch escrow addresses from the gas-api to provide the option to punish weight copiers.
  • Validators now dynamically fetch video_escrow, image_escrow, and audio_escrow addresses
  • Escrow address eligibility based validators provably running latest SN code.

Prompt Modality Support

  • Added modality column to prompts table (image, video, audio)
  • Database migration automatically adds column to existing installations
  • Fixed database migration order to prevent "no such column: modality" errors on existing databases
  • Prompts are now stored with their intended modality when generated
  • Challenge manager selects modality first, then samples matching prompts

Prompt Cache Cleanup

  • Fixed remove parameter in prompt sampling to actually delete prompts instead of just incrementing used_count
  • Added min_prompts_threshold (default 100) to prevent running out of prompts
  • Enabled prompt cleanup in generative challenge manager - prompts are now deleted after use (if enough remain)

Generative Callback Error Responses

  • Miners now receive detailed error messages when their submissions are rejected:
    • 400 - Empty binary payload for empty uploads
    • 400 - Corrupted or unreadable media for invalid media files
    • 400 - Duplicate content detected for perceptual hash matches
    • 400 - C2PA verification failed: no C2PA manifest for missing credentials
    • 400 - C2PA verification failed: untrusted issuer for non-trusted sources
  • Previously, all validation failures returned 200 OK silently, making it impossible for miners to debug issues
  • Made c2pa a hard dependency - removed conditional C2PA_AVAILABLE checks
  • Cleaned up inline imports, moved all imports to module scope

Miner Webhook Stats Tracking

  • Added WebhookStatsTracker to track webhook success/failure rates per validator IP
  • Stats are persisted to ~/.bitmind/webhook_stats.json and survive restarts
  • Automatic behaviors:
    • Saves to disk every 60 seconds (debounced to avoid excessive I/O)
    • Prints summary to logs every 5 minutes
    • Daily rotation at midnight - archives to webhook_stats_archive/webhook_stats_YYYY-MM-DD.json
    • Auto-cleanup of archives older than 7 days
  • Failure types are categorized:
    • empty_payload - validator received empty data
    • connection_timeout - connection timed out
    • connection_error - network/connection issues
    • http_400, http_401, http_404, http_500 - HTTP status codes
  • Helper functions available:
    • print_webhook_stats() - print summary to logs
    • get_webhook_stats_json() - get stats as dict
    • reset_webhook_stats() - clear all stats

LocalService Removed from Service Registry

  • LocalService (local Stable Diffusion/AnimateDiff) removed from SERVICE_MAP
  • Reason: LocalService doesn't produce C2PA-signed content, which validators now require
  • The service class remains in the codebase for potential future use
  • Valid services are now: openai, openrouter, stabilityai, or none
  • If no valid service is configured for a modality, requests for that modality will be rejected
  • Miners still free to generate whatever

C2PA Verification CLI

  • Added gascli generator verify-c2pa (standalone helper script at neurons/generator/helper/verify_c2pa.py) command for gen miners to test local files for C2PA credentials
  • Useful for miners to verify their generated content has valid C2PA before sending to validators
  • Supports --verbose for detailed output and --json for machine-readable output
Assets 2
Loading