Skip to content

v0.7.7

@nshkrdotcom nshkrdotcom tagged this 01 Jan 10:15
Add SnakeBridge.Defaults module centralizing all configurable values
with full documentation. This release addresses the 30s gRPC deadline
problem that broke LLM calls.

Runtime timeout changes:
- Default timeout increased from 30s to 120s (2 minutes)
- Default stream timeout increased from 5 minutes to 30 minutes
- Profile-based timeout system with built-in profiles:
  :default, :streaming, :ml_inference, :batch_job
- Per-library profile mapping via runtime config
- All runtime calls now apply timeout defaults via apply_runtime_defaults/3
- __runtime__ option is now a documented first-class feature

New configuration options:
- :introspector_timeout - Introspection timeout in ms (default: 30,000)
- :introspector_max_concurrency - Max concurrent introspection tasks
- :pytorch_index_base_url - PyTorch wheel index URL for private mirrors
- :cuda_thresholds - Extensible CUDA version to variant mapping
- :session_max_refs - Maximum refs per session (default: 10,000)
- :session_ttl_seconds - Session TTL in seconds (default: 3,600)

Introspector now supports both nested and flat key configuration for
backwards compatibility. Generated module docs include Runtime Options
section explaining timeout configuration.

Updates snakepit dependency to 0.8.8.
Assets 2
Loading