Skip to content

0.36.0

SKaiNET-transformers 0.36.0
@michalharakal michalharakal tagged this 12 Jul 08:52 
Ships against SKaiNET engine 0.36.0.

Headline: BERT is now completely defined on the DSL path, and the
deprecated hand-coded eager BERT stack is removed (BREAKING).

Added
- bertNetwork() is a numerically complete tokens -> hidden-states encoder:
  new BertEmbeddings module (word + absolute-position + token-type,
  index-free narrow-based lookups; traced graph has exactly one
  non-parameter leaf), post-norm residual wiring via two blocks per layer.
- BertEncoderRuntime: DIRECT (eager, default) or OPTIMIZED (traced,
  LLM-pipeline-optimized ComputeGraph, shape-specialized with LRU;
  bit-exact vs DIRECT) + masked mean pooling, optional bias-free 2_Dense
  projection, L2 normalization; exportTape() lowers to StableHLO.
- One-call factories: BertEmbeddingModel.fromHuggingFace(...) /
  fromSafeTensors(...) behind the neutral EmbeddingModel SPI, with
  built-in HF Hub download (HF_TOKEN-aware, cached, offline-safe).
- T5 encoder-decoder runtime (llm-inference/t5) with GtrEmbedder, and
  vec2text embedding inversion (llm-inference/vec2text): greedy corrector
  loop reconstructing text from GTR embeddings, verified end-to-end on
  real gtr-t5-base weights.
- New docs: "Getting Started with LEAF" tutorial and "BERT Completely
  Defined in the DSL" explanation.

Fixed
- BERT post-norm residual wiring (FFN residual grabbed the pre-LayerNorm
  value).
- Bias-free 2_Dense projection heads are applied instead of silently
  dropped (LEAF ships bias=false; embeddings now project as advertised).
- Multi-token attention traces: engine 0.36.0's permute-axes replay fix
  (SKaiNET#803) is consumed; the interim local handler is removed.

Removed (BREAKING)
- BertRuntime, BertRuntimeWeights/BertLayerWeights, loadBertWeights,
  BertWeightMapper, BertTensorNames, BertIngestion. Migrate to
  createBertEncoderRuntime(config, tensors, ctx) or
  BertEmbeddingModel.fromSafeTensors(...)/fromHuggingFace(...).

Performance
- Downstream leaf-cli e2e: the 56-chunk reference corpus indexes in
  44.5 s vs 676.9 s on the removed eager runtime (~15x), identical
  embeddings.

Full notes: CHANGELOG.md, section 0.36.0.
Assets 2
Loading