Clorch v0.4.0

Production training and tensor primitives developed while implementing TabICLv2.

Added

• Added PyTorch-compatible empty-like, zeros-like, ones-like, full-like, rand-like, and randn-like tensor factories. — @antlobach
• Added std, prod, median, nanmean, cummax, isin, unique, scatter, contiguous, and expand-as tensor operations. — @antlobach
• Added clorch.nn.functional/one-hot. — @antlobach
• Added public AMP unscale! with correct single-unscale bookkeeping before gradient clipping. — @antlobach
• Added recursive set-trainable! and retained CPU clone-flat-state-dict for parameter freezing and best-checkpoint restoration. — @antlobach
• Added bias-free LayerNorm support compatible with official TabICLv2 regressor checkpoints. — @antlobach
• Added optimizer parameter-group learning-rate inspection and mutation for learning-rate schedulers. — @antlobach

Fixed

• Kept combined floating attention and padding masks on the source CUDA device. — @antlobach
• Made training/evaluation mode traversal ignore scalar and map configuration values. — @antlobach

Documentation and verification

• Documented the new tensor, module, AMP lifecycle, distributed-training, and optimizer APIs. — @antlobach
• Added CPU behavioral coverage and CUDA regression coverage for the new paths. — @antlobach
• Verified 23 focused JVM 25 tests with 82 assertions and no failures or errors on RunPod. — @antlobach
• Verified CUDA, NCCL, DDP, AMP, fused attention, combined CUDA masks, checkpoint round-trips, and downstream TabICLv2 fine-tuning. — @antlobach

Clorch v0.3.1

Training-mode lifecycle patch for custom defmodel models.

Fixed

• Fixed nn/train and evaluation-mode changes for defmodel models containing direct Parameter fields. — @antlobach
• Stopped treating every Clojure collection as a trainable module; Parameter records are now correctly skipped during recursive mode changes. — @antlobach
• Preserved recursive mode propagation through module vectors and maps, which already implement IModule. — @antlobach
• Added behavioral regression coverage using a custom model containing LayerNorm parameters and Dropout; evaluation mode now reaches Dropout without attempting to apply module lifecycle operations to parameters. — @antlobach

Impact

• Fixes the No implementation of method: :-train ... for class: clorch.nn.Parameter failure that prevented models such as TabICL from starting an optimization loop. — @antlobach
• No public API signatures changed. — @antlobach

Verification

• GitHub CI passed on Java 21, 22, 23, 24, and 25. — @antlobach
• Focused model lifecycle suite passed: 3 tests, 11 assertions, 0 failures. — @antlobach
• clj-kondo passed with 0 errors and 0 warnings; cljfmt is clean. — @antlobach
• TabICL smoke configuration completed 10 scoped AdamW CUDA steps on RunPod with OpenJDK 25.0.4.1; cross-entropy decreased from 1.112408 to 0.666133. — @antlobach
• Full TabICL paper configuration completed CUDA forward, cross-entropy, backward, and AdamW update; all 276 parameter tensors received finite gradients. — @antlobach
• Full-configuration decoder weight sum changed from 1.683216 to 1.688014, proving optimizer mutation. — @antlobach

Install

io.github.antlobach/clorch
{:git/tag "v0.3.1"
 :git/sha "f09654e3ecd50fae0356d4f70d09cd85b89dab3b"}

Contributor

• @antlobach — lifecycle fix, regression coverage, and GPU training verification.

Full changelog: v0.3.0...v0.3.1

Clorch v0.3.0

Transformer, SafeTensors, model lifecycle, and runtime performance release.

Highlights

• Added reusable batch-first multihead-attention with packed QKV projection, self/cross-attention, attention and padding masks, causal mode, RoPE, and incremental key/value caching. — @antlobach
• Added transformer-encoder-layer with pre-norm and post-norm layouts plus ReLU, GELU, and custom activations. — @antlobach
• Added transformer-encoder stacks with optional per-layer key/value cache returns. — @antlobach
• Added safe, pickle-free SafeTensors loading, saving, metadata access, atomic writes, dtype/offset validation, and flat state-dict loading. — @antlobach
• Removed reflection from tensor operations, reducing small-operation dispatch from roughly 70–100 µs to roughly 2 µs in the contributor's measurements. — @michaelwhitford

Added

• Added clorch.nn.functional/combine-attention-masks for boolean and additive-float SDPA masks combined with module-style key-padding masks. — @antlobach
• Added interleaved adjacent-pair support to torch/precompute-rope-freqs and torch/apply-rope; the existing half-split layout remains the default. — @antlobach
• Added :device to torch/precompute-rope-freqs so tables can be constructed for CUDA inputs. — @antlobach
• Added clorch.nn.init/trunc-normal! using inverse-CDF truncated-normal initialization. — @antlobach
• Added nn/load-flat-state-dict! and nn/flat-parameter-names for native modules, records, parameters, tensors, and nested collections. — @antlobach
• Added primitive float[] tensor construction for binary checkpoint readers. — @antlobach
• Added transformer numerical-comparison scripts against PyTorch and CPU transformer benchmarks. — @antlobach
• Added transformer, SafeTensors, initialization, model lifecycle, and tensor regression coverage. — @antlobach
• Added SafeTensors, transformer, functional, initialization, tensor, and PyTorch-parity documentation. — @antlobach

Fixed

• Fixed defmodel recursive train and to traversal: generated methods now look up declared fields by keyword rather than incorrectly using field values as keys. — @antlobach
• Fixed defmodel conversion of direct Parameter fields and nested model data. — @antlobach
• Preserved leaf status and requires-grad when custom parameters move between devices or dtypes, keeping them optimizer-visible. — @antlobach
• Fixed torch/copy- to copy values instead of rebinding tensor storage with set_. — @antlobach
• Added a return type hint to ->tensor, type-hinted all affected JavaCPP call sites, enabled reflection warnings in every source namespace, and made release checks reject new reflection warnings. — @michaelwhitford
• Fixed max and min calls with a dimension, clip, dimensioned argsort, and native-module load-state-dict; the reflection audit exposed that these paths previously selected invalid overloads. — @michaelwhitford
• Cached native module forward-method lookup once per class because JavaCPP's base Module does not declare forward. — @michaelwhitford
• Stabilized linalg-inv-prop by generating bounded, strictly diagonally dominant float32 matrices instead of unseeded matrices with uncontrolled conditioning. — @antlobach

Documentation and project metadata

• Added context7.json for Context7 indexing. — @antlobach
• Claimed the Context7 library using its public key. — @antlobach
• Added Mintlify, DeepWiki, and Context7 README badges. — @antlobach
• Simplified compatibility matrices and documented tested native stacks. — @antlobach
• Updated installation coordinates for v0.2.0 after that release and for v0.3.0 in this release. — @antlobach

Compatibility

• Java 21 through 25. — @antlobach
• Clojure 1.12.x. — @antlobach
• JavaCPP PyTorch 2.10.0-1.5.13. — @antlobach
• CPU execution and single-device CUDA for the new transformer APIs; the existing distributed CUDA APIs remain available. — @antlobach

Verification

• GitHub CI passed on Java 21, 22, 23, 24, and 25. — @antlobach
• Full GPU release suite passed: 175 tests, 773 assertions, 0 failures, and no reflection warnings. — @antlobach
• CUDA/NCCL/DDP/AMP/fused-attention/checkpoint smoke passed. — @antlobach
• CUDA multi-head attention forward/backward produced finite output and input gradients. — @antlobach
• CUDA transformer encoder forward/backward gave all 12 parameters finite CUDA gradients. — @antlobach
• CUDA interleaved RoPE produced finite output; SafeTensors CUDA round-trip max error was 0.0. — @antlobach
• TabICL consumer GPU smoke produced finite CUDA logits of shape [2 3 3] across 94 parameters. — @antlobach
• PyTorch 2.1.2 numerical comparison max errors: self-attention 2.98e-8, cross-attention 5.96e-8, transformer layer 1.34e-6, and interleaved RoPE 0.0, all within the 2e-5 tolerance. — @antlobach
• The stabilized inverse property passed a 500-case probe, ten repeated namespace runs, and the Java 21–25 CI matrix. — @antlobach
• MkDocs strict build and changed-file clj-kondo checks passed. — @antlobach

Install

io.github.antlobach/clorch
{:git/tag "v0.3.0"
 :git/sha "6157e97d89f57789587bcbb4d7eee1fa94da68f2"}

Contributors

• @antlobach — transformer modules, SafeTensors, model lifecycle and checkpoint support, tests, GPU validation, and documentation.
• @michaelwhitford — reflection elimination, JavaCPP call-site fixes, and reflection-gated release checks.

Full changelog: v0.2.0...v0.3.0

Clorch v0.2.0

Distributed CUDA training release.

Highlights

• One managed worker JVM per CUDA device with rank environment setup, lifecycle management, failure propagation, file-backed logs, and cleanup.
• Direct NCCL process groups with all-reduce, broadcast, reduce, gather, scatter, all-to-all, point-to-point operations, barriers, and asynchronous work handles.
• Synchronous DistributedDataParallel with parameter verification, rank-zero initialization, gradient buckets, accumulation through no-sync, and synchronized optimizer steps.
• Deterministic distributed sampling, AMP for float16 and bfloat16, fused scaled-dot-product attention, and rank-zero checkpoints covering model, optimizer, sampler, scaler, RNG, and training state.
• Runnable two-GPU training example and setup guide covering Java, Clojure, CUDA, cuDNN, NCCL, launch, training, diagnostics, and checkpoint restore.
• CPU CI across Java 21, 22, 23, 24, and 25.

Runtime

• Java 21 through 25
• Clojure 1.12.x
• JavaCPP PyTorch 2.10.0-1.5.13
• Linux and NVIDIA GPUs for distributed training
• CUDA 13.1 user-space libraries, cuDNN 9, and NCCL 2

The managed launcher targets multiple GPUs on one host. It currently supports NCCL, requires one distinct CUDA device per rank, and does not provide FSDP, RPC, elastic membership, or tensor parallelism.

Verification

• GitHub CI: Java 21–25 matrix, 5 of 5 checks passed
• Linux CPU: 163 tests, 722 assertions, 0 failures, 0 errors
• Dual RTX A5000 CUDA: 163 tests, 722 assertions, 0 failures, 0 errors
• Two-rank DDP smoke: every model parameter synchronized
• Deterministic two-rank Clorch/PyTorch comparison: all 12 float32 losses and trained parameters matched exactly after float32 conversion
• MkDocs strict build passed
• clj-kondo: 0 errors and 0 warnings in changed Clojure files

Install

io.github.antlobach/clorch
{:git/tag "v0.2.0"
 :git/sha "07642acdbc522e8aa2a20cd223912247614d2239"}

Read the multi-GPU setup and training guide and the distributed API guide.

Clorch v0.1.0

First public Clorch release.

Highlights:

• Idiomatic Clojure API following PyTorch tensor, autograd, neural-network, optimizer, data, and device concepts.
• CPU execution plus automatic NVIDIA CUDA backend detection and explicit tensor/model placement.
• More than 100 tensor operations, broad neural-network coverage, and modern LLM building blocks including RMSNorm, RoPE, grouped-query attention, SwiGLU, KV caching, and autoregressive generation.
• 100% pass rate across 40 current cross-language numerical scenarios; approximately 81.7% of the repository's tracked feature catalog implemented.
• Deterministic native-memory scopes for repeated training and generation workloads.

Install from Clojure CLI:

io.github.antlobach/clorch {:git/tag "v0.1.0"
                            :git/sha "25a6b1005b7ada7259aaca680e9507d7eb4b03ac"}

Documentation: https://antlobach.github.io/clorch/
Parity details: https://antlobach.github.io/clorch/pytorch-parity/