v0.3.0

What's Changed

• Take a layer's name first and its hyperparameters by keyword by @jessegrabowski in #140
• Add linear one-cycle schedule by @nightcityblade in #116
• Number repeated names so an unnamed stack can be checkpointed by @jessegrabowski in #141
• Save the generator and optimizer state a checkpoint was missing by @jessegrabowski in #142
• Detect draws that share a generator inside an inner graph by @jessegrabowski in #143
• Let the loss decide the target it reads by @jessegrabowski in #144
• Count every application of a reused BatchNorm in its statistics by @jessegrabowski in #145
• Carry a BatchNorm's statistics through the loop it runs in by @jessegrabowski in #146
• Fold a degenerate dropout away during rewrites by @jessegrabowski in #147
• Substitute the running statistics for a batch norm's write-back outputs by @jessegrabowski in #148
• Specialize the graph a loop runs at every step for prediction by @jessegrabowski in #149
• Read a model's data inputs off the graph instead of hardcoding one by @jessegrabowski in #150
• Let each rule name the state it allocates by @jessegrabowski in #151
• Add a GroupNorm layer by @jessegrabowski in #152
• Add nearest-neighbor and bilinear Upsample layers by @jessegrabowski in #153
• Add the QuickGELU activation by @jessegrabowski in #154
• Add a gated feed-forward variant by @jessegrabowski in #155
• Add cross-attention to MultiheadAttention by @jessegrabowski in #156
• Load HuggingFace checkpoints through a builder registry by @jessegrabowski in #157
• Load and run the SDXL VAE from its diffusers checkpoint by @jessegrabowski in #158
• Add an RMSNorm layer and fix float16 normalization returning NaN by @jessegrabowski in #159
• Make safetensors an optional dependency by @jessegrabowski in #161
• Allocate optimizer state afresh on every rule invocation by @jessegrabowski in #163
• Add an L-BFGS rule with a ring-buffered memory and an mlx dispatch by @jessegrabowski in #162
• Split the layers CI group and cut its redundant tests by @jessegrabowski in #167
• Add a zoom (strong Wolfe) line search by @jessegrabowski in #169
• Search every L-BFGS step with a zoom line search by default by @jessegrabowski in #170
• Align the JAX causal mask bottom-right when queries and keys differ by @jessegrabowski in #172
• Build attention's scale and causal mask at the queries' dtype by @jessegrabowski in #173

New Contributors

• @nightcityblade made their first contribution in #116

Full Changelog: v0.2.2...v0.3.0