Python SQL Parser and Transpiler

torch-optimizer -- collection of optimizers for Pytorch

On the Variance of the Adaptive Learning Rate and Beyond

AdalFlow: The library to build & auto-optimize LLM applications.

SAM: Sharpness-Aware Minimization (PyTorch)

PyTorch native quantization and sparsity for training and inference

Portfolio optimization and back-testing.

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

Scour - An SVG Optimizer / Cleaner

Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models

TorchOpt is an efficient library for differentiable optimization built upon PyTorch.

AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights (ICLR 2021)

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

End-to-end Generative Optimization for AI Agents

lookahead optimizer (Lookahead Optimizer: k steps forward, 1 step back) for pytorch

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793

A tool to automate and optimize DraftKings and FanDuel lineup construction.

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch