Adaptive Linesearch Algorithm
-
Updated
Jan 18, 2022 - Python
Adaptive Linesearch Algorithm
This directory contains the source code of the experiments as shown in our main paper. It is still work in progress.
Super-Convergence on CIFAR10
Regularization, Bayesian Model Selection and k-fold Cross-Validation Selection
sophia optimizer further projected towards flat areas of loss landscape
Stochastic Second-Order Methods in JAX
NG+: A new second-order optimizer for deep learning
Newton’s second-order optimization methods in python
The repository contains code to reproduce the experiments from our paper Error Feedback Can Accurately Compress Preconditioners available below:
Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning
Implementation of PSGD optimizer in JAX
FOSI library for improving first order optimizers with second order information
Minimalist deep learning library with first and second-order optimization algorithms made for educational purpose
This package is dedicated to high-order optimization methods. All the methods can be used similarly to standard PyTorch optimizers.
Hessian-based stochastic optimization in TensorFlow and keras
PyTorch implementation of the Hessian-free optimizer
Tensorflow implementation of preconditioned stochastic gradient descent
This repository implements FEDL using pytorch
FEDL-Federated Learning algorithm using TensorFlow (Transaction on Networking 2021)
Distributed K-FAC Preconditioner for PyTorch
Add a description, image, and links to the second-order-optimization topic page so that developers can more easily learn about it.
To associate your repository with the second-order-optimization topic, visit your repo's landing page and select "manage topics."