SSN [Paper].
Install the ssn optimizer.
pip install git+https://github.com/IssamLaradji/ssn.git
Install the Haven library for managing the experiments.
pip install -r requirements.txt
Use Ssn
in your code by adding the following script.
import ssn
opt = ssn.Ssn(model.parameters())
for epoch in range(100):
opt.zero_grad()
closure = lambda : torch.nn.MSELoss() (model(X), Y)
opt.step(closure=closure)
Run an a synthetic experiment with the logistic loss with the command below,
python trainval.py -e syn_logistic -sb ../results -r 1
where -e
is the experiment group, -sb
is the result directory.
Other experiment groups are defined in exp_configs.py
, which are the following:
- "syn_squared_hinge"
- "mushrooms_logistic"
- "mushrooms_squared_hinge"
- "ijcnn_logistic"
- "ijcnn_squared_hinge"
- "rcv1_logistic"
- "rcv1_squared_hinge"
@inproceedings{meng2020fast,
title={Fast and furious convergence: Stochastic second order methods under interpolation},
author={Meng, Si Yi and Vaswani, Sharan and Laradji, Issam Hadj and Schmidt, Mark and Lacoste-Julien, Simon},
booktitle={International Conference on Artificial Intelligence and Statistics},
pages={1375--1386},
year={2020}
}