BackOBS = BackPACK + DeepOBS

BackOBS allows you to use BackPACK on DeepOBS testproblems (in a limiting way, see details below)

⚠ The package and its API are experimental. Beware of rough edges!

Unsupported problems include:

• Variational autoencoder problems (mnist_vae, fmnist-vae)
• Problems with batch normalization (cifar100_wrn164, cifar100_wrn404, svhn_wrn164)

Installation

pip install -e git://github.com/f-dangel/backobs.git@master#egg=backobs

Basic examples

• How to extend a testproblem in DeepOBS with BackPACK (file):

  python example/extend.py
      

• How to extend a testproblem with BackPACK and get access to the unreduced loss (here):

  python example/extend_with_access_unreduced_loss.py
      

• A DeepOBS testproblem runner for SGD, extended with BackPACK functionality (here):

  python example/run.py mnist_logreg --lr 0.1
      

Important

• ℓ₂ regularization is not supported:

  # ℓ₂ not supported: this will crash!
  # python example/run.py cifar10_3c3d --lr 0.1
  
  # ℓ₂ disabled: works
  python example/run.py cifar10_3c3d --lr 0.1 --l2_reg 0.0