Support for pytorch-lightning

[RoyBenjamin]

I'm using pytorch-lightning and am experiencing this issue TypeError: step() got an unexpected keyword argument 'closure' when using PESG. Would appreciate your assistance if you managed to use LibAUC and pytorch-lightning together.

Thanks for your time and amazing work!

Usage is as follows (some lines of code omitted to keep it concise):

import pytorch_lightning as pl
...
# for AUC margin loss
from libauc.losses import AUCMLoss
from libauc.optimizers import PESG

# implementation for the model used in self.classifier
...

class Module(pl.LightningModule):
# implementation of methods unrelated to the error message
...

  def configure_optimizers(self):
    opt = PESG(self.classifier, a=self.criterion.a, b=self.criterion.b, alpha=self.criterion.alpha,
               lr=self.learning_rate, gamma=self.gamma, margin=self.margin, weight_decay=self.weight_decay)
    return opt

[ahmed1996said]

Any progress with this? @RoyBenjamin

[RoyBenjamin]

Unfortunately no.

Any progress with this? @RoyBenjamin

[yzhuoning]

Thanks for your feedback! I just pushed an update that allows you to work with pytorch-lightning. You can use pip install -U libauc==1.1.9rc0 to update your local library. Let me know if this works for you.

[RoyBenjamin]

From my initial testing it seems to work perfect, I have one fix to suggest. For libuc/losses/losses.py in AUCM_MultiLabel change to what I have here. This solves a device issue, the reason it is written this way is for consistency with what you have for AUCMLoss.

def __init__(self, margin=1.0, imratio=None, device=None):
    super(AUCMLoss, self).__init__()
    if not device:
        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    else:
        self.device = device
    self.margin = margin
    self.p = imratio
    # https://discuss.pytorch.org/t/valueerror-cant-optimize-a-non-leaf-tensor/21751
    self.a = torch.zeros(1, dtype=torch.float32, device=self.device, requires_grad=True).to(self.device)
    self.b = torch.zeros(1, dtype=torch.float32, device=self.device, requires_grad=True).to(self.device)
    self.alpha = torch.zeros(1, dtype=torch.float32, device=self.device, requires_grad=True).to(self.device)

[yzhuoning]

Good suggestion! I will fix it in next update. Please reopen the issue if you have other questions.