Problem with PolynomialLR

[Nacriema]

Thank you very much for sharing the code with us. Recently, I inspect and test all the code that you provided. I notice that your custom PolynomialLR inside the schedulers packet returns the result the same as the ConstantLR does. Please let me know if it is a bug. Thank you in advance.

[monniert]

Hi thanks for the interest in the project! Indeed after a quick inspection this part is buggy, it is a feature I started to implement but I have never used it in the end. I will remove it, if you are interested in other schedulers there are now great built-in modules in pytorch (see https://pytorch.org/docs/stable/optim.html)
Best, Tom

[Nacriema]

Hi, thank you for your response. Your code is very clean, I learned a lot from your coding style. Thank you very much !

[Nacriema]

Hi, from this discussion thread: https://discuss.pytorch.org/t/solved-learning-rate-decay/6825/4, I have been implemented the PolynomialLR scheduler, but I'm not sure if it is correct. Could you please check it, thank you in advance:

class PolynomialLR(_LRScheduler):
    def __init__(self, optimizer, max_iter, decay_iter=1, power=0.9, last_epoch=-1):
        self.decay_iter = decay_iter
        self.max_iter = max_iter
        self.power = power
        super(PolynomialLR, self).__init__(optimizer, last_epoch)

    def get_lr(self):
        if self.last_epoch % self.decay_iter == 0 or self.last_epoch % self.max_iter == 0:
            factor = (1 - self.last_epoch / float(self.max_iter)) ** self.power
            return [base_lr * factor for base_lr in self.base_lrs]
        return [group['lr'] for group in self.optimizer.param_groups]

    def __str__(self):
        params = [
            'optimizer: {}'.format(self.optimizer.__class__.__name__),
            'decay_iter: {}'.format(self.decay_iter),
            'max_iter: {}'.format(self.max_iter),
            'gamma: {}'.format(self.gamma),
        ]
        return '{}({})'.format(self.__class__.__name__, ', '.join(params))

[monniert]

Mmh from a quick look it looks good to me, to be sure you should try on a basic example and print the LR when it changes during training

[Nacriema]

Hi, this is the learning rate scheme I've got for testing:

• With max_iter=100, decay_iter=1, power=1.0
  
• With max_iter=100, decay_iter=5, power=1.0
  
• With max_iter=100, decay_iter=1, power=2.0
  
• With max_iter=100, decay_iter=5, power=2.0