Skip to content

Critical DDP bug fixes

Compare
Choose a tag to compare
@williamFalcon williamFalcon released this 27 Apr 13:06
· 8023 commits to master since this release
694f1d7

We made a few changes to Callbacks to test ops on detached GPU tensors to avoid CPU transfer. However, it made callbacks unpicklable which will crash DDP.

This release fixes that core issue

Changed

  • Allow logging of metrics together with hparams (#1630)
  • Allow metrics logged together with hparams (#1630)

Removed

  • Removed Warning from trainer loop (#1634)

Fixed

  • Fixed ModelCheckpoint not being fixable (#1632)
  • Fixed CPU DDP breaking change and DDP change (#1635)
  • Tested pickling (#1636)

Contributors

@justusschock, @quinor, @williamFalcon