Permalink
Branch: master
Commits on Feb 5, 2019
  1. Better FP16 support in pytorch fp16 utils.

    jma127 committed Feb 1, 2019
    This commit adds an FP16Model class as a successor to network_to_half.
    
    The benefits of this class are:
    
    - Preservation of single-precision for BatchNorm layers. The models
      generated by network_to_half() convert BatchNorm moment tensors to
      half-precision, then back to single-precision, which hurts the
      accuracy of the moment estimators and occasionally results in NaNs.
    - Support for multi-argument nn.Modules (self-explanatory from code).