RuntimeWarning: overflow encountered in long_scalars

[ngcthuong]

Hello,
Does anyone encounter this issue? A negative flops.

C:\Anaconda3\envs\pytorch\lib\site-packages\ptflops\flops_counter.py:285: RuntimeWarning: overflow encountered in long_scalars
overall_conv_flops = conv_per_position_flops * active_elements_count
- Flops: -27913.09 MMac
- Params: 556.1 k

The tested network includes Convolution, Batchnorm, and Relu only.
Thank you

[sovrasov]

Could you share the network's code?

[ngcthuong]

I use DnCNN pytorch version from https://github.com/cszn/DnCNN/tree/master/TrainingCodes/dncnn_pytorch

[ngcthuong]

The following is a customized code

import argparse
import re
import os, glob, datetime, time
import numpy as np
import torch
import torch.nn as nn
from torch.nn.modules.loss import _Loss
import torch.nn.init as init
from torch.utils.data import DataLoader
import torch.optim as optim
from torch.optim.lr_scheduler import MultiStepLR
import data_generator as dg
from data_generator import DenoisingDataset
import torch.nn.functional as F
from ptflops import get_model_complexity_info


# Params
parser = argparse.ArgumentParser(description='PyTorch DnCNN')
parser.add_argument('--model', default='DnCNN', type=str, help='choose a type of model')
parser.add_argument('--batch_size', default=128, type=int, help='batch size')
parser.add_argument('--train_data', default='data/Train400', type=str, help='path of train data')
parser.add_argument('--sigma', default=25, type=int, help='noise level')
parser.add_argument('--epoch', default=180, type=int, help='number of train epoches')
parser.add_argument('--lr', default=1e-3, type=float, help='initial learning rate for Adam')
parser.add_argument('--depth', default = 17, type=int, help='number of convolutions for DnCNN')
parser.add_argument('--n_channels', default = 64, type=int, help='number of intermediate channels for DnCNN')

args = parser.parse_args()

batch_size = args.batch_size
cuda = torch.cuda.is_available()
n_epoch = args.epoch
sigma = args.sigma

save_dir = os.path.join('models', args.model+'_' + 'sigma' + str(sigma))

if not os.path.exists(save_dir):
    os.mkdir(save_dir)

class DnCNN(nn.Module):
    def __init__(self, depth=17, n_channels=64, image_channels=1, use_bnorm=True, kernel_size=3):
        super(DnCNN, self).__init__()
        kernel_size = 3
        padding = 1
        layers = []
    
        layers.append(nn.Conv2d(in_channels=image_channels, out_channels=n_channels, kernel_size=kernel_size, padding=padding, bias=True))
        layers.append(nn.ReLU(inplace=True))
        for _ in range(depth-2):
            layers.append(nn.Conv2d(in_channels=n_channels, out_channels=n_channels, kernel_size=kernel_size, padding=padding, bias=False))
            layers.append(nn.BatchNorm2d(n_channels, eps=0.0001, momentum = 0.95))
            layers.append(nn.ReLU(inplace=True))

        layers.append(nn.Conv2d(in_channels=n_channels, out_channels=image_channels, kernel_size=kernel_size, padding=padding, bias=False))
        self.dncnn = nn.Sequential(*layers)
        self._initialize_weights()

    def forward(self, x):
        y = x
        out = self.dncnn(x)
        return y-out

    def _initialize_weights(self):
        for m in self.modules():
            if isinstance(m, nn.Conv2d):
                init.orthogonal_(m.weight)
                print('init weight')
                if m.bias is not None:
                    init.constant_(m.bias, 0)
            elif isinstance(m, nn.BatchNorm2d):
                init.constant_(m.weight, 1)
                init.constant_(m.bias, 0)


if __name__ == '__main__':
    # model selection
    print('===> Building model')
    model = DnCNN()
    
    model = model.cpu() 
    #model.
    with torch.cuda.device(0):
        flops, params = get_model_complexity_info(model, (1, 256, 256), as_strings=True, print_per_layer_stat=False)
        print('      - Flops:  ' + flops)
        print('      - Params: ' + params)

[ngcthuong]

I guess the number is too big beyond the default type of python.

[sovrasov]

It's strange because that means really huge amount of computations in such a small network.
I've launched your code with the latest version of ptflops (0.3):

===> Building model
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
init weight
      - Flops:  36.51 GMac
      - Params: 556.1 k

May be the problem is in strange behavior of python in Windows. Do you use python2? I use python3.6 in ubuntu 16.04.

[sovrasov]

Fixed in #43