memory leak when using pytorch dataloader

[techkang]

When I use tqdm with pytorch, I found a memory leak.

This code will use more and more memory until break down.

Comment code dummy = tqdm(total=100) in main function or set num_workers=0 will work well.

from torch.utils.data import Dataset, DataLoader
from tqdm import tqdm


class MinDataset(Dataset):

    def __init__(self):
        super().__init__()

    def __getitem__(self, item):
        return '{:.4f}'.format(item) * 1000000

    def __len__(self):
        return 1000`


if __name__ == '__main__':

    min_dataloader = DataLoader(MinDataset(), 32, num_workers=4)
    dummy = tqdm(total=100)
    while True:
        for i, batch in tqdm(enumerate(min_dataloader), total=1000):
            if i == 2:
                break
            pass

I used PyTorch 1.0.1 and tqdm 4.31.1.

[casperdcl]

as mentioned in the documentation, use enumerate(tqdm(x)) instead of tqdm(enumerate(x))

[techkang]

It works well when I use tqdm(enumerate(x)). Thank you.

[guaguablue]

I feel puzzled about the last reply:"It works well when I use tqdm(enumerate(x))", according to another reply before:"use enumerate(tqdm(x)) instead of tqdm(enumerate(x))". I am wondering if Mr.techkang wanted to say: "It works well when I use enumerate(tqdm(x))".
And I read the docunment, it says: "Replace tqdm(enumerate(...)) with enumerate(tqdm(...)) or tqdm(enumerate(x), total=len(x), ...)". However, I met the oppsite situation. When I use tqdm in in my code like below:
for batch_i, (imgs, targets, paths, shapes) in enumerate(tqdm(dataloader, desc='Computing mAP')):
Some times it will make my program stuck in this step, and if I use tqdm(enumerate(dataloader), total=len(dataloader), ...) instead, the program will not work any more. I also tried to use tqdm(enumerate(dataloader)) the computing is ok but the tqdm not works well that I can not see the progress bar like: ‘Computing mAP: 5it [00:05, 1.13s/it]‘.
pytorch version:1.1.0, tqdm:4.43.0
So could some help me about that? Thanks a lot!

[casperdcl]

Also I should mention tqdm.contrib.tenumerate