too many files open

[seguler]

seguler@seguler-sdk-test:~/test/linux_amd64$ ./blobporter -f /mnt/20x1GB/* -c bigdata -n blobporter/
BlobPorter
Copyright (c) Microsoft Corporation.
Version: 0.5.25

2018/02/06 22:59:12 container name not specified
seguler@seguler-sdk-test:~/test/linux_amd64$ ./blobporter -f "/mnt/20x1GB/*" -c bigdata -n blobporter/
BlobPorter
Copyright (c) Microsoft Corporation.
Version: 0.5.25

Files to Transfer (file-blockblob) :
Source: /mnt/20x1GB/xam Size:1073741824
Source: /mnt/20x1GB/xuma Size:1073741824
Source: /mnt/20x1GB/yuop Size:1073741824
Source: /mnt/20x1GB/rat Size:1073741824
Source: /mnt/20x1GB/btk Size:1073741824
Source: /mnt/20x1GB/hat Size:1073741824
Source: /mnt/20x1GB/luf Size:1073741824
Source: /mnt/20x1GB/pat Size:1073741824
Source: /mnt/20x1GB/sup Size:1073741824
Source: /mnt/20x1GB/tuv Size:1073741824
Source: /mnt/20x1GB/abc Size:1073741824
Source: /mnt/20x1GB/jap Size:1073741824
Source: /mnt/20x1GB/oss Size:1073741824
Source: /mnt/20x1GB/ffa Size:1073741824
Source: /mnt/20x1GB/kat Size:1073741824
Source: /mnt/20x1GB/nap Size:1073741824
Source: /mnt/20x1GB/upmr Size:1073741824
Source: /mnt/20x1GB/ups Size:1073741824
Source: /mnt/20x1GB/ztv Size:1073741824
Source: /mnt/20x1GB/cuv Size:1073741824
--> 43 % [||||||||||...............] Committed Count: 0 Buffer Level: 100%Error while opening the file open /mnt/20x1GB/luf: too many open files
2018/02/06 23:00:05 open /mnt/20x1GB/luf: too many open files

[giventocode]

Thanks for reporting this, more details about the environment will helpful - including the number of cores.

[seguler]

20 cores; Ubuntu 16.04.

Here is what happens to the open file descriptors during the run:
seguler@seguler-sdk-test:$ lsof | grep REG | wc -l
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/113/gvfs
Output information may be incomplete.
62
seguler@seguler-sdk-test:$ lsof | grep REG | wc -l
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/113/gvfs
Output information may be incomplete.
25626
seguler@seguler-sdk-test:$ lsof | grep REG | wc -l
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/113/gvfs
Output information may be incomplete.
30415
seguler@seguler-sdk-test:$ lsof | grep REG | wc -l
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/113/gvfs
Output information may be incomplete.
33771
seguler@seguler-sdk-test:$ lsof | grep REG | wc -l
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/113/gvfs
Output information may be incomplete.
62
seguler@seguler-sdk-test:$

[seguler]

35K-40K open file handles.. Obviously it leaks somewhere... The whole list is full of open handles to the files I am trying to upload

blobporte 38254 38303 seguler 789r REG 8,17 1073741824 4194313 /mnt/20x1GB/luf

[giventocode]

Fixed in #87