fput_object on large file creates 3 unterminated processes

[U039b]

Summary

Uploading large file creates 3 unterminated Unix processes (in green on the following screenshot):

Those processes are destroyed when their parent dies. I use Minio in Celery task and the Celery worker is always running. Having those unterminated processes causes a huge memory leak (approximately the size of the uploaded file).

System:

• LSB Release

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=17.04
DISTRIB_CODENAME=zesty
DISTRIB_DESCRIPTION="Ubuntu 17.04"

• Kernel

Linux [edited] 4.10.0-42-generic #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

• Python

Python 3.5.3
minio == 3.0.0.

To reproduce

Put a large file named payload1.apk in the same directory as the following script.

from minio import Minio
from minio.error import ResponseError
import time

client = Minio('localhost:9000',
               access_key='mykey',
               secret_key='mykey',
               secure = False,
               )
try:
    client.make_bucket('my-bucketname')
except Exception as err:
    print(err)
# Put an object 'my-objectname' with contents from 'my-filepath'
try:
    client.fput_object('my-bucketname', 'payload1.apk', 'payload1.apk')
except ResponseError as err:
    print(err)

time.sleep(200)

Execute this script and check created processes. You will see 3 extra processes.

[harshavardhana]

This bug can be closed as duplicate of #595

[nitisht]

@U039b closing this as duplicate, please follow #595 for progress