Limit memory usage on PutObject [faq]

[pizixi]

What should I do if I use a lot of memory when uploading large files?

[klauspost]

@pizixi Limit PartSize or disable multipart altogether.

[harshavardhana]

@pizixi Limit PartSize or disable multipart altogether.

Or provide the correct content length for SDK to automatically calculate the optimal part size.

[pizixi]

hi：
I used your code to test @klauspost (https://play.golang.org/p/jGxIrOFE97U) and kept uploading a file of more than 300 megabytes, The memory usage increased to about 1.3G and there was no increase, there should be no memory leak, but 1.3G memory is too big for me, I tried to test with python client api, The memory usage has been stable at 33m, which is the ideal way. Is there any good way to solve this problem of minio-go? Thank you

[harshavardhana]

	go func() {
		defer wg.Done()
		uinfo, uerr = mClient.PutObject(context.TODO(),
			"test",
			fmt.Sprintf("%s-%c", fileName, time.Now().Unix()),
			pReader,
			-1,
			minio.PutObjectOptions{ContentType: "application/octet-stream"})
		log.Printf("err %v %v", uinfo, uerr)
	}()

I used your code to test @klauspost (https://play.golang.org/p/jGxIrOFE97U) and kept uploading

If you are passing -1 as length SDK has to allocate 512MiB buffer to cater for the largest file possible to upload i.e 5TiB. So don't pass -1 as content-length - instead, specify the correct length. Such as 300MiB length in bytes.