ntlmssp.Negotiator is storing the entire body in memory and causing out of memory errors

[ncw]

I had a bug report in rclone which was complaining about huge memory use.

I tracked it down from the log to this

github.com/Azure/go-ntlmssp.Negotiator.RoundTrip({{0x27ff220?, 0xc0006ce230?}}, 0xc0009d6000)
	github.com/Azure/go-ntlmssp@v0.0.0-20221128193559-754e69321358/negotiator.go:52 +0x2bd fp=0xc0007aac68 sp=0xc0007aa990 pc=0x191645d
github.com/Azure/go-ntlmssp.(*Negotiator).RoundTrip(0xc0007aad48?, 0x418f48?)
	<autogenerated>:1 +0x31 fp=0xc0007aac90 sp=0xc0007aac68 pc=0x1918a31

Which points to this bit of code

go-ntlmssp/negotiator.go

Lines 49 to 59 in 754e693

	// Save request body
	body := bytes.Buffer{}
	if req.Body != nil {
	_, err = body.ReadFrom(req.Body)
	if err != nil {
	return nil, err
	}
	req.Body.Close()
	req.Body = ioutil.NopCloser(bytes.NewReader(body.Bytes()))
	}

Where we see, yes this does load the entire body into memory.

The user is complaining about out of memory errors when trying to upload a very large file (10GB).

This might be an error we are making in the rclone code - I didn't write this bit of code so maybe we are using the library wrong, but loading a 10GB body into memory is a bit unfriendly!

Here is the rclone code if anyone wants to take a look

https://github.com/rclone/rclone/blob/1ebbc74f1d62457aea660b9e4378cc085c0dd25e/backend/webdav/webdav.go#L462