Skip to content

net/http: 302 redirect of DELETE to GET isn't RFC compliant #41377

@ncw

Description

@ncw

What version of Go are you using (go version)?

$ go version
go version go1.15 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ncw/.cache/go-build"
GOENV="/home/ncw/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ncw/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ncw/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/go1.15"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/go1.15/pkg/tool/linux_amd64"
GCCGO="/usr/bin/gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build061961387=/tmp/go-build -gno-record-gcc-switches"

What did you do?

We used http.Client against a Microsoft Ondrive service.

When we issued a DELETE response the server responded with a 302 redirect and http.Client retried with a GET request.

We issue a DELETE request

2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : DELETE /v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: graph.microsoft.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Accept-Encoding: gzip

The server responds with a 302 redirect

2020/09/12 22:44:36 DEBUG : HTTP RESPONSE (req 0xc00066b300)
2020/09/12 22:44:36 DEBUG : HTTP/1.1 302 Found
Cache-Control: private
Client-Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Content-Type: text/plain
Date: Sun, 13 Sep 2020 02:44:35 GMT
Location: https://redacted-my.sharepoint.com/_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Request-Id: 9f245298-ad4b-4c78-b6c3-478208784d04
Strict-Transport-Security: max-age=31536000
X-Ms-Ags-Diagnostic: {"ServerInfo":{"DataCenter":"North Central US","Slice":"SliceC","Ring":"3","ScaleUnit":"003","RoleInstance":"AGSFE_IN_30"}}
Content-Length: 0

We then do a GET request on the redirected URL which doesn't work.

2020/09/12 22:44:36 DEBUG : HTTP REQUEST (req 0xc0004be400)
2020/09/12 22:44:36 DEBUG : GET /_vti_bin/client.svc/v2.0/drives('b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff')/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK HTTP/1.1
Host: redacted-my.sharepoint.com
User-Agent: rclone/v1.53.0
Authorization: XXXX
Referer: https://graph.microsoft.com/v1.0/drives/b!-Jr3ZPe__kCZ7xLdl2FMoffWFSmIUF1AoWygTwxjtywWlD7heAFZQpK_8QbDA7ff/items/01HSWNV2LEGZAH5A2BLJE3R3I4ACF3QDRK
Accept-Encoding: gzip

What did you expect to see?

I expected http.Client to redirect with a DELETE request.

According to RFC7231

6.4.3. 302 Found

The 302 (Found) status code indicates that the target resource
resides temporarily under a different URI. Since the redirection
might be altered on occasion, the client ought to continue to use the
effective request URI for future requests.

The server SHOULD generate a Location header field in the response
containing a URI reference for the different URI. The user agent MAY
use the Location field value for automatic redirection. The server's
response payload usually contains a short hypertext note with a
hyperlink to the different URI(s).

  Note: For historical reasons, a user agent MAY change the request
  method from POST to GET for the subsequent request.  If this
  behavior is undesired, the 307 (Temporary Redirect) status code
  can be used instead.

I note that this only talks about POST to GET and not other methods in the historical reasons section.

So I'm not sure Go's implementation is RFC compliant here - the RFC doesn't say that user agents may change any other sort of method from POST to GET.

The code is here:

go/src/net/http/client.go

Lines 496 to 512 in 66e66e7

// redirectBehavior describes what should happen when the
// client encounters a 3xx status code from the server
func redirectBehavior(reqMethod string, resp *Response, ireq *Request) (redirectMethod string, shouldRedirect, includeBody bool) {
switch resp.StatusCode {
case 301, 302, 303:
redirectMethod = reqMethod
shouldRedirect = true
includeBody = false
// RFC 2616 allowed automatic redirection only with GET and
// HEAD requests. RFC 7231 lifts this restriction, but we still
// restrict other methods to GET to maintain compatibility.
// See Issue 18570.
if reqMethod != "GET" && reqMethod != "HEAD" {
redirectMethod = "GET"
}
case 307, 308:

Which says for all methods other than GET or HEAD, change them to GET, whereas the RFC says you can change POST to GET only.

So I think backwards compatibility might demand that POST becomes GET so something like this would be more RFC compliant.

		if reqMethod == "POST" {
			redirectMethod = "GET"
		}

The current code seems to think the major distinction between 301/302 and 307/308 is whether the body is resent. I'm not sure the RFC supports that, but using that distinction I think the code code could look like

		if ireq.outgoingLength() != 0 {
			redirectMethod = "GET"
		}

Reading the RFC it says nothing about request bodies vs headers. So I think to be most useful and most RFC compliant the 301,302 redirects should work just like the 307,308 redirects now we have the capability to resend the body if needed.

See: #18570
See: https://forum.rclone.org/t/purge-command-fails-on-onedrive/19048

Metadata

Metadata

Assignees

No one assigned

    Labels

    NeedsDecisionFeedback is required from experts, contributors, and/or the community before a change can be made.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions