Following a 303 after a PUT requests uses PUT instead of GET

[PofMagicfingers]

I know you had issues opened with this kind of title, and people were using -X PUT to force PUT, but not this time.

I did this

touch somefile.txt
curl -T 'somefile.txt' -v -L https://httpstat.us/301 > /dev/null

( as adviced here : #578 (comment) )

I expected the following

curl should send somefile.txt with a PUT request, then follow the 303 redirect with a GET request. Instead it uses the PUT verb.

Issue does not occur when server returns a 301 or 302 redirect.

If it isn't the right way to send a put request and follow a 303 response, I'm sorry to have bothered you, and would be glad to know how to do that ! 🙂

Output of the command

(some tls and progress bar logs have been redacted for readability)

*   Trying 23.99.0.12:443...
* Connected to httpstat.us (23.99.0.12) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
> PUT /301 HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.69.1
> Accept: */*
> Content-Length: 0
>
< HTTP/1.1 301 Moved Permanently
< Cache-Control: private
< Content-Length: 21
< Content-Type: text/plain; charset=utf-8
< Location: https://httpstat.us
< Server: Microsoft-IIS/10.0
< X-AspNetMvc-Version: 5.1
< Access-Control-Allow-Origin: *
< X-AspNet-Version: 4.0.30319
< Request-Context: appId=cid-v1:7585021b-2db7-4da6-abff-2cf23005f0a9
< Access-Control-Expose-Headers: Request-Context
< X-Powered-By: ASP.NET
< Set-Cookie: ARRAffinity=dc9bb7185987c0d65790987c33534f3a4e8956d9544852ccf290ce45876d7754;Path=/;HttpOnly;Domain=httpstat.us
< Date: Tue, 14 Apr 2020 17:16:50 GMT
<
* Ignoring the response-body
* Connection #0 to host httpstat.us left intact
* Issue another request to this URL: 'https://httpstat.us/'
* Found bundle for host httpstat.us: 0x55e7f74485c0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host httpstat.us
* Connected to httpstat.us (23.99.0.12) port 443 (#0)
> PUT / HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.69.1
> Accept: */*
> Content-Length: 0
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Length: 8553
< Content-Type: text/html; charset=utf-8
< Server: Microsoft-IIS/10.0
< X-AspNetMvc-Version: 5.1
< Access-Control-Allow-Origin: *
< X-AspNet-Version: 4.0.30319
< Request-Context: appId=cid-v1:7585021b-2db7-4da6-abff-2cf23005f0a9
< Access-Control-Expose-Headers: Request-Context
< X-Powered-By: ASP.NET
< Set-Cookie: ARRAffinity=dc9bb7185987c0d65790987c33534f3a4e8956d9544852ccf290ce45876d7754;Path=/;HttpOnly;Domain=httpstat.us
< Date: Tue, 14 Apr 2020 17:16:50 GMT
<
* Connection #0 to host httpstat.us left intact

curl/libcurl version

curl 7.69.1 (x86_64-pc-linux-gnu) libcurl/7.69.1 OpenSSL/1.1.1f zlib/1.2.11 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh2/1.9.0 nghttp2/1.40.0
Release-Date: 2020-03-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

operating system

Linux arceus 5.4.30-1-MANJARO #1 SMP PREEMPT Thu Apr 2 17:31:45 UTC 2020 x86_64 GNU/Linux

[jay]

curl should send somefile.txt with a PUT request, then follow the 303 redirect with a GET request. Instead it uses the PUT verb.

I'm pretty sure that applies only to POST, @jzakrzewski's comment seems wrong. libcurl has CURLOPT_POSTREDIR but there is no similar option for redirection on PUT.

edit: See 8b7fff3, #4859 (comment)

[jzakrzewski]

Yes, I don't see any reference to changing the method to GET on PUT redirects. So I stand corrected on that one.
However what about this wording:

If the 301 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

For me it means the redirects must work the same for POST and PUT.

[PofMagicfingers]

According to most resources and documentation, a user agent following a 303 redirect must use the HEAD or GET verb :

Wikipedia:

 If a server responds to a POST or other non-idempotent request with a 303 See Other response and a value for the location header, the client is expected to obtain the resource mentioned in the location header using the GET method; to trigger a request to the target resource using the same method, the server is expected to provide a 307 Temporary Redirect response

https://en.m.wikipedia.org/wiki/HTTP_303

MDN says :

The method used to display this redirected page is always GET

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/303

And of the official specifications states :

A user agent can perform a retrieval request targeting that URI (a GET or HEAD request if using HTTP), which might also be redirected, and present the eventual result as an answer to the original request

https://tools.ietf.org/html/rfc7231#section-6.4.4

(sorry for the short reply and lack of formating, I'm on mobile)

[michael-o]

I don't see anything in rfc7231#section-6.4.4 which says this applies to anything else, but POST.

[jzakrzewski]

For me it's a bit unclear. Yes, section 6.4 only names POST specifically but there's also:

Automatic redirection needs to done with
care for methods not known to be safe, as defined in Section 4.2.1,
since the user might not wish to redirect an unsafe request.

And in 4.2.1:

Of the request methods defined by this specification, the GET, HEAD,
OPTIONS, and TRACE methods are defined to be safe.

From the other hand in 4.2.2:

Of the request methods
defined by this specification, PUT, DELETE, and safe request methods
are idempotent.

So it should at least be safe to issue PUT after redirection.

[michael-o]

When it doubt, file an issue: https://github.com/httpwg/http-core.

Here is a similiar one, applies to POST only: apache/httpcomponents-client@53e1725

[jzakrzewski]

Oh, I think I've found the key phrase.
In 4.3.4:

If the origin server will not make the requested PUT state change to
the target resource and instead wishes to have it applied to a
different resource, such as when the resource has been moved to a
different URI, then the origin server MUST send an appropriate 3xx
(Redirection) response; the user agent MAY then make its own decision
regarding whether or not to redirect the request.

So I guess curl's behaviour is at least correct according to this specs. Even if a bit surprising for the user.

[PofMagicfingers]

I don't see anything in rfc7231#section-6.4.4 which says this applies to anything else, but POST.

Section 6.4.4 is saying :

A user agent can perform
a retrieval request targeting that URI (a GET or HEAD request if
using HTTP)

And few lines below :

This status code is applicable to any HTTP method

About this :

From the other hand in 4.2.2:

Of the request methods
defined by this specification, PUT, DELETE, and safe request methods
are idempotent.

So it should at least be safe to issue PUT after redirection.

I don't get you point, it is probably safe to issue a PUT request with the same parameters to the same URL, but it doesn't means anything about using the PUT verb to issue a redirect to another URI.

[jzakrzewski]

I don't get you point, it is probably safe to issue a PUT request with the same parameters to the same URL, but it doesn't means anything about using the PUT verb to issue a redirect to another URI.

My point is that PUT is not among safe methods what would imply the method should be changed to GET on redirect but being idempotent not changing the method is at least safe, so it's not a dangerous bug (if a bug at all).

In any case my next comment citing section 4.3.4 seems to make it clear that not changing the method is in fact an allowed behaviour (unless I don't understand / miss something).

[PofMagicfingers]

I really don't see anything about changing the verb or not in 4.3.4, it only states the server MUST send a 3xx redirect if the resource has changed URI and the user agent MAY follow or not that redirection. But nothing is said about verbs.

In other hand, there is a specific 307/308 redirect status code designed to redirect without changing verb in any case.

I'm really confused about where is the ambiguity here as the spec clearly states in 6.4.4 :

A user agent can perform a retrieval request targeting that URI (a GET or HEAD request if using HTTP)

[...]

This status code is applicable to any HTTP method

Maybe I'm getting this wrong but it seems to be clear that the specs are saying : a 303 redirect can be followed by user agent, with a GET or HEAD request.

I will probably open an issue in http-core send an email to their mailing list, to get the official answer

[PofMagicfingers]

I sent an email regarding this issue to their mailing list. I'll let you know if I get any answers

By the way I also stumbled upon the specs of the fetch standard:

https://fetch.spec.whatwg.org/#http-redirect-fetch

It states :

If one of the following is true

• actualResponse’s status is 301 or 302 and request’s method is POST

• actualResponse’s status is 303 and request’s method is not GET or HEAD

then:

• Set request’s method to GET and request’s body to null.

• For each headerName of request-body-header name, delete headerName from request’s header list.

[michael-o]

fetch is for browsers, not sure whether this applies here.

[PofMagicfingers]

I'm pretty sure following redirect in fetch is basic http implementation. I'm not sure while it would differ.

fetch is after all, implementing the http spec as any user agent would.