lib: add CURLFOLLOW_OBEYCODE and *FIRSTONLY

[bagder]

With this change, the argument passed to the CURLOPT_FOLLOWLOCATION option is treated as a mode option instead of a boolean.

CURLFOLLOW_ALL

This is the old value and makes redirects with custom methods work exactly like before: it changes the method for all requests, independently of what response codes are returned.

CURLFOLLOW_OBEYCODE

libcurl obeys the HTTP server instruction in the response code: for 301, 302, 303 it changes to GET. For 307 and 308 it keeps the custom method even in the redirect.

CURLFOLLOW_FIRSTONLY

If this is set, the method is stored to either GET or POST/PUT on the first redirect. The custom method is only ever used in the first outgoing request.

---

This change is forward compatible because CURLOPT_FOLLOWLOCATION has been documented to accept the exact value of '1' to enable redirect following and therefore all other bits were left unused and undefined. We now add two other non-zero values. Starting in 8.13.0, the values 2 and 3 add more meaning.

This change is backward compatible in the following way: setting the modes in a program that still uses an older libcurl installation at runtime will have no effect. This is because older libcurl code checked if the value was non-zero and then enabled redirect following. Of course older libcurl will always let the set CURLOPT_CUSTOMMETHOD string override the method, disregarding what the HTTP response code suggests.

libtest 1571 to to 1581 verify

[bagder]

#16487 and #16482 should ideally fix them. I will rebase on top of master once they both have been merged.

[vszakats]

Confirmed this also affects gcc-13 and gcc-14, thus gcc-specific in general (and not gcc-12 in particular):
https://github.com/curl/curl/actions/runs/13534146044/job/37822591439?pr=16492

[bagder]

There's perhaps one question to decide:

If we replace GET as in curl -X MOO -L localhost and that 301-redirects to a new URL.

With *OBEYCODE set, should it change the follow-up method to GET or not? The current code does not, since the user has replaced the method for GET.

[bagder]

Maybe we need another bit for ONLY-FOR-FIRST-REQUEST that then resets back unconditionally on the subsequent... Maybe that's the behavior people actually want?

[jay]

Maybe we need another bit for ONLY-FOR-FIRST-REQUEST that then resets back unconditionally on the subsequent... Maybe that's the behavior people actually want?

I think there are websites that generate curl code and use -X POST and -X GET so there's some code out there using that even though it's wrong. What if there was just one bit CURLFOLLOW_IGNORE_CUSTOMREQ where it would not pass on the custom request in any redirects no matter what? Is there need for a more refined behavior like what you are doing?

(actually I see now you have added CURLFOLLOW_FIRSTONLY which is same? however I think the name FIRSTONLY implies like that it's only going to follow the first redirect and that may be harder to understand for someone just reading the code without a lot of libcurl knowledge)

edit: CURLFOLLOW_IGNORE_CUSTOMREQ looks like a flag not a separate enum so maybe when seen by itself would be hard to understand as well hm

[bagder]

What if there was just one bit CURLFOLLOW_IGNORE_CUSTOMREQ where it would not pass on the custom request in any redirects no matter what?

(actually I see now you have added CURLFOLLOW_FIRSTONLY which is same? however I think the name FIRSTONLY implies like that it's only going to follow the first redirect and that may be harder to understand for someone just reading the code without a lot of libcurl knowledge)

Names are hard, but I think your name could easily be read to imply the other way around: that it disables the custom method completely. I think it is hard for us to pick a name that will explain it clearly without the user having to read the documentation for it.

I wouldn't mind other names if we come up with better!

[bagder]

Any further remarks, ideas, complaints or name improvements?