Prevent hard-coded Content-Type for --post-data

[ildar-shaimordanov]

With the --post-data option the header Content-Type: application/x-www-form-urlencoded is hard-coded and there is no way to specify a custom Content-Type header.

I think the code should be modified to turn off the hard-coded header when the end user specifies its own (almost in the same way as it has been done for other headers in wget_user_headers[]).

Update 1: I guess this PR should be submitted to upstream first. But it's a bit faster to report it to you and you forward it further.
Update 2: This PR completely is agreed with my previous PR #232.

[rmyorston]

Upstream BusyBox wget, GNU wget and the recently released GNU wget2 all hard-code the content type for POST data. The wget documentation explicitly states:

Wget does not currently support "multipart/form-data" for transmitting POST data; only "application/x-www-form-urlencoded".

Please take this upstream.

[ildar-shaimordanov]

Yes. I know that all of them hard coded the content-type for POST data. However they are flexible and allow to overwrite the hard-coded header by the user-defined one. I've got dove into the wget/wget2 sources. They differ significantly, but the logic is the same: they replace the default header if the end-user defined its own header.

wget/cygwin

Below are examples of real usecases with wget/cygwin.

wget/cygwin failed without other headers:

$ wget -qO - https://api.github.com/markdown/raw --post-data "example text"
$ echo $?
8
$ wget -O - https://api.github.com/markdown/raw --post-data "example text"
--2021-10-07 18:38:45--  https://api.github.com/markdown/raw
Resolving api.github.com (api.github.com)... 140.82.114.6
Connecting to api.github.com (api.github.com)|140.82.114.6|:443... connected.
HTTP request sent, awaiting response... 415 Unsupported Media Type
2021-10-07 18:38:45 ERROR 415: Unsupported Media Type.

wget/cygwin success with a user-defined header:

$ wget -qO - https://api.github.com/markdown/raw --post-data "example text" --header "Content-Type: text/plain"
<p>example text</p>

wget/busybox fails in both cases.

wget/linux

wget/linux acts the same way but recently I don't have any linux host opened to the Internet to show the same behavior. I can only emulate by sending requests and capturing request headers.

no custom Content-Type (wget sends the original header for POST data):

$ wget --post-data "to be or not to be" -S -qO - localhost:8081

$ nc -l -p 8081
POST / HTTP/1.1
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Host: localhost:8081
Connection: Keep-Alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 18

to be or not to be^C

with custom Content-Type (the default content-type for POST data was overwritten):

$ wget --post-data "to be or not to be" -S -qO - localhost:8081 --header "Content-Type: text/plain"

$ nc -l -p 8081
POST / HTTP/1.1
User-Agent: Wget/1.14 (linux-gnu)
Accept: */*
Host: localhost:8081
Connection: Keep-Alive
Content-Type: text/plain
Content-Length: 18

to be or not to be^C

wget/busybox

The applet acts differently. It sends both the user-defined and hard-coded headers. There is no superseding of the default header with the user-defined one:

>busybox wget --post-data "to be or not to be" localhost:8081 --header "Content-Type: text/plain"
Connecting to localhost:8081 (127.0.0.1:8081)
wget: error getting response

>busybox nc -l -p 8081
POST / HTTP/1.1
Host: localhost:8081
User-Agent: Wget
Connection: close
Content-Type: text/plain
Content-Type: application/x-www-form-urlencoded
Content-Length: 18

to be or not to be^C

[ildar-shaimordanov]

Just a humble reminder that the upstream issued the new version (unstable for now) with the suggested patch. The third commit is for the changes resolving the merge conflict and for applying the changes corresponding the upstream.

[rmyorston]

The latest release of busybox-w32, FRP-4621-gf3c5e8bc3, pulls in the fix for this issue from upstream.