Don't set user-agent to Go-http-client/1.1

[dsseng]

What does this PR do?

Force User-Agent header to "" if it's not set to avoid it being replaced with Go-http-client/1.1".

https://github.com/golang/go/blob/7d30af8e17d62932f8a458ad96f483b9afec6171/src/net/http/request.go#L614-L619

Motivation

Fixes #5621, but only for v2.1.

More

• Added/updated tests
• Added/updated documentation

Additional Notes

Probably we should add a backport for this to v1.7, since the issue was reported to occur on that version. If the backport is needed, I can create it.

[mpl]

@sh7dm According to the code you pointed at https://github.com/golang/go/blob/7d30af8e17d62932f8a458ad96f483b9afec6171/src/net/http/request.go#L614-L619
the intent is apparently to set a default only if the header does not exist. i.e. if it exists but it is blank (""), then the stdlib does not touch it.
To that extent, I'm wondering if curl is not the one at fault here, and that maybe it should voluntarily set the header to an empty string when -A "" is used.
WDYT?

[dsseng]

If I have time I'll try wget instead of curl

[mpl]

@sh7dm Nevermind, I think you are right. With respect to the client, Traefik is a server, and it should not be blindly using the default behaviour that is set when creating a client (the code you pointed out).
And indeed, what you're proposing is what the director of the stdlib reverseproxy does too: https://github.com/golang/go/blob/master/src/net/http/httputil/reverseproxy.go#L119

[juliens]

LGTM

[ldez]

Thanks 👍