http: improve AWS HTTP v4 Signature

[dwagin]

• Add support services without region and service prefixes in
  the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc)
  by providing region and service parameters via aws-sigv4 option.
• Add [:region[:service]] suffix to aws-sigv4 option;
• Refactor memory management.
• Use Curl_http_method() instead STRING_CUSTOMREQUEST.
• Refactor canonical headers generating.
• Remove repeated sha256_to_hex() usage.
• Add some docs fixes.
• Add some codestyle fixes.
• Add providing credentials via url.
• Update tests.

[bagder]

I realize you didn't actually add this, but the code must not assign data->set.headers. That's the set of custom headers passed to libcurl from the application and it should not be interfered with by internals. It will make repeated transfers and handle duplications misbehave.

[dwagin]

I'll try to fix it tomorrow.

[outscale-mgo]

if(!curl_slist_append(data->set.headers, timestamp)) {
  ret = CURLE_FAILED_INIT;
  goto fail;
}
if(!curl_slist_append(data->set.headers, date_header)) {
  ret = CURLE_FAILED_INIT;
  goto fail;
}

would this work ?
Note: if I ask you to split your commit, it's partially because this would enable @bagder to merge bug fix rapidly (like this one) and give us more time to review other improvement .
Edit: dwagin fix seems to be the good thing to do, and work on my machine

[dwagin]

I realize you didn't actually add this, but the code must not assign data->set.headers. That's the set of custom headers passed to libcurl from the application and it should not be interfered with by internals. It will make repeated transfers and handle duplications misbehave.

@bagder, see my fix

[bagder]

@outscale-mgo, you might want to give this a review/comment too

[outscale-mgo]

I would prefer if you split "http: improve AWS HTTP v4 Signature" commit
by making one commit for each "improvements"
Also I find it a little odd, to modify test1933 in one commit and modify the data in another one.

[outscale-mgo]

Overall I like the changes, thanks for improving this code :)

[dwagin]

@bagder, strndup() implementation turned out to be a big deal :).
I can finish it as part of this pull request or roll back these changes and rewrite without stndup().
Please tell me how best to do it?

[bagder]

An additional request: please also fix the names in the four tests! 1933 1934 1935 1936 all use the same non-telling names. They should better identify what makes them unique. Compare with other tests.

[bagder]

@bagder, strdup() implementation turned out to be a big deal :).

You mean strndup?

I can finish it as part of this pull request or roll back these changes and rewrite without stndup().

I don't know what you want from me on this. Maybe you can survive with using Curl_memdup() instead?

[dwagin]

@bagder, strdup() implementation turned out to be a big deal :).

You mean strndup?

Yes.

I can finish it as part of this pull request or roll back these changes and rewrite without stndup().

I don't know what you want from me on this. Maybe you can survive with using Curl_memdup() instead?

I did everything without strndup()

[outscale-mgo]

Once that fix, I don't think I will have any more comment.
But please note that having only one patch that contain all your change, make it really hard to read, as we have only one big diff, understand what change between to the old version and your version take extra effort.
Especially as you move some code (and it was the right move), but because of that the diff really don't understand what's going on and give you unrelated visual.

[bagder]

This looks good to me. I'll give a little time for everyone else to also comment before I merge. I would like to have this done asap so that this is ready for the release on Wednesday, when this feature is first exposed to the world!

[dwagin]

After this refactoring, this authentication method became fully usable. Thank you all for your help! Hello, world )

[bagder]

Thanks!