Path-style S3 access is deprecated

[DaveCTurner]

It was recently announced that path-style access to S3 is deprecated will be rejected by S3 starting 2020-09-30. However, since #34721 (i.e. 7.0), the repository-s3 plugin exclusively uses path-style access, to make it easier to use Minio-backed repositories. This needs a re-think.

[elasticmachine]

Pinging @elastic/es-distributed

[messense]

Hitting this issue with Alibaba Cloud OSS service (S3 like service), it only supports virtual host style access.

[original-brownbear]

IMO, as nice as it was to get rid of this config option, we should simply add back the setting to choose between path and DNS style access.
I don't think we can determine the correct access pattern on the fly in a stable way and it's not that hard to make the correct choice of setting as a user with a little documentation on the compatibility of this anyway.

[messense]

Adding the config option back would be great. Thanks!

[original-brownbear]

We discussed this during today's team sync. The fact that AWS will stop supporting the path style access pattern makes the solution build in #34721 not viable long term.

I will open a PR (to go into 7.2+) shortly that adds back the option to choose path style access and default it to false and add documentation of this breaking change accordingly.