Allow disabling virtual-hosted-style addressing

[DamienMatias]

Tell us more about this new feature.

Hello,

I was wondering if there was a way to force the usage of path style requests instead of the default virtual-hosted-style addressing ?
This is actually possible with mountpoint-s3 as you can see here in their documentation.

To disable virtual-hosted-style addressing, use the --force-path-style command-line flag to instead send requests to https://example.com/docexamplebucket/.

This would allow us using the LakeFS S3 Gateway and potentially other usages that leverage tools still built around the path style addressing.

Thank you 🙏

[fuatbasik]

Hello, @DamienMatias !
Thank you for your interest in Amazon S3 Connector for PyTorch. We'll discuss this internally report back here when we have something concrete to share.

[N-o-Z]

@DamienMatias Hi - lakeFS maintainer here 👋🏽
As a WA for this issue you can configure LAKEFS_GATEWAYS_S3_DOMAIN_NAME to enable lakeFS to work with a virtual host

[devel4848]

I'am also interested in disabling virtual-hosted-style addressing. I would like to use S3 Connector for PyTorch with a S3-compatible Ceph storage that can not be configured for virtual-hosted-style addressing because of the DNS implication. Only path-style addressing can be used in our context.

[jamesbornholt]

The S3 client we use supports disabling virtual-hosted-style addressing here, so I think this would just be a matter of plumbing through a new flag from the various constructors (S3IterableDataset.from_prefix and friends) to the Rust constructor here, similar to #195. I'm not sure we're going to get to this in the short term, but we'd happily review a PR!