File URI Scheme Interpretation

[tustvold]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

The file URI scheme implemented in ObjectStoreRegistry does not appear to follow the specification described in RFC 3986.

In particular:

• It does not handle the host component
• It accepts non-absolute paths

Describe the solution you'd like

The file:// URIs should be handled in a spec-compliant way. Unfortunately this on its own would prevent the use of local paths, which would likely be annoying for users.

I would therefore propose we special case URIs without a scheme, and canonicalise them within ObjectStoreRegistry.

Describe alternatives you've considered

We could not do this

Additional context

I encountered this whilst working on #2489, as the object_store crate purposefully does not handle relative paths.

Thoughts @yahoNanJing @thinkharderdev

[thinkharderdev]

Makes sense. This also came up in apache/datafusion-ballista#481 where we had a regression when we stripped the schema from file URIs. We actually use the scheme to resolve the correct ObjectStore when deserializing plans in Ballista.

[alamb]

cc @andygrove and @mingmwang