Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easier to register object stores #10616

Closed
alamb opened this issue May 22, 2024 · 2 comments · Fixed by #10617
Closed

Make it easier to register object stores #10616

alamb opened this issue May 22, 2024 · 2 comments · Fixed by #10617
Labels
help wanted Extra attention is needed

Comments

@alamb
Copy link
Contributor

alamb commented May 22, 2024

Came up in #9280

The problem is that when trying to use urls in ListingTables (or ParquetExec), if you try with a scheme like file:// it won't work unless you have registered an object store handler

However, the current user experience is bad -- the error message is mysterious and confusing and it is not obvious how to register the correct listing table

"Error: Internal("No suitable object store found for file://")"

I would recommend two things:

Suggestion 1: Change the error message

Change the error message to include instructions on how to fix the issue

For example, instead of

No suitable object store found for file://

Perhaps it could be something more like

No object store registered for url scheme 'file://'. See RuntimeEnv::register_object_store

Suggestion 2: Add an example to the docs

Once the error message directs to RuntimeEnv::register_object_store then it would be ideal if the docs

https://docs.rs/datafusion/latest/datafusion/execution/runtime_env/struct.RuntimeEnv.html#method.register_object_store

actually had an example

So that would perhaps take the form of adapting the example above as a doc example

Does that make sense?

Originally posted by @alamb in #9280 (comment)

@alamb alamb added the help wanted Extra attention is needed label May 22, 2024
@alamb alamb changed the title Improve docs and examples for register_object_store Make it easier to register object stores May 22, 2024
@aditanase
Copy link
Contributor

@alamb sent a quick PR - let me know if you were thinking of more than this. Thanks for the pointers!

@alamb
Copy link
Contributor Author

alamb commented May 22, 2024

Thanks again @aditanase 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
2 participants