Skip to content

Bug: ObjectStoreRegistry get_by_uri does not return correct path when "scheme" is provided #2525

@timvw

Description

@timvw

Describe the bug
object_store_registry.get_by_uri does not work as expected.

To Reproduce && Expected behavior

    #[test]
    fn test_get_by_uri_s3() {
        let sut = ObjectStoreRegistry::default();
        sut.register_store("s3".to_string(), Arc::new(LocalFileSystem {}));
        let uri = "s3://bucket/key";
        let (_, path) = sut.get_by_uri(uri).unwrap();
        assert_eq!(path, "bucket/key");
    }

    #[test]
    fn test_get_by_uri_file() {
        let sut = ObjectStoreRegistry::default();
        let uri = "file:///bucket/key";
        let (_, path) = sut.get_by_uri(uri).unwrap();
        assert_eq!(path, "/bucket/key");
    }

    #[test]
    fn test_get_by_uri_local() {
        let sut = ObjectStoreRegistry::default();
        let uri = "/bucket/key";
        let (_, path) = sut.get_by_uri(uri).unwrap();
        assert_eq!(path, "/bucket/key");
    }

Additional context
This issue is not uncovered with local files, because LocalFileSystem does some additional "magic" to workaround the issue:

impl ObjectStore for LocalFileSystem {
    async fn list_file(&self, prefix: &str) -> Result<FileMetaStream> {
        let prefix = if let Some((_scheme, path)) = prefix.split_once("://") {
            path
        } else {
            prefix
        };
        list_all(prefix.to_owned()).await
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions