DataLoaders 6: first-class support for Incompatible
#4565
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
teh-cmc
added
🧑💻 dev experience
This was referenced Dec 17, 2023
teh-cmc
added
do-not-merge
emilk
approved these changes
Dec 19, 2023
|
Oh also: use |
teh-cmc
changed the title
DataLoaders 6: first-class support for NotSupportedDataLoaders 6: first-class support for Incompatible
teh-cmc
added a commit
that referenced
this pull request
Dec 19, 2023
Add support for registering custom loaders, the same way we offer support for registering custom space views and store subscribers. Checks: - [x] cargo r -p custom_data_loader -- Cargo.toml --- Part of a series of PRs to make it possible to load _any_ file from the local filesystem, by any means, on web and native: - #4516 - #4517 - #4518 - #4519 - #4520 - #4521 - #4565 - #4566 - #4567
teh-cmc
added a commit
that referenced
this pull request
Dec 22, 2023
Some docs for the whole thing. It heavily relies on linking to the examples for exact details and to avoid rot. It's probably extremely bad. Help welcome. --- Part of a series of PRs to make it possible to load _any_ file from the local filesystem, by any means, on web and native: - #4516 - #4517 - #4518 - #4519 - #4520 - #4521 - #4565 - #4566 - #4567 Co-authored-by: Nikolaus West <niko@rerun.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduce a first-class protocol for
DataLoaders -- whether they are builtin, custom, or external -- to announce that they don't support loading a given piece of data.This fixes one of the oldest issue when loading files in Rerun: loading unsupported data would always bottom out in either the image or websocket paths, leading to unrelated errors very confusing to users.
The loading process is now a two-pass process:
1.1. If at least one compatible loader is found, go to 2
1.2. Otherwise, fail early with a nice error message for the end user
This has important/subtle ramifications on the threading model, but other than that is what you'd expect.
Checks:
Part of a series of PRs to make it possible to load any file from the local filesystem, by any means, on web and native:
DataLoaders 0: utility for hierarchicalEntityPathfrom file path #4516DataLoaders 1: introduce, and migrate to,DataLoaders #4517DataLoaders 2: add text-basedDataLoader(.txt,.md) #4518DataLoaders 3: add 3D point cloudDataLoader(.ply) #4519DataLoaders 4: add generic folderDataLoader#4520DataLoaders 5: add support for external binaryDataLoaders (PATH) #4521DataLoaders 6: first-class support forIncompatible#4565DataLoaders 7: support for customDataLoaders #4566DataLoaders 8: docs, guide, etc #4567Checklist