Open image and mesh files with drag-drop and File->Open #3116

emilk · 2023-08-28T07:07:15Z

What

Part of All files Rerun can open should support command line, drag, and open from menu #2229

Supports mixing of:

.rrd over HTTP
File paths
WebSockets urls

in rerun command-line arguments. All will be streamed to the viewer in parallel.

File->Open and drag-and-dropping of files will use the same code path, and now supports meshes and images on native.

Saving for later PR:

Drag-and-drop of images and meshes on web
Loading images and meshes over HTTP

How

Created a new re_data_source crate with an enum DataSource plus all the functions to stream data from all these sources.

I also removed RemoteViewerApp which I originally created to have a custom GUI for selecting what server to connect to, but that GUI has been disabled since forever, so high time to remove it.

On web, care is taken to wake up the ui thread when loading data from web-sockets or HTTP.
On native, we just spin up another thread that notices new messages and wakes up the ui.

Checklist

I have read and agree to Contributor Guide and the Code of Conduct
I've included a screenshot or gif (if applicable)
I have tested demo.rerun.io (if applicable)

github-actions · 2023-08-28T08:20:19Z

Size changes

Name	Previous	Current	Change
Wasm	13.71 MiB	14.40 MiB	-4.79%

abey79

Great stuff! Lot's of the heavy lifting is beyond me, but I still got a few comments along the way (some may be me trying to understand more than anything else).

abey79 · 2023-08-28T13:54:22Z

crates/re_data_source/src/data_source.rs

+        } else if uri.starts_with("ws://") || uri.starts_with("wss://") {
+            DataSource::WebSocketAddr(uri)
+
+        // Now we are into heuristics territory:


I'm curious as to what the use cases for that heuristic. Wouldn't it be cleaner to bail and produce a proper error? On CLI, it certainly feels so ("Argument XXX couldn't be resolved and was ignored"). Try to construct a Path from the input and checking for file existence feels like it should behave OK for CLI.

For drag and drop/File->Open, the OS should provide a well formed path by definition.

This is all for making the CLI life nicer.

rerun 127.0.0.1 rerun foo.jpg rerun www.host.com/bar.rrd …

it's nice not having to spell out the protocol for each of these

I guess what I fail to understand here is the use of looks_like_a_file_path(). Isn't that handled by feeding whatever we have to Path first, and check for existence? The other heuristics do make sense to me.

crates/re_data_source/src/data_source.rs

crates/re_smart_channel/src/receive_set.rs

crates/re_viewer/src/app.rs

crates/re_viewer/src/app_state.rs

Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>

abey79

That welcome screen workflow is not done haunting us :)

crates/re_viewer/src/app.rs

abey79 · 2023-08-28T16:58:41Z

crates/re_viewer/src/app.rs

                SmartChannelSource::File(_) | SmartChannelSource::RrdHttpStream { .. } => {
-                    return true;
+                    return false;


This is not going to cut it either, I should have been clearer earlier—sorry for that. This also breaks the demo workflow: the loading screen is shown initially, but if you then close the recording, we're back to the loading screen—but that time it should be the welcome screen. This is way I originally used is_connected().

I'm not quite sure how to proceed here. That "is_connected" trick might be valid only if this is the only stream. The hole we dug with that whole "when to show welcome screen" thing is only getting deeper now 😓

Maybe that's something we can revisit later (maybe when my brain fog dissipate), in this case an issue with 0.9 milestone will do.

Argh

I think #3125 covers it well

Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>

Broke in #3116

Broke in #3116 Tried to use tokio from a rayon thread, which doesn't work out-of-the-box. But the rayon use is unnecessary - each call to `data_source.stream` is non-blocking anyways. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested [demo.rerun.io](https://demo.rerun.io/pr/3160) (if applicable) - [PR Build Summary](https://build.rerun.io/pr/3160) - [Docs preview](https://rerun.io/preview/11d5fea9d4982d50c808bc571299a9ded4d37177/docs)  - [Examples preview](https://rerun.io/preview/11d5fea9d4982d50c808bc571299a9ded4d37177/examples)  - [Recent benchmark results](https://ref.rerun.io/dev/bench/) - [Wasm size tracking](https://ref.rerun.io/dev/sizes/)

emilk added 10 commits August 28, 2023 09:15

Break up re_smart_channel in modules

7cc3bb8

Add ReceiveSet

1f88122

Arc SmartChannelSource

4e5361c

Simplify some code

e996d84

Integrate the ReceiveSet into App

4832ed5

Silence clippy

836f454

Rename ArgumentCategory to DataSource

fa5d00a

Use the set in run.rs

347d1e5

SmartChannelSource::Files -> SmartChannelSource::File

49e6b25

Move data_source into own crate

7c77d75

emilk force-pushed the emilk/channel-set branch from 578c0e7 to 7c77d75 Compare August 28, 2023 07:41

Integrate DataSource into app.rs

e10e059

emilk added 🕸️ web regarding running the viewer in a browser 📺 re_viewer affects re_viewer itself labels Aug 28, 2023

emilk added 13 commits August 28, 2023 10:21

Use DataSource on drag-drop

1ef3408

extend vscode debug launch.json

7b8a707

Fix bug in receiver set

3dbee96

Remove some dead code

1732162

Make sure we wake up the ui thread

89cd870

Better error messages

3c3261e

Break up module into several files

11059f4

Load files in background thread

d787590

Code cleanup

040cc36

Remove RemoteViewerApp

ab29413

typo

8d0c542

Load screen cleanup

0b7e52c

Load .rrd files in background threads

8e1ca38

emilk force-pushed the emilk/channel-set branch from 6c16a87 to 8e1ca38 Compare August 28, 2023 10:28

Profile scopes

8958e16

emilk added 8 commits August 28, 2023 12:33

Cleanup for symmetry

3526f0e

Remove TODO

e176796

Explain why we cannot ban PathBuf on clippy

11bb770

Support loading images and meshes with File->Open on native

36badba

Code cleanup

6df8ad0

More code cleanup

90cb11b

Update to poll-promise 0.3

b9c32be

File->Open multiple files on web

7869ec0

emilk marked this pull request as ready for review August 28, 2023 13:41

emilk added 2 commits August 28, 2023 15:42

Remove unnecessary bail

38a44ab

cargo fmt

cd76378

abey79 approved these changes Aug 28, 2023

View reviewed changes

emilk and others added 10 commits August 28, 2023 17:02

typo

6f7e02e

Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>

Another typo

fdca533

Add clarifying comment

8413d3b

Code cleanup

a375294

Tweak welcome-screen heuristic

500a693

Simplify is_known_file_extension

13b02e1

Merge branch 'main' into emilk/channel-set

9921aa7

Document re_data_source crate

ec049e9

Fix doclinks

097014d

Fix test

7e0a07f

abey79 approved these changes Aug 28, 2023

View reviewed changes

typo

cfd5d11

Co-authored-by: Antoine Beyeler <49431240+abey79@users.noreply.github.com>

emilk merged commit 315f87e into main Aug 28, 2023
20 checks passed

emilk deleted the emilk/channel-set branch August 28, 2023 17:11

abey79 mentioned this pull request Aug 28, 2023

Add fade-in, fade-out to Welcome screen and simplify Welcome Screen vs. Loading logic #3125

Closed

emilk added a commit that referenced this pull request Aug 30, 2023

Bug fix: cargo rerun websocket-url

11d5fea

Broke in #3116

emilk mentioned this pull request Aug 30, 2023

Bug fix: cargo rerun websocket-url #3160

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Open image and mesh files with drag-drop and File->Open #3116

Open image and mesh files with drag-drop and File->Open #3116

emilk commented Aug 28, 2023 •

edited by github-actions bot

github-actions bot commented Aug 28, 2023

abey79 left a comment

abey79 Aug 28, 2023

emilk Aug 28, 2023

abey79 Aug 28, 2023

abey79 left a comment

abey79 Aug 28, 2023

emilk Aug 28, 2023

Open image and mesh files with drag-drop and File->Open #3116

Open image and mesh files with drag-drop and File->Open #3116

Conversation

emilk commented Aug 28, 2023 • edited by github-actions bot

What

Saving for later PR:

How

Checklist

github-actions bot commented Aug 28, 2023

Size changes

abey79 left a comment

Choose a reason for hiding this comment

abey79 Aug 28, 2023

Choose a reason for hiding this comment

emilk Aug 28, 2023

Choose a reason for hiding this comment

abey79 Aug 28, 2023

Choose a reason for hiding this comment

abey79 left a comment

Choose a reason for hiding this comment

abey79 Aug 28, 2023

Choose a reason for hiding this comment

emilk Aug 28, 2023

Choose a reason for hiding this comment

emilk commented Aug 28, 2023 •

edited by github-actions bot