Rerun streams proxy #4615
Assignees
Labels
CLI
Related to the Rerun CLI
enhancement
New feature or request
user-request
This is a pressing issue for one of our users
Milestone
Comments
nikolausWest
added
enhancement
|
#4834 adds a Running just |
emilk
added a commit
that referenced
this issue
Jan 17, 2024
* Closes #4615 * Closes #1342 ## New `--serve` flag Rerun now has `--serve` flag which acts very similar to the `serve()` sink of the logging SDK. It hosts a Rerun Server which you can connect to with another rerun viewer (native or web). Loading an .rrd and serving it: > $ rerun ../objectron.rrd --serve > [2024-01-16T20:57:04Z INFO re_data_source::load_file] Loading "../objectron.rrd"… > [2024-01-16T20:57:04Z INFO re_ws_comms::server] Hosting a WebSocket server on ws://localhost:9877. You can connect to this with a native viewer (`rerun ws://localhost:9877`) or the web viewer (with `?url=ws://localhost:9877`). > [2024-01-16T21:11:18Z INFO re_sdk::web_viewer] Hosting a web-viewer at http://localhost:9090?url=ws://localhost:9877 Listen for SDK connection, and serving it to viewers: > $ rerun --serve > [2024-01-16T20:58:03Z INFO re_sdk_comms::server] Hosting a SDK server over TCP at 0.0.0.0:9876. Connect with the Rerun logging SDK. > [2024-01-16T20:58:03Z INFO re_ws_comms::server] Hosting a WebSocket server on ws://localhost:9877. You can connect to this with a native viewer (`rerun ws://localhost:9877`) or the web viewer (with `?url=ws://localhost:9877`). > [2024-01-16T21:11:18Z INFO re_sdk::web_viewer] Hosting a web-viewer at http://localhost:9090?url=ws://localhost:9877 The option `--web-viewer` is very similar - it does the same thing as `serve`, but also opens a browser on the user machine. It is kept for convenience and backwards compatibility. ## Improved `--help` To help the user find the new `--serve`, the `rerun --help` text has been improved a lot. I've tightened it up a bit, and appended a list of examples at the end. I put it as the end so it is always visible in the terminal without having to scroll. `rerun --help` output: ``` The Rerun Viewer and Server Usage: rerun [OPTIONS] [URL_OR_PATHS]... [COMMAND] Commands: analytics Configure the behavior of our analytics compare Compares the data between 2 .rrd files, returning a successful shell exit code if they match print Print the contents of an .rrd file reset Reset the memory of the Rerun Viewer help Print this message or the help of the given subcommand(s) Arguments: [URL_OR_PATHS]... Any combination of: - A WebSocket url to a Rerun Server - An HTTP(S) URL to an .rrd file to load - A path to an rerun .rrd recording - A path to an image or mesh, or any other file that Rerun can load (see https://www.rerun.io/docs/howto/open-any-file) If no arguments are given, a server will be hosted which a Rerun SDK can connect to. Options: --bind <BIND> What bind address IP to use [default: 0.0.0.0] --drop-at-latency <DROP_AT_LATENCY> Set a maximum input latency, e.g. "200ms" or "10s". If we go over this, we start dropping packets. The default is no limit, which means Rerun might eat more and more memory and have longer and longer latency, if you are logging data faster than Rerun can index it. --memory-limit <MEMORY_LIMIT> An upper limit on how much memory the Rerun Viewer should use. When this limit is reached, Rerun will drop the oldest data. Example: `16GB` or `50%` (of system total). [default: 75%] --server-memory-limit <SERVER_MEMORY_LIMIT> An upper limit on how much memory the WebSocket server should use. The server buffers log messages for the benefit of late-arriving viewers. When this limit is reached, Rerun will drop the oldest data. Example: `16GB` or `50%` (of system total). [default: 25%] --persist-state Whether the Rerun Viewer should persist the state of the viewer to disk. When persisted, the state will be stored at the following locations: - Linux: /home/UserName/.local/share/rerun - macOS: /Users/UserName/Library/Application Support/rerun - Windows: C:\Users\UserName\AppData\Roaming\rerun --port <PORT> What TCP port do we listen to (for SDKs to connect to)? [default: 9876] --profile Start with the puffin profiler running --save <SAVE> Stream incoming log events to an .rrd file at the given path --screenshot-to <SCREENSHOT_TO> Take a screenshot of the app and quit. We use this to generate screenshots of our exmples. Useful together with `--window-size` --serve Serve the recordings over WebSocket to one or more Rerun Viewers. This will also host a web-viewer over HTTP that can connect to the WebSocket address, but you can also connect to with the native binary. `rerun --serve` will act like a proxy, listening for incoming TCP connection from logging SDKs, and forwarding it to Rerun viewers. --skip-welcome-screen Do not display the welcome screen --version Print version and quit --web-viewer Start the viewer in the browser (instead of locally). Requires Rerun to have been compiled with the 'web_viewer' feature. This implies `--serve`. --web-viewer-port <WEB_VIEWER_PORT> What port do we listen to for hosting the web viewer over HTTP. A port of 0 will pick a random port [default: 9090] --window-size <WINDOW_SIZE> Set the screen resolution (in logical points), e.g. "1920x1080". Useful together with `--screenshot-to` --ws-server-port <WS_SERVER_PORT> What port do we listen to for incoming websocket connections from the viewer A port of 0 will pick a random port [default: 9877] --test-receive Ingest data and then quit once the goodbye message has been received. Used for testing together with `RERUN_PANIC_ON_WARN=1`. Fails if no messages are received, or if no messages are received within a dozen or so seconds. -h, --help Print help (see a summary with '-h') Environment variables: RERUN_SHADER_PATH The search path for shader/shader-imports. WARNING: Shaders are embedded in some build configurations. RERUN_TRACK_ALLOCATIONS Track memory allocations to diagnose memory leaks in the viewer. WARNING: slows down the viewer by a lot! RUST_LOG Change the log level of the viewer, e.g. `RUST_LOG=debug`. WGPU_BACKEND Overwrites the graphics backend used, must be one of `vulkan`, `metal`, `dx12`, `dx11`, or `gl`. Default is `vulkan` everywhere except on Mac where we use `metal`. What is supported depends on your OS. WGPU_POWER_PREF Overwrites the power setting used for choosing a graphics adapter, must be `high` or `low`. (Default is `high`) Examples: Open a Rerun Viewer that listens for incoming SDK connections: rerun Load some files and show them in the Rerun Viewer: rerun recording.rrd mesh.obj image.png https://example.com/recording.rrd Open an .rrd file and stream it to a Web Viewer: rerun recording.rrd --web-viewer Host a Rerun Server which listens for incoming TCP connections from the logging SDK, buffer the log messages, and host the results over WebSocket: rerun --serve Host a Rerun Server which serves a recording over WebSocket to any connecting Rerun Viewers: rerun --serve recording.rrd Connect to a Rerun Server: rerun ws://localhost:9877 Listens for incoming TCP connections from the logging SDK and stream the results to disk: rerun --save new_recording.rrd ``` ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using newly built examples: [app.rerun.io](https://app.rerun.io/pr/4834/index.html) * Using examples from latest `main` build: [app.rerun.io](https://app.rerun.io/pr/4834/index.html?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [app.rerun.io](https://app.rerun.io/pr/4834/index.html?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG - [PR Build Summary](https://build.rerun.io/pr/4834) - [Docs preview](https://rerun.io/preview/9c96b628f09b33b7cca772540ebf43a252da5537/docs) <!--DOCS-PREVIEW--> - [Examples preview](https://rerun.io/preview/9c96b628f09b33b7cca772540ebf43a252da5537/examples) <!--EXAMPLES-PREVIEW--> - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) --------- Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
For some of our users it's very important to have multiple viewers viewing the same stream of logs.
You can currently achieve this by proxying through a web server (
rerun --web-server) but this is both awkward (pops up a browser tab) and hard to discover.I propose we introduce a dedicated proxy mode for the cli to make it easier route log streams between SDKs and viewers.
The simplest thing would be to implement taking only TCP, stdin, and files as input and outputting to stdout, file or websocket. If it can be done with a reasonable amount of effort, outputting to TCP would also be nice.
The text was updated successfully, but these errors were encountered: