Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add rerun --serve and improve --help #4834

Merged
merged 14 commits into from Jan 17, 2024
Merged

Add rerun --serve and improve --help #4834

merged 14 commits into from Jan 17, 2024

Conversation

emilk
Copy link
Member

@emilk emilk commented Jan 16, 2024

New --serve flag

Rerun now has --serve flag which acts very similar to the serve() sink of the logging SDK.
It hosts a Rerun Server which you can connect to with another rerun viewer (native or web).

Loading an .rrd and serving it:

$ rerun ../objectron.rrd --serve
[2024-01-16T20:57:04Z INFO re_data_source::load_file] Loading "../objectron.rrd"…
[2024-01-16T20:57:04Z INFO re_ws_comms::server] Hosting a WebSocket server on ws://localhost:9877. You can connect to this with a native viewer (rerun ws://localhost:9877) or the web viewer (with ?url=ws://localhost:9877).
[2024-01-16T21:11:18Z INFO re_sdk::web_viewer] Hosting a web-viewer at http://localhost:9090?url=ws://localhost:9877

Listen for SDK connection, and serving it to viewers:

$ rerun --serve
[2024-01-16T20:58:03Z INFO re_sdk_comms::server] Hosting a SDK server over TCP at 0.0.0.0:9876. Connect with the Rerun logging SDK.
[2024-01-16T20:58:03Z INFO re_ws_comms::server] Hosting a WebSocket server on ws://localhost:9877. You can connect to this with a native viewer (rerun ws://localhost:9877) or the web viewer (with ?url=ws://localhost:9877).
[2024-01-16T21:11:18Z INFO re_sdk::web_viewer] Hosting a web-viewer at http://localhost:9090?url=ws://localhost:9877

The option --web-viewer is very similar - it does the same thing as serve, but also opens a browser on the user machine. It is kept for convenience and backwards compatibility.

Improved --help

To help the user find the new --serve, the rerun --help text has been improved a lot.
I've tightened it up a bit, and appended a list of examples at the end. I put it as the end so it is always visible in the terminal without having to scroll.

rerun --help output:

The Rerun Viewer and Server

Usage: rerun [OPTIONS] [URL_OR_PATHS]... [COMMAND]

Commands:
  analytics  Configure the behavior of our analytics
  compare    Compares the data between 2 .rrd files, returning a successful shell exit code if they match
  print      Print the contents of an .rrd file
  reset      Reset the memory of the Rerun Viewer
  help       Print this message or the help of the given subcommand(s)

Arguments:
  [URL_OR_PATHS]...
          Any combination of:
          - A WebSocket url to a Rerun Server
          - An HTTP(S) URL to an .rrd file to load
          - A path to an rerun .rrd recording
          - A path to an image or mesh, or any other file that Rerun can load (see https://www.rerun.io/docs/howto/open-any-file)
          
          If no arguments are given, a server will be hosted which a Rerun SDK can connect to.

Options:
      --bind <BIND>
          What bind address IP to use
          
          [default: 0.0.0.0]

      --drop-at-latency <DROP_AT_LATENCY>
          Set a maximum input latency, e.g. "200ms" or "10s".
          
          If we go over this, we start dropping packets.
          
          The default is no limit, which means Rerun might eat more and more memory and have longer and longer latency, if you are logging data faster than Rerun can index it.

      --memory-limit <MEMORY_LIMIT>
          An upper limit on how much memory the Rerun Viewer should use.
          When this limit is reached, Rerun will drop the oldest data.
          Example: `16GB` or `50%` (of system total).
          
          [default: 75%]

      --server-memory-limit <SERVER_MEMORY_LIMIT>
          An upper limit on how much memory the WebSocket server should use.
          The server buffers log messages for the benefit of late-arriving viewers.
          When this limit is reached, Rerun will drop the oldest data.
          Example: `16GB` or `50%` (of system total).
          
          [default: 25%]

      --persist-state
          Whether the Rerun Viewer should persist the state of the viewer to disk.
          When persisted, the state will be stored at the following locations:
          - Linux: /home/UserName/.local/share/rerun
          - macOS: /Users/UserName/Library/Application Support/rerun
          - Windows: C:\Users\UserName\AppData\Roaming\rerun

      --port <PORT>
          What TCP port do we listen to (for SDKs to connect to)?
          
          [default: 9876]

      --profile
          Start with the puffin profiler running

      --save <SAVE>
          Stream incoming log events to an .rrd file at the given path

      --screenshot-to <SCREENSHOT_TO>
          Take a screenshot of the app and quit. We use this to generate screenshots of our exmples. Useful together with `--window-size`

      --serve
          Serve the recordings over WebSocket to one or more Rerun Viewers.
          
          This will also host a web-viewer over HTTP that can connect to the WebSocket address, but you can also connect to with the native binary.
          
          `rerun --serve` will act like a proxy, listening for incoming TCP connection from logging SDKs, and forwarding it to Rerun viewers.

      --skip-welcome-screen
          Do not display the welcome screen

      --version
          Print version and quit

      --web-viewer
          Start the viewer in the browser (instead of locally).
          
          Requires Rerun to have been compiled with the 'web_viewer' feature.
          
          This implies `--serve`.

      --web-viewer-port <WEB_VIEWER_PORT>
          What port do we listen to for hosting the web viewer over HTTP. A port of 0 will pick a random port
          
          [default: 9090]

      --window-size <WINDOW_SIZE>
          Set the screen resolution (in logical points), e.g. "1920x1080". Useful together with `--screenshot-to`

      --ws-server-port <WS_SERVER_PORT>
          What port do we listen to for incoming websocket connections from the viewer A port of 0 will pick a random port
          
          [default: 9877]

      --test-receive
          Ingest data and then quit once the goodbye message has been received.
          
          Used for testing together with `RERUN_PANIC_ON_WARN=1`.
          
          Fails if no messages are received, or if no messages are received within a dozen or so seconds.

  -h, --help
          Print help (see a summary with '-h')


Environment variables:
    RERUN_SHADER_PATH         The search path for shader/shader-imports. WARNING: Shaders are embedded in some build configurations.
    RERUN_TRACK_ALLOCATIONS   Track memory allocations to diagnose memory leaks in the viewer. WARNING: slows down the viewer by a lot!
    RUST_LOG                  Change the log level of the viewer, e.g. `RUST_LOG=debug`.
    WGPU_BACKEND              Overwrites the graphics backend used, must be one of `vulkan`, `metal`, `dx12`, `dx11`, or `gl`.
                              Default is `vulkan` everywhere except on Mac where we use `metal`. What is supported depends on your OS.
    WGPU_POWER_PREF           Overwrites the power setting used for choosing a graphics adapter, must be `high` or `low`. (Default is `high`)


Examples:
    Open a Rerun Viewer that listens for incoming SDK connections:
        rerun

    Load some files and show them in the Rerun Viewer:
        rerun recording.rrd mesh.obj image.png https://example.com/recording.rrd

    Open an .rrd file and stream it to a Web Viewer:
        rerun recording.rrd --web-viewer

    Host a Rerun Server which listens for incoming TCP connections from the logging SDK, buffer the log messages, and host the results over WebSocket:
        rerun --serve

    Host a Rerun Server which serves a recording over WebSocket to any connecting Rerun Viewers:
        rerun --serve recording.rrd

    Connect to a Rerun Server:
        rerun ws://localhost:9877

    Listens for incoming TCP connections from the logging SDK and stream the results to disk:
        rerun --save new_recording.rrd

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG

@emilk emilk added include in changelog CLI Related to the Rerun CLI labels Jan 16, 2024
@emilk emilk mentioned this pull request Jan 16, 2024
@emilk emilk marked this pull request as ready for review January 16, 2024 21:27
Copy link
Member

@teh-cmc teh-cmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works great.


Don't know whether that's new but the CLI fails to build without default features:
image

crates/rerun/src/run.rs Outdated Show resolved Hide resolved
crates/rerun/src/run.rs Outdated Show resolved Hide resolved
@@ -540,29 +593,30 @@ async fn run_impl(
server_memory_limit,
)
.await?;
let ws_server_url = ws_server.server_url();
let _ws_server_url = ws_server.server_url();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let _ws_server_url = ws_server.server_url();
let ws_server_url = ws_server.server_url();

Comment on lines +600 to +601
#[cfg(feature = "web_viewer")]
{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[cfg(feature = "web_viewer")]
{
if cfg!(feature = "web_viewer") {

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sadly no; we don't want to pull in the web viewer crate unless the feature is explicitly on

emilk and others added 3 commits January 17, 2024 09:45
Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
Co-authored-by: Clement Rey <cr.rey.clement@gmail.com>
@emilk emilk merged commit 288cef7 into main Jan 17, 2024
39 of 40 checks passed
@emilk emilk deleted the emilk/proxy branch January 17, 2024 09:27
@nikolausWest
Copy link
Member

While this includes a super awesome improvement to the CLI docs I still think it warrants a "How to" guide on the docs page explaining how to set up different remote topologies. (Not in this PR)

We should also add a page under docs/reference that documents the rerun CLI. It could just be the same info as here. I think generally we shouldn't require someone to install and run Rerun to find the documentation describing what Rerun can do

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLI Related to the Rerun CLI include in changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rerun streams proxy Document how to connect remotely
3 participants