RustBoard tasks

[wchargin]

Feature tasks

If you’re interested, let me know, and I’ll assess whether it’s a good idea and
provide guidance if appropriate.

Small-ish

• @bileschi: Plugin listing (rust: implement ListPlugins RPC #4487)
  • touches: server
  • straightforward; good starter task
• @wchargin: Data location and experiment metadata (rust: implement data location #4751)
  • touches: server, Python data provider, RPC protos
  • pretty straightforward; good starter task
• @wchargin: Make CRC checking optional (rust: make CRC checking optional #4573)
  • touches: core, CLI
  • straightforward; good starter task
• @nfelt: Online commits (commit data periodically while runs are loading) (rust: make RunLoader commit periodically while loading a large run #4604)
  • touches: core only
  • fairly straightforward; good starter task
• @psybuzz: Samples-per-plugin (Plumb --samples_per_plugin to Rust data server #4689)
  • touches: core, CLI, Python ingester
  • probably pretty straightforward? could be a good starter task
  • optional extra: support unbounded reservoirs (rust: support unbounded reservoirs #4754)

Involved

• @nfelt: Tensors core (rust: implement tensor RPCs and client #4726)
  • touches: core, server, Python data provider
• @nfelt: Tensors data compat
  • histograms (rust: support legacy TF 1.x histogram summaries #4740)
  • PR curves (rust: implement data_compat.rs basic tensor support #4734)
  • hparams (rust: interpret summaries with no value as null tensors #4735)
  • text (rust: implement data_compat.rs basic tensor support #4734)
• @wchargin: Blob sequences (rust: implement blob sequence RPCs and client #4563, et al.)
  • touches: core, server, Python data provider
  • needs generic blob sequence support plus compat for images,
    audio, graphs
  • expect some moderately tricky stuff on the server since
    ReadBlob is a server-streaming RPC
• @wchargin: Blob data compat
  • images (rust: support images and generic blob sequences #4564)
  • audio (rust: support audio #4566)
  • graphs (rust: load graphs into memory as blob sequences #4544)
  • graphs sub-plugins (Keras, etc.) (rust: support tagged run metadata graphs #4568, rust: support graph sub-plugins #4569)
• @wchargin: Parallelize across runs (rust: load runs in parallel #4634)
  • touches: core only
  • expect to think about lifetimes
  • consider crossbeam for scoped threads and maybe SegQueue rayon!
• Optimize run-tag filters over small input sets
  • i.e.: iterate over the filter, not the map
  • touches: server only
  • will be harder than it sounds :-)
  • mostly for learning purposes; will be faster, but won’t matter
    in practice
• Reload-multifile
  • touches: core, CLI
  • note: current logic does always read from all files; it just
    never marks old files as dead

These are mostly for feature parity. There are further ideas for new
features—indexing, deferred blob loading, etc.—but let’s hold off on
those.

Infra tasks

(I plan to take care of these.)

• Filesystems (rust: extract Logdir trait for other filesystems #4636)
  • touches: core, CLI
• Pull main.rs into rustboard_core::cli
  • dovetails with filesystems: bring your own entry point
• Pip packaging
  • set up package structure and integrate with TensorBoard
  • release Pip package (pre-release is okay)
  • add Pip dependency from tensorboard to tensorboard-data-server (pip: depend on tensorboard-data-server #4676)
• Cross-platform builds
  • trivial to cross-compile to pc-windows-gnu, less so to pc-windows-msvc
  • osxcross?
  • just build ’em all on GitHub Actions? has VMs for all relevant
    platforms
• Benchmarks
  • touches: core, build
  • status: have a starting point; can expand as needed

Other tasks that I plan to investigate:

• Graph plugin parity (graph: use data provider API exclusively #4473, et al.)
  • touches: Python graph plugin, both data_compat
  • profile data, non-run graphs with data provider
• Projector plugin parity (projector: read run names from data provider #4494, et al.)
  • touches: Python projector plugin
  • just needs to work without multiplexer; can still hit local disk
    rather than actually using the data provider
• Mesh plugin parity (mesh: add generic data support #4775)
  • touches: Python mesh plugin
  • just needs to rewire multiplexer to data provider, using tensors
    (see discussion on rust: implement data_compat.rs basic tensor support #4734)
• Custom scalars plugin parity (custom_scalar: add generic data support #4774)
  • touches: Python custom scalars plugin, one-line changes to both datacompats
• DebuggerV2 well-behavedness (debugger_v2: start multiplexer lazily #4659)
  • needs to not hammer GCS
  • not strictly RustBoard, but stops --load_fast from being fast

[bileschi]

May I reserve plugin listing?

[wchargin]

Sure! You’ll want to update the list_plugins RPC handler to iterate
over the map of data and extract the plugin names from the summary
metadata, rather than just returning a hard-coded "scalars".

You can unit test this easily (follow existing examples), and for an
end-to-end test you can compare a logdir with scalars to an empty logdir
(since we don’t yet have tensor/blob sequence support).

[wchargin]

I’ve grabbed blob sequence support. It’s more involved than tensors but
teaches approx. the same lessons, so I figure that I can grab it.

[wchargin]

I ended up also grabbing “make CRCs optional” because I was curious and
it turns out to make things 15–60% faster (#4573), so I didn’t want to
just leave that on the table that it might potentially get dropped.

[wchargin]

Thanks to @bileschi, @psybuzz, and @nfelt for your contributions! We’ve
gotten all the important stuff by this point, so I’m closing this.