Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API examples overhaul & roundtrip tests #3204

Merged
merged 21 commits into from Sep 6, 2023
Merged

API examples overhaul & roundtrip tests #3204

merged 21 commits into from Sep 6, 2023

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Sep 4, 2023

Commit by commit

This PR overhauls API examples to make them roundtrippable and checks those roundtrips on CI.

These roundtrips serve two purposes: A) the obvious one: they make sure that using our different SDKs in similar ways actually yield similar data, but more importantly B) they act as regression tests since it is highly unlikely that two or more tests written in different languages fail in the exact same way at the exact same time.

As I've painfully found out during the MsgSender migration, a lot of our tests had bitrotten and simply did not work in one way or another anymore (in fact I ended up fixing yet another batch for this PR).
This PR should provide the foundations so that this doesn't happen again.

A nice side-effect of this is the introduction of the _RERUN_TEST_FORCE_SAVE environment variable, which forces all RecordingStreams instantiated across all 3 languages to write to an rrd file on disk rather than do whatever they were asked to do.
This makes testing things much easier.

What

Checklist

Base automatically changed from cmc/msgsender_be_gone to main September 5, 2023 07:04
@teh-cmc teh-cmc force-pushed the cmc/roundtrip_docs branch 3 times, most recently from 185f377 to ed46e8e Compare September 5, 2023 09:04
@teh-cmc teh-cmc added 🔨 testing testing and benchmarks 🧑‍💻 dev experience developer experience (excluding CI) examples Issues relating to the Rerun examples labels Sep 5, 2023
@teh-cmc teh-cmc changed the title WIP: API examples overhaul & roundtrip tests API examples overhaul & roundtrip tests Sep 5, 2023
@teh-cmc teh-cmc marked this pull request as ready for review September 5, 2023 09:34
@@ -1097,7 +1144,12 @@ impl RecordingStream {
/// terms of data durability and ordering.
/// See [`Self::set_sink`] for more information.
pub fn connect(&self, addr: std::net::SocketAddr, flush_timeout: Option<std::time::Duration>) {
self.set_sink(Box::new(crate::log_sink::TcpSink::new(addr, flush_timeout)));
// NOTE: `forced_sink` is only used for tests, it's ok to unwrap.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙈

Really all of our sink APIs should uniformly return a RecordingStreamResult. Having connect actually fail if it can't talk to the server with an option for our current "keep retrying in a background thread" behavior would be a more predictable user experience.

crates/re_sdk/src/recording_stream.rs Show resolved Hide resolved
docs/code-examples/README.md Outdated Show resolved Hide resolved
docs/code-examples/README.md Outdated Show resolved Hide resolved
@@ -1,28 +1,37 @@
import rerun as rr
import rerun.experimental as rr2
from rerun.experimental import dt as rrd
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from rerun.experimental import dt as rrd
from rerun.experimental import dt as rrd

I think we should avoid rrd as the convention here due to name collision with rrd the file extension.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs/code-examples/arrow3d_simple.rs Outdated Show resolved Hide resolved
docs/code-examples/depth_image_3d.py Outdated Show resolved Hide resolved
docs/code-examples/depth_image_3d.rs Outdated Show resolved Hide resolved
docs/code-examples/depth_image_simple.py Outdated Show resolved Hide resolved
Copy link
Member

@jleibs jleibs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aside from the tensorid/meshid thing this looks great.

I also added an issue to check for TODOs in code examples (#3224). I think it's fine to have TODOs in main, we just need to be careful about not deploying them.

docs/code-examples/roundtrips.py Outdated Show resolved Hide resolved
{0.f, 1.f, 1.f},
};
rr_stream.log("segments", rr::LineStrips3D(points));
// TODO(#3202): I want to do this!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: we don't want to leak these kinds of TODOs into the embedded snippets in the doc-pages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't do anything about that for now; we'll cleanup during the pre-release doc pass.

#3224 is there to make sure we don't forget!

@teh-cmc
Copy link
Member Author

teh-cmc commented Sep 6, 2023

@github-actions
Copy link

github-actions bot commented Sep 6, 2023

Size changes

Name main 3204/merge Change
plots.rrd 194.56 kiB 184.32 kiB -5.26%

@teh-cmc teh-cmc merged commit 8b428ce into main Sep 6, 2023
19 checks passed
@teh-cmc teh-cmc deleted the cmc/roundtrip_docs branch September 6, 2023 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🧑‍💻 dev experience developer experience (excluding CI) examples Issues relating to the Rerun examples 🔨 testing testing and benchmarks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants