Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix case where resource_name not set in stream error #746

Merged
merged 1 commit into from
Mar 10, 2024

Conversation

allada
Copy link
Member

@allada allada commented Mar 10, 2024

The RBE protocol specifies that if the first message in a stream has the resource_name set all subsequent messages do not need to have it set. With this patch we now honor that requirement for GrpcStore hotpath.

closes: #745


This change is Reviewable

Copy link
Member Author

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+@aaronmondal
+@zbirenbaum

fyi: @chrisstaite-menlo

Reviewable status: 0 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, publish-image, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable, zig-cc ubuntu-20.04, zig-cc ubuntu-22.04 (waiting on @aaronmondal and @zbirenbaum)

Copy link
Member Author

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, publish-image, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable (waiting on @aaronmondal and @zbirenbaum)


nativelink-util/src/proto_stream_utils.rs line 1 at r1 (raw file):

// Copyright 2023-2024 The NativeLink Authors. All rights reserved.

fyi: Sadly the diff didn't help here, but this file is just removed parts of grpc_store.rs merged with write_request_stream_wrapper.rs renamed.

Copy link
Member Author

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, publish-image, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable (waiting on @aaronmondal and @zbirenbaum)


nativelink-util/src/proto_stream_utils.rs line 1 at r1 (raw file):

// Copyright 2023-2024 The NativeLink Authors. All rights reserved.

This link might be helpful:
https://www.diffchecker.com/VifJIlx2/

It's a copy-paste of what the merged files (only added, no modifications) diffed against the this new file.

Copy link
Contributor

@zbirenbaum zbirenbaum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 9 of 9 files at r1, all commit messages.
Reviewable status: 1 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, publish-image, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable (waiting on @aaronmondal)


nativelink-util/src/proto_stream_utils.rs line 1 at r1 (raw file):

Previously, allada (Nathan (Blaise) Bruer) wrote…

This link might be helpful:
https://www.diffchecker.com/VifJIlx2/

It's a copy-paste of what the merged files (only added, no modifications) diffed against the this new file.

Thanks, the diffchecker helps a ton

Copy link
Member

@aaronmondal aaronmondal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 8 of 9 files at r1, all commit messages.
Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), docker-compose-compiles-nativelink (22.04), integration-tests (20.04), integration-tests (22.04), macos-13, publish-image, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable


nativelink-util/src/proto_stream_utils.rs line 123 at r1 (raw file):

        // If we successfully got a message, update our internal state with the
        // message meta data.
        let maybe_message = match maybe_message {

nit: Is there some way to make this maybe_message logic (and the let above) more readable?

The RBE protocol specifies that if the first message in a stream
has the resource_name set all subsequent messages do not need to
have it set. With this patch we now honor that requirement for
GrpcStore hotpath.

closes: TraceMachina#745
@allada allada force-pushed the fix-resource-name-not-set-in-proto branch from ed68586 to 2674584 Compare March 10, 2024 21:33
Copy link
Member Author

@allada allada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 of 2 LGTMs obtained, and pending CI: Analyze (javascript-typescript), Vercel, asan / ubuntu-22.04, docker-compose-compiles-nativelink (20.04), pre-commit-checks, ubuntu-20.04 / stable, vale


nativelink-util/src/proto_stream_utils.rs line 123 at r1 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

nit: Is there some way to make this maybe_message logic (and the let above) more readable?

Done. This is about as far as I think we should take it.

@allada allada enabled auto-merge (squash) March 10, 2024 22:10
@allada allada merged commit a651f2c into TraceMachina:main Mar 10, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Hot path for GrpcStore requires resource_name be set on every message
3 participants