Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hot path for GrpcStore requires resource_name be set on every message #745

Closed
allada opened this issue Mar 10, 2024 · 0 comments · Fixed by #746
Closed

Hot path for GrpcStore requires resource_name be set on every message #745

allada opened this issue Mar 10, 2024 · 0 comments · Fixed by #746

Comments

@allada
Copy link
Member

allada commented Mar 10, 2024

Discovered here:
https://nativelink.slack.com/archives/C06758HJQF8/p1709975017849379

We currently violate spec, because we require every message of a GRPC write stream have the resource_name set and the spec explicitly states this is not required:
https://github.com/googleapis/googleapis/blob/4ce0ff67a3d4509be641cbe47a35844ddc1268fc/google/bytestream/bytestream.proto#L127

This is client dependent and bazel is one of the clients that only sets it on the first message and leaves it None for all further chunks.

Offending code:

nativelink/nativelink-store/src/grpc_store.rs

Line 191 in 3e6f154

Some(Ok(mut message)) => match ResourceInfo::new(&message.resource_name, IS_UPLOAD_TRUE) {

allada added a commit to allada/nativelink-fork that referenced this issue Mar 10, 2024
@allada
Fix case where resource_name not set in stream error
ed68586 
The RBE protocol specifies that if the first message in a stream
has the resource_name set all subsequent messages do not need to
have it set. With this patch we now honor that requirement for
GrpcStore hotpath.

closes: TraceMachina#745
@allada allada mentioned this issue Mar 10, 2024
Merged
allada added a commit to allada/nativelink-fork that referenced this issue Mar 10, 2024
@allada
Fix case where resource_name not set in stream error
2674584 
The RBE protocol specifies that if the first message in a stream
has the resource_name set all subsequent messages do not need to
have it set. With this patch we now honor that requirement for
GrpcStore hotpath.

closes: TraceMachina#745
allada added a commit that referenced this issue Mar 10, 2024
@allada
Fix case where resource_name not set in stream error (#746)
a651f2c 
The RBE protocol specifies that if the first message in a stream
has the resource_name set all subsequent messages do not need to
have it set. With this patch we now honor that requirement for
GrpcStore hotpath.

closes: #745
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix case where resource_name not set in stream error allada/nativelink-fork
1 participant
@allada