Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow clients to provide InputDigests to avoid unnecessary transfer #525

Closed
bshashank opened this issue Nov 8, 2023 · 2 comments
Closed

Comments

@bshashank
Copy link

Today the API for the SDK only allows clients to specify Inputs (files available to the client) and VirtualInputs(binary available to clients) (from command.go)
But I would like to extend this to support providing digests that are available in the remote server.

I have a fork with this code that I am able to use for this. But I wanted to ask folks here if it would make sense to add to the SDK?

This would be useful when trying to do "build-without-bytes" for example.

@mrahs
Copy link
Collaborator

mrahs commented Nov 8, 2023

That's sounds reasonable. It looks like VirtualInputs can be extended to support that. Currently, the actual bytes are expected, but another field can be added to set the digest instead. Now this of course introduces an inconsistency problem if both the bytes and the digests are set, but that can be discussed if needs be.

My concern with adding a separate field for digests is that it needs to replicate some meta fields to describe the digest, such as the file type and executable bit.

It would be great if you can check if extending VirtualInputs in a backwards compatible way can fulfill your additional requirements. I'm happy to review PRs for this.

@bshashank
Copy link
Author

Sounds good, yeah I think extending VirtualInputs will be possible, and cleaner.

Now this of course introduces an inconsistency problem if both the bytes and the digests are set

I extend the Command.Validate() method to throw a useful error in this case to make it clear that the inputs are malformed.

andusy added a commit that referenced this issue Mar 14, 2024
Allows virtual inputs to be used with digests. If Content is provided
with InputDigest or the digest does not exist in the CAS, return an error.

Bug: b/324349357, #525
@mrahs mrahs closed this as completed Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants