Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible to calculate cid using Writer interface instead of reading full file into memory? #120

Closed
shoce opened this issue Mar 28, 2021 · 3 comments
Labels
need/triage Needs initial labeling and prioritization

Comments

@shoce
Copy link

shoce commented Mar 28, 2021

As Prefix.Sum() accepts []byte i need to read the full file into memory and then feed to .Sum(). How can i make use of streaming with io.Copy() instead and reduce memory requirements?

@shoce shoce added the need/triage Needs initial labeling and prioritization label Mar 28, 2021
@welcome
Copy link

welcome bot commented Mar 28, 2021

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

@shoce
Copy link
Author

shoce commented Mar 29, 2021

After some investigation, I think it is more a question for github.com/multiformats/go-multihash codebase.

@warpfork
Copy link
Member

warpfork commented Apr 6, 2021

We did recently add features to go-multihash to make it usable in a streaming fashion :)

Unfortunately, you're right, go-cid still doesn't have exposed APIs for streaming usage today. More work is needed on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/triage Needs initial labeling and prioritization
Projects
None yet
Development

No branches or pull requests

2 participants