Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support generator functions #37

Open
yarikoptic opened this issue Mar 11, 2021 · 1 comment
Open

support generator functions #37

yarikoptic opened this issue Mar 11, 2021 · 1 comment
Assignees

Comments

@yarikoptic
Copy link
Member

the simplest implementation would be to detect if a function to be wrapped is producing a generator function (inspect.isgeneratorfunction), and thus fscacher would cache all yielded results and wrapped function would also be a generator possibly just re-yielding them from the cache.

use case which inspired it -- caching of hash computation which would yield progress status while computing it: dandi/dandi-cli#400 . But I think, although could be used, in this case we need some different construct since we would not be interested in all the progress indicators if value is cached. So, although inspired by the use case, and could be used, it would be not a complete optimal solution for it.

@jwodder
Copy link
Member

jwodder commented Mar 11, 2021

@yarikoptic The naïve way to implement this would be to just wrap the generator output in a list, but that would not work for our use case, as then the digester would block before returning the progress information all at once. Other than that, I don't see how this would be implemented without being able to directly query & modify the cache.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants