Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extractor yaml files should have --cache / mount options to avoid model re-downloads (and similar) #139

Open
yenicelik opened this issue Nov 19, 2023 · 1 comment

Comments

@yenicelik
Copy link
Contributor

Especially when using larger machine learning models (i.e. from huggingface), mounting a drive to the local huggingface cache will be useful. An optional "mount" parameter in the model yaml file could be a good way to implement this

@yenicelik
Copy link
Contributor Author

Instead (or in addition to) of an mount option, we could also accept another input file to accept an extension to a Dockerfile (which uses the FROM diptanu/indexify-base-extractor base image)

@yenicelik yenicelik changed the title Extractor yaml files should have mount options Extractor yaml files should have --cache / mount options to avoid model re-downloads (and similar) Dec 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant