-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add flag to disable file downloads #505
Conversation
a3b7501
to
1dee0d5
Compare
1dee0d5
to
f1920e0
Compare
82a3901
to
ca15281
Compare
Hm, this doesn't actually work as expected because a lot of importers just implement It's probably a good idea to clean that up in some future PRs so that it works nicely for all importers. Downloaders are a bit better at this because they implement |
ca15281
to
ee65dbb
Compare
3828e19
to
06525af
Compare
06525af
to
aae49c4
Compare
This is pretty much just a refactor at this point, since it relies on the importers to properly split |
I did'nt find a flag to set this option when adding entries ? I missed something? So I added the flag myself. |
This is an improvement to #387 that tries to avoid downloading files all together (and all the additional traffic!) when the right conditions are met. For this, it adds:
only_data
flag toget_matching_importer_or_downloader
only_data
flag toget_matching_importer_by_name
(new function inpapis.utils
)so that they only call
Importer.fetch_files
if asked. It updatespapis add
andpapis update
to use these new functions.It probably needs a bit more testing, but it seems pretty clean. @alejandrogallo What do you think?