-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch all dependencies to pure python and release ArchiveBox pip package #177
Comments
awesome, can't wait to see that one fly! :) let me know if you need help testing the stuff or get stuck. |
Anything you're fetching with |
wpull only officially supports Python 3.4 and 3.5, even now it seems. The most recent commit was in Oct. 2019, and the version on PyPI is still outdated. It's a cool tool, but I would not recommend using it, and it doesn't seem to be well maintained. If you still want to use it anyway, you can install it from Git, and then use a Python dependency manager to only use Python 3.5 for it, but I would not recommend that. Git install:
|
Yeah I looked at wpull recently and came to the same conclusion. Wget2 looks more promising than wpull. I think I'm going to close this issue for now, as we start to expand the suite of archiving methods it's looking more and more like many of them will be node-based. Considering we already support |
The other issue I see with this is managing conflicting versions of Python dependencies for these tools. I would personally recommend Poetry for that, as it's popular and I've had great experiences with it, but whatever you choose, I still think is an important step. Apologies if you were already going to do this. I also don't see the value in replacing |
I originally thought moving to Python-only dependencies would be intractable, but after some more research I now realize this is quite straightforward.
apt install curl
->pip install requests archivenow
(requests docs, archivenow docs)apt install wget
->pip install wpull pywb
(wpull docs, pywb docs)apt install git
->pip install GitPython
(docs)apt install youtube-dl
->pip install youtube-dl
(docs)apt install chromium-browser
->pip install pyppeteer
(docs)Then we wont need users to install any system dependencies anymore, and we can move to using only
requirements.txt
andsetup.py
to install ArchiveBox viapip
.The text was updated successfully, but these errors were encountered: