Warc Creator

Warc Creator is a simple web server to asynchronously archive web pages and post them to a callback URL.

Install

Install PhantomJS.

Install warc_creator:

pip install warc_creator

Start web server:

gunicorn -b '127.0.0.1:8001' warc_creator

Start celery:

celery --app=warc_creator worker

Begin warc-creation job:

curl http://127.0.0.1:8001/target_url=http://example.com&callback_url=http://127.0.0.1/handle_warc&user_agent=foo

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
requirements.txt		requirements.txt
tasks.py		tasks.py
warc_creator.py		warc_creator.py