Skip to content
This repository has been archived by the owner on Dec 9, 2018. It is now read-only.

multiprocessing with dryscrape #50

Closed
deter3 opened this issue Mar 20, 2016 · 0 comments
Closed

multiprocessing with dryscrape #50

deter3 opened this issue Mar 20, 2016 · 0 comments

Comments

@deter3
Copy link

deter3 commented Mar 20, 2016

This is not an issue , but sharing how to deal with xvfb .

I am using celery to run multiple sessions of webkit-server . I need to remove ' dryscrape.start_xvfb() ' in the code and put xvfb-run before celery command , like this

xvfb-run -a celery -A googlescrape worker --loglevel=INFO --concurrency=10 -Q googlescrape_task

I spent couple of hours to figure out and hope can help somebody to save the time .

@deter3 deter3 closed this as completed Mar 20, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant