No description, website, or topics provided.
JavaScript PHP CSS Python Other
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
common
data
mcso
scripts
setup
view
README
example.txt
scrapy.cfg

README

I use this scraper for good, not evil.

requirements:

    python 2.x
    scrapy
    flask
    flask-login
    py-bcrypt
    local mail server

set up storage the first time:

    python setup/setup.py boot

update storage after updating source:

    python setup/setup.py

crawl:

    scrapy crawl mcso

view:
    
    start server:
    env PYTHONPATH=. view/server.py
    view:
    visit localhost:5000/ in web browser

user management:

    scripts/add_user.py deactivate_user.py show_users.py