1024Spider

Only for researching, please pay attention to your health.

I mainly want to learn about web spider using Python. But, nothing to crawl. Anyway, the 1024 website is a good example.

Configure

Use image.py as a example, it is a spider to crawl images of 1024.

1. Modify the value of DIR_PATH in line 14.
2. Find a useful url of 1024 and modify the value of URL in line 17.

Then, you can enjoy it.

Installation

1024Spider is depended on Pysipder. (https://github.com/binux/pyspider)

You can install Pyspider firstly and then create a project on WebUI. Copy my code into it.

If you need more infomation, please read the Pyspider Documentation(http://docs.pyspider.org/en/latest/Quickstart/).