one easy and light crawler framework
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
crawlProxy
seaeels/seaeels
.gitignore
README.MD

README.MD

one minimum available application are as below:

class DoubanModel(BaseModels):
        # name = StringField(pattern='.title.string')  # StringField id required to be declare
        name = 'title'

    range_site = RangeSite(name='douban', model=DoubanModel)

    @range_site.route(start_url='https://book.douban.com/tag/互联网?start=',
                      page_range=xrange(0, 100, 20))
    def crawl_douban():
        # write_to_output(db_name='douban.db')
        print 'done ---'