Github spider with scrapy
skills | state |
---|---|
selectors | ✔️ |
items | ✔️ |
piplines | ✔️ |
spider with args | ✔️ |
middlewares | ❌ |
scrapy crawl github -a username="your github username"
or
scrapy crawl github -a usernames=['1','2']
for setting output path
scrapy crawl github -a username="your github username" -o test.json