You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm working on a side project with some friends and I stumbled across your scrapy repo. It looks really cool and exactly what I need to scrape the web for a bunch of firmware! I'm unfamiliar with the scrapy package, and your readme's recommendation of:
scrapy crawl dlink
has yielded nothing but errors and hanging for me. I'm still reading though scrapy docs and your code, but I'm struggling to get any of the spiders to crawl. I was hoping you might be able to give me some guidance on how to go about using your tool. If there are any config files, scrapy command line options, unlisted dependencies, or other things I might be missing, a bit of direction on how to start web-crawling would help me a lot.
Regardless, thank you for all your work on what looks like an amazing open source tool!
The text was updated successfully, but these errors were encountered:
I'm working on a side project with some friends and I stumbled across your scrapy repo. It looks really cool and exactly what I need to scrape the web for a bunch of firmware! I'm unfamiliar with the scrapy package, and your readme's recommendation of:
scrapy crawl dlink
has yielded nothing but errors and hanging for me. I'm still reading though scrapy docs and your code, but I'm struggling to get any of the spiders to crawl. I was hoping you might be able to give me some guidance on how to go about using your tool. If there are any config files, scrapy command line options, unlisted dependencies, or other things I might be missing, a bit of direction on how to start web-crawling would help me a lot.
Regardless, thank you for all your work on what looks like an amazing open source tool!
The text was updated successfully, but these errors were encountered: