-
Notifications
You must be signed in to change notification settings - Fork 0
Cheedge/Scraper
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Realized functions: 0. basic scraper 1. scrap all pages -- (1. yeild new url; 2. request follow() 3. crawl follow) 2. scrap and download -- (add new piplines) 3. scrap with links -- (add new parse func: scrapy.Request) 4. store into file/database -- (pymysql) 5. scrap post request -- (scrapy.FormRequest()) Projects: 1. Arxiv Description: scrap arxiv newest physics paper with (title, authors, abstraction, and download link) 2. Translate Description: scrap translations from Googl Translate (https://translate.google.com/?sl=en&tl=de)
About
No description, website, or topics provided.
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published