-
Notifications
You must be signed in to change notification settings - Fork 4
/
爬虫导图.txt
executable file
·38 lines (20 loc) · 930 Bytes
/
爬虫导图.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
运行爬虫:scrapy crawl my
response.xpath('//*[@id="J_goodsList"]/ul/li[2]/div/div[4]/a/em/text()[2]').extract_first()
分析:
请求:http://www.gsxt.gov.cn/corp-query-search-test.html?searchword=名称
name = scrapy.Field()
id_code = scrapy.Field()
url_x = scrapy.Field()
people = scrapy.Field()
time = scrapy.Field()
response.xpath('//*[@id="advs"]/div/div[2]/a/@href').extract_first()
状态
//*[@id="advs"]/div/div[2]/a[1]/div[1]/span
//*[@id="advs"]/div/div[2]/a[3]/div[1]/span
//*[@id="advs"]/div/div[2]/a[4]/div[1]/span
#advs > div > div.search_result.g9
-------------------------------------------
shell测试:scrapy shell -s USER_AGENT='Mozilla/5.0' https://search.jd.com/Search?keyword=iphone&enc=utf-8&wq=iphone
运行爬虫:scrapy crawl my
----------------------
response.xpath('//*[@id="J_goodsList"]/ul/li[2]/div/div[4]/a/em/text()[2]').extract_first()