Closed
Description
Hi, I'm trying to run scrapy from a script like this:
import scrapy
from scrapy.crawler import CrawlerProcess
class MySpider(scrapy.Spider):
name = "basic"
allowed_domains = ["web"]
start_urls = ['http://www.example.com']
def parse(self, response):
l = ItemLoader(item=PropertiesItem(), response = response)
l.add_xpath('title', '//h1[1]/text()')
return l.load_item()
process = CrawlerProcess({
'USER_AGENT': 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)'
})
process.crawl(MySpider)
process.start()
However, when I run this script I get the following error:
File "/Library/Python/2.7/site-packages/Twisted-16.7.0rc1-py2.7-macosx-10.11-
intel.egg/twisted/internet/_sslverify.py", line 38, in <module>
TLSVersion.TLSv1_1: SSL.OP_NO_TLSv1_1,
AttributeError: 'module' object has no attribute 'OP_NO_TLSv1_1'
Does anyone know how to fix this? Thanks in advance.