We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Not sure if it has to do with the new update to this software or not. This is my first time trying scraper.
Am using python V 2.7.6 and scrapy v1.1.0
when trying scrapy crawl dlink, here is the output:
2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A', 'date': datetime.datetime(2013, 7, 16, 0, 0), 'description': u'Firmware (4.10B15)', 'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP', 'product': u'DXS-3326GSR', 'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.10B15.EXE', 'vendor': 'dlink', 'version': u'4.10B15'} Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks current.result = callback(current.result, _args, *_kw) File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item requests = arg_to_iter(self.get_media_requests(item, info)) File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests for x in ["mib", "url"] if x in item] AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD' 2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A', 'date': datetime.datetime(2013, 7, 16, 0, 0), 'description': u'Firmware (4.20B11)', 'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP', 'product': u'DXS-3326GSR', 'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.20B11.ZIP', 'vendor': 'dlink', 'version': u'4.20B11'} Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks current.result = callback(current.result, _args, *_kw) File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item requests = arg_to_iter(self.get_media_requests(item, info)) File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests for x in ["mib", "url"] if x in item] AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD' 2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A', 'date': datetime.datetime(2013, 7, 16, 0, 0), 'description': u'Firmware (4.30B11)', 'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP', 'product': u'DXS-3326GSR', 'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.30B11.ZIP', 'vendor': 'dlink', 'version': u'4.30B11'} Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks current.result = callback(current.result, _args, *_kw) File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item requests = arg_to_iter(self.get_media_requests(item, info)) File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests for x in ["mib", "url"] if x in item] AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD'
This Attribute error is actually coming up on all models that I tried to scrapy. Any idea why?
The text was updated successfully, but these errors were encountered:
Should be fixed in 15883b0
Sorry, something went wrong.
No branches or pull requests
Not sure if it has to do with the new update to this software or not. This is my first time trying scraper.
Am using python V 2.7.6 and scrapy v1.1.0
when trying scrapy crawl dlink, here is the output:
2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A',
'date': datetime.datetime(2013, 7, 16, 0, 0),
'description': u'Firmware (4.10B15)',
'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP',
'product': u'DXS-3326GSR',
'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.10B15.EXE',
'vendor': 'dlink',
'version': u'4.10B15'}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
current.result = callback(current.result, _args, *_kw)
File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item
requests = arg_to_iter(self.get_media_requests(item, info))
File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests
for x in ["mib", "url"] if x in item]
AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD'
2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A',
'date': datetime.datetime(2013, 7, 16, 0, 0),
'description': u'Firmware (4.20B11)',
'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP',
'product': u'DXS-3326GSR',
'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.20B11.ZIP',
'vendor': 'dlink',
'version': u'4.20B11'}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
current.result = callback(current.result, _args, *_kw)
File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item
requests = arg_to_iter(self.get_media_requests(item, info))
File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests
for x in ["mib", "url"] if x in item]
AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD'
2016-06-04 20:35:02 [scrapy] ERROR: Error processing {'build': u'A',
'date': datetime.datetime(2013, 7, 16, 0, 0),
'description': u'Firmware (4.30B11)',
'mib': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_MIBS_4.40B02.ZIP',
'product': u'DXS-3326GSR',
'url': u'ftp://FTP2.DLINK.COM/PRODUCTS/DXS-3326GSR/REVA/DXS-3326GSR_FIRMWARE_4.30B11.ZIP',
'vendor': 'dlink',
'version': u'4.30B11'}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
current.result = callback(current.result, _args, *_kw)
File "/usr/local/lib/python2.7/dist-packages/scrapy/pipelines/media.py", line 44, in process_item
requests = arg_to_iter(self.get_media_requests(item, info))
File "/root/firmadyne/sources/scraper/firmware/pipelines.py", line 61, in get_media_requests
for x in ["mib", "url"] if x in item]
AttributeError: 'FirmwarePipeline' object has no attribute 'FILES_URLS_FIELD'
This Attribute error is actually coming up on all models that I tried to scrapy. Any idea why?
The text was updated successfully, but these errors were encountered: