Repository navigation
InvalidSchema? #3
Description
With the latest commit (e41e3a8), when running the Docker instance of CarbonDate in local mode, as per http://ws-dl.blogspot.com/2016/09/2016-09-20-carbon-dating-web-version-30.html, I occasionally get an exception like the following.
# sudo docker run --rm -it carbon ./main.py -l search http://www.google.com
cdGetBitly.py::GetBitlyJson(), please set bitly access token in config
(<class 'requests.exceptions.InvalidSchema'>, InvalidSchema("No connection adapters were found for 'hive.org.uk/wayback/archive/20080304103855/http://www.google.com/'",), <traceback object at 0x7f6baa816088>)
Traceback (most recent call last):
File "/usr/src/app/modules/cdGetArchives.py", line 135, in getArchives
date = getRealDate(archives[archive]["link"],archives[archive]["time"])
File "/usr/src/app/modules/cdGetArchives.py", line 85, in getRealDate
response = requests.get(url,headers=headers)
File "/usr/local/lib/python3.5/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python3.5/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 590, in send
adapter = self.get_adapter(url=request.url)
File "/usr/local/lib/python3.5/site-packages/requests/sessions.py", line 672, in get_adapter
raise InvalidSchema("No connection adapters were found for '%s'" % url)
requests.exceptions.InvalidSchema: No connection adapters were found for 'hive.org.uk/wayback/archive/20080304103855/http://www.google.com/'
runtime in seconds: 13
{
"URI": "http://www.google.com",
"Estimated Creation Date": "2003-01-14T00:00:00",
"Bitly.com": "",
"Google.com": "2003-01-14T00:00:00",
"Bing.com": "",
"Pubdate tag": "",
"Last Modified": "",
"Archives": [
[
"Earliest",
""
],
[
"By_Archive",
{}
]
],
"Twitter.com": "2006-04-13T02:58:51",
"Backlinks": ""
}
Activity
ibnesayeed commentedon Oct 5, 2016
As far as I remember, there is a
-eflag to exclude modules. This exception is happening because you did not provide Bitly key and did not exclude that module. However, I do understand that it needs better documentation in theREADMEas well as the exception should be caught and gracefully handled with a more friendly message orSTDERR./cc @DarkAngelZT
shawnmjones commentedon Oct 5, 2016
Well, I would agree, but I always get the "please set bitly access token in config" Bitly message, and do not always get this exception.
Instead, as mentioned in the
requests.exceptions.InvalidSchemaexception thrown by the Python requests module, the problem appears to be that something along the way discovered a memento at URIhive.org.uk/wayback/archive/20080304103855/http://www.google.com/and this URI does not have a scheme (i.e., no "http://", "https://", etc.), causing the requests.get on line 85 ofmodules/cdGetArchives.pyto fail.ibnesayeed commentedon Oct 5, 2016
Well, which means some sanity check needs to be placed (and fixed) before making the requests call.