Checking service status returns TypeError

[samoturk]

Doing:

from chembl_webresource_client import *
targets = TargetResource()
targets.status()

Returns:

TypeError                                 Traceback (most recent call last)
<ipython-input-8-fdc433ae91b9> in <module>()
----> 1 targets.status()

/usr/lib/python2.7/site-packages/chembl_webresource_client/web_resource.pyc in status(self)
     89     def status(self):
     90         service = self.get_service()
---> 91         if not 'status' in service:
     92             return False
     93         return service['status'] == 'UP'

TypeError: argument of type 'NoneType' is not iterable

If I check https://www.ebi.ac.uk/chemblws/status/ I get "UP". So ChEMBL seems to be working..

[mnowotka]

Hi Samo,
I've just tried execute the same code from my machine and I can't reproduce the problem.

I have some suggestions:

1. Can you delete chembl_webresource_client.sqlite from the current repository and try again?
2. Can you check if earlier in your code you are not setting any custom values to settings:

from chembl_webresource_client.settings import Settings
Settings.Instance().WEBSERVICE_DOMAIN = 'www.haha.wont.work.com'

1. Can you provide version of the library? (pip freeze | grep chembl_webresource_client)

2. Can you provide your settings?

   s = Settings.Instance()
   print s.WEBSERVICE_PROTOCOL
   print s.WEBSERVICE_DOMAIN
   print s.WEBSERVICE_PREFIX
   print s.CACHING
   print s.FAST_SAVE
   print s.CONCURRENT_SIZE
   print s.ASYNC_TRESHOLD
   print s.CACHE_NAME
   print s.RESPECT_RATE_LIMIT
   print s.TIMEOUT
   print s.UTILS_SPORE_URL
   print s.PROXIES

3. Apart from the status (you can skip this part as this only checks if everything is OK) does the rest of your calls (i.e. retrieving target data) work?

[samoturk]

Hi Michał,

• deleted, doesn' help

• didn't set any custom values

• I pulled latest version from git (0.7)

• Settings:

  https
  www.ebi.ac.uk
  /chemblws
  True
  True
  50
  10
  chembl_webresource_client
  True
  3.0
  https://www.ebi.ac.uk/chembl/api/utils/spore
  None

• retrieving data doesn't seem to work. I just get None (and I used targetID from you demo)

[mnowotka]

OK. Looking at the stack trace it seems that get_service function (https://github.com/chembl/chembl_webresource_client/blob/master/chembl_webresource_client/web_resource.py#L61) returns None which happens when requests return status code different than 200 or the exception is thrown.

Can you do:

import requests
res = requests.get('https://www.ebi.ac.uk/chemblws/status/')
print (res.ok == True)

If this prints True, debugging this problem would require cloning the library from git, only then we will be able to modify get_service function and investigate why it fails. This issue, make me think about debugging, I will crate new enhancement issues which will hopefully make it easier to debug in future.

[mnowotka]

Sorry Samo, there is one more thing you can do!
Can you please run this code, and check the output:

from chembl_webresource_client import TargetResource

class TargetDebugResource(TargetResource):
    def get_service(self):
        url = '%s/status/' % Settings.Instance().webservice_root_url
        print 'retrieving %url' % url
        res = requests.get(url, timeout=Settings.Instance().TIMEOUT)
        if not res.ok:
            print "not OK, %s" % res.status_code
            return None
        js = res.json()
        if not 'service' in js:
            return False
        return js['service']

targets = TargetDebugResource()
targets.status()

[samoturk]

It's getting strange.
If I try:

import requests
res = requests.get('https://www.ebi.ac.uk/chemblws/status/')
print (res.ok == True)

It returns True. But if I from chembl_webresource_client import * and rerun last two lines from above box I get:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-b7beff07589f> in <module>()
----> 1 res = requests.get('https://www.ebi.ac.uk/chemblws/status/')
      2 
      3 print (res.ok == True)

/usr/lib/python2.7/site-packages/requests/api.pyc in get(url, **kwargs)
     58 
     59     kwargs.setdefault('allow_redirects', True)
---> 60     return request('get', url, **kwargs)
     61 
     62 

/usr/lib/python2.7/site-packages/requests/api.pyc in request(method, url, **kwargs)
     47 
     48     session = sessions.Session()
---> 49     return session.request(method=method, url=url, **kwargs)
     50 
     51 

/usr/lib/python2.7/site-packages/requests_cache/core.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert)
    112             auth, timeout,
    113             allow_redirects, proxies,
--> 114             hooks, stream, verify, cert
    115         )
    116         if self._is_cache_disabled:

/usr/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    455         }
    456         send_kwargs.update(settings)
--> 457         resp = self.send(prep, **send_kwargs)
    458 
    459         return resp

/usr/lib/python2.7/site-packages/requests_cache/core.pyc in send(self, request, **kwargs)
     89         response, timestamp = self.cache.get_response_and_time(cache_key)
     90         if response is None:
---> 91             return send_request_and_cache_response()
     92 
     93         if self._cache_expire_after is not None:

/usr/lib/python2.7/site-packages/requests_cache/core.pyc in send_request_and_cache_response()
     81 
     82         def send_request_and_cache_response():
---> 83             response = super(CachedSession, self).send(request, **kwargs)
     84             if response.status_code in self._cache_allowable_codes:
     85                 self.cache.save_response(cache_key, response)

/usr/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, **kwargs)
    567 
    568         # Send the request
--> 569         r = adapter.send(request, **kwargs)
    570 
    571         # Total elapsed time of the request (approximately)

/usr/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
    360                     decode_content=False,
    361                     retries=Retry(self.max_retries, read=False),
--> 362                     timeout=timeout
    363                 )
    364 

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.pyc in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, **response_kw)
    514             httplib_response = self._make_request(conn, method, url,
    515                                                   timeout=timeout,
--> 516                                                   body=body, headers=headers)
    517 
    518             # If we're going to release the connection in ``finally:``, then

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.pyc in _make_request(self, conn, method, url, timeout, **httplib_request_kw)
    302 
    303         # Trigger any extra validation we need to do.
--> 304         self._validate_conn(conn)
    305 
    306         # conn.request() calls httplib.*.request, not the method in

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.pyc in _validate_conn(self, conn)
    720         # Force connect early to allow us to validate the connection.
    721         if not getattr(conn, 'sock', None):  # AppEngine might not have  `.sock`
--> 722             conn.connect()
    723 
    724         if not conn.is_verified:

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.pyc in connect(self)
    227                                     ca_certs=self.ca_certs,
    228                                     server_hostname=hostname,
--> 229                                     ssl_version=resolved_ssl_version)
    230 
    231         if self.assert_fingerprint:

/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.pyc in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version)
    121             context.load_cert_chain(certfile, keyfile)
    122         if HAS_SNI:  # Platform-specific: OpenSSL with enabled SNI
--> 123             return context.wrap_socket(sock, server_hostname=server_hostname)
    124         return context.wrap_socket(sock)
    125 

/usr/lib/python2.7/ssl.pyc in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname)
    348                          suppress_ragged_eofs=suppress_ragged_eofs,
    349                          server_hostname=server_hostname,
--> 350                          _context=self)
    351 
    352     def set_npn_protocols(self, npn_protocols):

TypeError: __init__() got an unexpected keyword argument 'server_hostname'

Same error appears if I run the code from your last suggestion. I downgraded urllib3 and requests libraries to an earlier versions (that worked) and I still have the same problem.

[mnowotka]

OK, so apparently the problem is with python version - "urllib3/gevent became incompatible with the latest python 2.7.8 version":

searx/searx#120

I have Python 2.7.6 and it works fine, can you check yours?

[samoturk]

You are corect, it was the python itself! I had 2.7.9 and after downgrading to 2.7.8 everything works normally.
PS. Thanks for your help!