Error while accessing from Linux env

[patilnikhilh80]

Issue:

I have a python application which uses domain_tools api. My dev environment is on my windows PC and it work fine but when I deploy it to linux server it give following error

Traceback (most recent call last):
File "/home/myuser/myapplication/lib/work/work.py", line 98, in run
execute_func(msg)
File "/home/myuser/myapplication/lib/work/fraud/mywork.py", line 49, in execute
riskscore = json.loads(str(self.domaintools_client.risk(url)))
File "/home/myuser/myapplication/mylibrary/myclient/domaintools_client.py", line 24, in risk
return self.api.risk(domain)
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/api.py", line 254, in risk
return self._results('risk', '/v1/risk', items_path=('components',), domain=domain, cls=Reputation,
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/api.py", line 66, in _results
self._rate_limit()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/api.py", line 60, in _rate_limit
for product in self.account_information():
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 197, in iter
return self._items().iter()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 180, in _items
response = self.response()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 161, in response
response = self.data()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 92, in data
results = self._get_results()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 74, in _get_results
data = self._make_request()
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/domaintools/base_results.py", line 68, in _make_request
return session.get(url=self.url, params=self.kwargs, verify=self.api.verify_ssl,
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/requests/sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/requests/sessions.py", line 636, in send
kwargs.setdefault('proxies', self.rebuild_proxies(request, self.proxies))
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/requests/sessions.py", line 305, in rebuild_proxies
username, password = get_auth_from_url(new_proxies[scheme])
File "/home/myuser/myapplication/venv/lib64/python3.8/site-packages/requests/utils.py", line 948, in get_auth_from_url
parsed = urlparse(url)
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/urllib/parse.py", line 372, in urlparse
url, scheme, _coerce_result = _coerce_args(url, scheme)
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/urllib/parse.py", line 124, in _coerce_args
return _decode_args(args) + (_encode_result,)
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/urllib/parse.py", line 108, in _decode_args
return tuple(x.decode(encoding, errors) if x else '' for x in args)
File "/opt/rh/rh-python38/root/usr/lib64/python3.8/urllib/parse.py", line 108, in
return tuple(x.decode(encoding, errors) if x else '' for x in args)
AttributeError: 'dict' object has no attribute 'decode'
'dict' object has no attribute 'decode'

For same input the Windows gives correct o/p while Linux gives error.
If anyone has any suggestions, it will be a great help

[patilnikhilh80]

if anyone can help
@ChuckWoodraska @jmeridth @timothycrosley

[jmeridth]

@patilnikhilh80 Can you please provide the code snippet you're using that works on windows but fails on Linux? My first thought is a path issue. But your snippet will help confirm my suspicion.

Please note that I don't work for DomainTools anymore but willing to help.

[patilnikhilh80]

Line where I get error in mywork.py
riskscore = json.loads(str(self.domaintools_client.risk(url)))
domaintools_client is

import domaintools
import requests
class DomainToolsClient:

    def __init__(self, api_name, api_key, proxy=None):
        self.api_name = api_name
        self.api_key = api_key
        self.proxy = None
        self.api = domaintools.API(self.api_name, self.api_key)
        if proxy:
            self.proxy = {"https_proxy": proxy}
            self.api = domaintools.API(self.api_name, self.api_key, proxy_url=proxy, verify_ssl=False)

    def investigate(self, domain):
        params = {"api_username": self.api_name,
                  "api_key": self.api_key}
        res = requests.get(f"https://api.domaintools.com/v1/iris-investigate/{domain}", params=params,
                           proxies=self.proxy)
        return res.json()

    def risk(self, domain):
        return self.api.risk(domain)

@jmeridth

[patilnikhilh80]

This issue was caused because of improper proxy settings.
Generally we set proxies as dictionary, but here we need to set it as string,
Ex. Generally
proxy = {'http': 'something', 'https': 'something'}In this case
proxy = 'something'

It too so long to identify its a proxy issue due to improper error handling in package. Difficult to find out from following error.

AttributeError: 'dict' object has no attribute 'decode'
'dict' object has no attribute 'decode'

maybe we can improve the error msg to understand it more easily. Like invalid proxy or something.

Can I open PR, seems don't have access though.

[jmeridth]

@patilnikhilh80 nice catch.

You can fork the repository and issue a pull request from there. You won't have access to create a branch on this repo directly.

[mccarthym]

Thank you for the comment @patilnikhilh80, per @ChuckWoodraska's comment above, we've tried to improve our messaging for issues such as this.