Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installing flownet #79

Closed
tayloris opened this issue Jun 27, 2020 · 5 comments
Closed

Installing flownet #79

tayloris opened this issue Jun 27, 2020 · 5 comments

Comments

@tayloris
Copy link
Collaborator

tayloris commented Jun 27, 2020 •
edited
Loading

I'm installing flownet in a new Ubuntu 20.04.

There a problem in build_enviroment.sh, I got the following error message

Collecting pip
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl
ERROR: Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))

Every line with
pip install <package>
has to be modified by
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package>

This will solves the first error but there is a new problem in the pip calls inside ERT

Finished installing libres
Collecting git+https://github.com/equinor/ert@c74e1e6
  Cloning https://github.com/equinor/ert (to revision c74e1e6) to /tmp/pip-req-build-k17vq1n3
  Running command git clone -q https://github.com/equinor/ert /tmp/pip-req-build-k17vq1n3
  WARNING: Did not find branch or tag 'c74e1e6', assuming revision or ref.
  Running command git checkout -q c74e1e6
    ERROR: Command errored out with exit status 1:
     command: /home/manuel/repos/projects/Flownet/flownet/venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k17vq1n3/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-k17vq1n3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-2jm5uwsd
         cwd: /tmp/pip-req-build-k17vq1n3/
    Complete output (113 lines):
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)'))': /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl
    ERROR: Exception:
    Traceback (most recent call last):
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 665, in urlopen
        httplib_response = self._make_request(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 376, in _make_request
        self._validate_conn(conn)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 994, in _validate_conn
        conn.connect()
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connection.py", line 352, in connect
        self.sock = ssl_wrap_socket(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/util/ssl_.py", line 370, in ssl_wrap_socket
        return context.wrap_socket(sock, server_hostname=server_hostname)
      File "/usr/lib/python3.8/ssl.py", line 500, in wrap_socket
        return self.sslsocket_class._create(
      File "/usr/lib/python3.8/ssl.py", line 1040, in _create
        self.do_handshake()
      File "/usr/lib/python3.8/ssl.py", line 1309, in do_handshake
        self._sslobj.do_handshake()
    ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/adapters.py", line 439, in send
        resp = conn.urlopen(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
        return self.urlopen(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
        return self.urlopen(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 747, in urlopen
        return self.urlopen(
      [Previous line repeated 2 more times]
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/connectionpool.py", line 719, in urlopen
        retries = retries.increment(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/urllib3/util/retry.py", line 436, in increment
        raise MaxRetryError(_pool, url, error or ResponseError(cause))
    pip._vendor.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 188, in _main
        status = self.run(options, args)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 185, in wrapper
        return func(self, options, args)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/commands/wheel.py", line 159, in run
        requirement_set = resolver.resolve(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 179, in resolve
        discovered_reqs.extend(self._resolve_one(requirement_set, req))
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 362, in _resolve_one
        abstract_dist = self._get_abstract_dist_for(req_to_install)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/resolution/legacy/resolver.py", line 314, in _get_abstract_dist_for
        abstract_dist = self.preparer.prepare_linked_requirement(req)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 467, in prepare_linked_requirement
        local_file = unpack_url(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 255, in unpack_url
        file = get_http_url(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 129, in get_http_url
        from_path, content_type = _download_http_url(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 277, in _download_http_url
        download = downloader(link)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/download.py", line 189, in __call__
        resp = _http_get_download(self._session, link)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/download.py", line 135, in _http_get_download
        resp = session.get(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 543, in get
        return self.request('GET', url, **kwargs)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_internal/network/session.py", line 421, in request
        return super(PipSession, self).request(method, url, *args, **kwargs)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 530, in request
        resp = self.send(prep, **send_kwargs)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/sessions.py", line 643, in send
        r = adapter.send(request, **kwargs)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/cachecontrol/adapter.py", line 53, in send
        resp = super(CacheControlAdapter, self).send(request, **kw)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pip/_vendor/requests/adapters.py", line 514, in send
        raise SSLError(e, request=request)
    pip._vendor.requests.exceptions.SSLError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/16/45/81b5262c0efc08882bdf183b788e6d28e3d684863990996d8b60967d48da/pytest_runner-5.2-py2.py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:1108)')))
    Traceback (most recent call last):
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/installer.py", line 128, in fetch_build_egg
        subprocess.check_call(cmd)
      File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/home/manuel/repos/projects/Flownet/flownet/venv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpyupahj5v', '--quiet', 'pytest-runner']' returned non-zero exit status 2.
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-k17vq1n3/setup.py", line 18, in <module>
        setup(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/dist.py", line 718, in fetch_build_eggs
        resolved_dists = pkg_resources.working_set.resolve(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 780, in resolve
        dist = best[req.key] = env.best_match(
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/dist.py", line 777, in fetch_build_egg
        return fetch_build_egg(self, req)
      File "/home/manuel/repos/projects/Flownet/flownet/venv/lib/python3.8/site-packages/setuptools/installer.py", line 130, in fetch_build_egg
        raise DistutilsError(str(e))
    distutils.errors.DistutilsError: Command '['/home/manuel/repos/projects/Flownet/flownet/venv/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpyupahj5v', '--quiet', 'pytest-runner']' returned non-zero exit status 2.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
@tayloris
Copy link
Collaborator Author

tayloris commented Jun 27, 2020 •
edited
Loading

As far I see, the SSL Error is on the pip installations. I can solve the problem for the installation of LIBECL, LIBRES but I done manage to install ERT. I downloaded the ERT repository and change the line that call pip install ... to the corresponding pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package>, but that didn't work. There should be a simple solution but I have not found it yet.

pypa/pip#5288

@anders-kiaer
Copy link
Collaborator

anders-kiaer commented Jun 28, 2020 •
edited
Loading

This is of course unrelated to FlowNet, but I guess the reason is that somewhere (maybe in openssl), security level has recently been tightened, without everything in the Python pip world following up (as of now).

This question is probably related: https://askubuntu.com/questions/1233186/ubuntu-20-04-how-to-set-lower-ssl-security-level

Note that I would guess trusted-host basically bypasses https / SSL verification, and therefore opening up for man-in-the-middle attacks, and thus not a recommended solution.

What is the pip and openssl versions?

@tayloris
Copy link
Collaborator Author

tayloris commented Jul 1, 2020

The versions are: pip 20.1.1, and OpenSSL 1.1.1f 31 Mar 2020

@tayloris
Copy link
Collaborator Author

tayloris commented Jul 1, 2020

It happens as well on a new installation of Ubuntu 18.04 :( . It is strange because it works on my laptop that is Ubuntu 18.04 but not in the PC :(.

@anders-kiaer
Copy link
Collaborator

Not sure what is the easiest, and still secure, way of solving this. A search on the Internet indicates that this is a quite new mismatch between pip and OpenSSL, and will probably be solved externally at some point.

I guess we close this issue here since it is unrelated to FlowNet, but we can still continue the discussion in the closed issue if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tayloris @anders-kiaer