ValueError in urllib3 when authenticating #153

bellerofonte · 2023-07-14T11:21:39Z

Describe the bug
Authentication fails with error:

File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/util/timeout.py", line 156, in _validate_timeout
    raise ValueError(
ValueError: Timeout value connect was <object object at 0x7f84696d49c0>, but it must be an int, float or None.

Full output

2023-07-14 10:56:27,418|I| ############ Starting IBeam version 0.4.6 ############
2023-07-14 10:56:27,420|I| Secrets source: env
2023-07-14 10:56:27,421|I| Health server started at port=5001
2023-07-14 10:56:27,421|I| Environment variable configuration:
{'INPUTS_DIR': '/srv/inputs/', 'OUTPUTS_DIR': '.', 'GATEWAY_DIR': '/home/pgm/clientportal.gw', 'CHROME_DRIVER_PATH': '/opt/chromedriver', 'GATEWAY_STARTUP': 20, 'GATEWAY_PROCESS_MATCH': 'ibgroup.web.core.clientportal.gw.GatewayStart', 'MAINTENANCE_INTERVAL': 60, 'SPAWN_NEW_PROCESSES': False, 'LOG_LEVEL': 'INFO', 'LOG_TO_FILE': True, 'LOG_FORMAT': '%(asctime)s|%(levelname)-.1s| %(message)s', 'REQUEST_RETRIES': 1, 'REQUEST_TIMEOUT': 15, 'RESTART_FAILED_SESSIONS': True, 'RESTART_WAIT': 15, 'REAUTHENTICATE_WAIT': 15, 'IBEAM_HEALTH_SERVER_PORT': 5001, 'GATEWAY_BASE_URL': 'http://localhost:5123', 'ROUTE_AUTH': '/sso/Login?forwardTo=22&RL=1&ip2loc=on', 'ROUTE_USER': '/v1/api/one/user', 'ROUTE_VALIDATE': '/v1/portal/sso/validate', 'ROUTE_REAUTHENTICATE': '/v1/portal/iserver/reauthenticate?force=true', 'ROUTE_AUTH_STATUS': '/v1/api/iserver/auth/status', 'ROUTE_TICKLE': '/v1/api/tickle', 'ROUTE_LOGOUT': '/v1/api/logout', 'USER_NAME_EL': None, 'PASSWORD_EL': 'password', 'SUBMIT_EL': 'button.btn.btn-lg.btn-primary', 'ERROR_EL': None, 'SUCCESS_EL_TEXT': 'Client login succeeds', 'OAUTH_TIMEOUT': 15, 'PAGE_LOAD_TIMEOUT': 15, 'ERROR_SCREENSHOTS': False, 'MAX_FAILED_AUTH': 5, 'MIN_PRESUBMIT_BUFFER': 5, 'MAX_PRESUBMIT_BUFFER': 30, 'MAX_IMMEDIATE_ATTEMPTS': 10, 'IBKEY_PROMO_EL_CLASS': 'ibkey-promo-skip', 'TWO_FA_EL_ID': 'twofactbase', 'TWO_FA_NOTIFICATION_EL': 'login-step-notification', 'TWO_FA_INPUT_EL_ID': 'chlginput', 'TWO_FA_HANDLER': None, 'STRICT_TWO_FA_CODE': True, 'TWO_FA_SELECT_EL_ID': 'sf_select', 'TWO_FA_SELECT_TARGET': 'IB Key'}
2023-07-14 10:56:27,450|I| Gateway not found, starting new one...
2023-07-14 10:56:27,450|I| Note that the Gateway log below may display "Open https://localhost:5000 to login" - ignore this command.
2023-07-14 10:56:27,450|I| Starting Gateway as Linux process with params: ['bash', 'bin/run.sh', 'root/conf.yaml']
running  
 runtime path : root:dist/ibgroup.web.core.iblink.router.clientportal.gw.jar:build/lib/runtime/*
 config file  : root/conf.yaml
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.netty.util.internal.ReflectionUtil (file:/home/pgm/clientportal.gw/build/lib/runtime/netty-common-4.1.15.Final.jar) to constructor java.nio.DirectByteBuffer(long,int)
WARNING: Please consider reporting this to the maintainers of io.netty.util.internal.ReflectionUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
 -> mount demo on /demo
Java Version: 11.0.19
****************************************************
version: a27ed42161ad96c53e715ca5c5e3e3fa4cff5262 Mon, 24 Apr 2023 15:41:53 -0400
****************************************************
This is the Client Portal Gateway
for any issues, please contact api@ibkr.com
and include a copy of your logs
****************************************************
https://www.interactivebrokers.com/api/doc.html
****************************************************
Open http://localhost:5123 to login
App demo is available after you login under: http://localhost:5123/demo#/
2023-07-14 10:56:28,489|I| Gateway started with pid: 3397449
2023-07-14 10:56:29,180|I| Gateway connection established
2023-07-14 10:56:29,429|I| No active sessions, logging in...
2023-07-14 10:56:29,430|I| Loading auth webpage at http://localhost:5123/sso/Login?forwardTo=22&RL=1&ip2loc=on
2023-07-14 10:56:30,497|E| Error encountered during authentication
Traceback (most recent call last):
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/ibeam/src/authenticate.py", line 253, in authenticate_gateway
    driver = start_driver(base_url, driver_path)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/ibeam/src/authenticate.py", line 467, in start_driver
    driver = new_chrome_driver(driver_path)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/ibeam/src/authenticate.py", line 66, in new_chrome_driver
    driver = webdriver.Chrome(driver_path, options=options)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/chrome/webdriver.py", line 76, in __init__
    RemoteWebDriver.__init__(
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 319, in execute
    response = self.command_executor.execute(driver_command, params)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/remote/remote_connection.py", line 374, in execute
    return self._request(command_info[0], url, body=data)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/selenium/webdriver/remote/remote_connection.py", line 397, in _request
    resp = self._conn.request(method, url, body=body, headers=headers)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/_request_methods.py", line 118, in request
    return self.request_encode_body(
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/_request_methods.py", line 217, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/poolmanager.py", line 432, in urlopen
    conn = self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/poolmanager.py", line 303, in connection_from_host
    return self.connection_from_context(request_context)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/poolmanager.py", line 328, in connection_from_context
    return self.connection_from_pool_key(pool_key, request_context=request_context)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/poolmanager.py", line 351, in connection_from_pool_key
    pool = self._new_pool(scheme, host, port, request_context=request_context)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/poolmanager.py", line 265, in _new_pool
    return pool_cls(host, port, **request_context)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/connectionpool.py", line 196, in __init__
    timeout = Timeout.from_float(timeout)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/util/timeout.py", line 190, in from_float
    return Timeout(read=timeout, connect=timeout)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/util/timeout.py", line 119, in __init__
    self._connect = self._validate_timeout(connect, "connect")
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/urllib3/util/timeout.py", line 156, in _validate_timeout
    raise ValueError(
ValueError: Timeout value connect was <object object at 0x7f84696d49c0>, but it must be an int, float or None.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/pgm/.conda/envs/ibkr/lib/python3.10/site-packages/ibeam/src/authenticate.py", line 447, in authenticate_gateway
    raise RuntimeError('Error encountered during authentication') from e
RuntimeError: Error encountered during authentication
2023-07-14 10:56:30,500|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f8466cf41c0> | Driver: None
2023-07-14 10:56:30,546|I| Authentication process failed
2023-07-14 10:56:30,550|I| Starting maintenance with interval 60 seconds

To Reproduce
Steps to reproduce the behavior:

Download and unpack IB clientportal gateway
Create new conda (miniforge) env, python=3.10
install IBeam and chromedriver
run ibeam_starter.py
See error

Expected behavior
Success authentication

Environment
IBeam version: 0.4.6
Docker image or standalone: standalone
Python version (standalone users only): 3.10.12
OS: Ubuntu 22.04.2 LTS

Additional context
urllib3 version: 2.0.3
selenium version: 3.141.0

Suggest a Fix
ibeam requires selenium version 3.* but it seems that issue is fixed in version 4.9.0
⁉️ possibly outdated setup.py

The text was updated successfully, but these errors were encountered:

Voyz · 2023-07-14T15:58:20Z

Hey @bellerofonte thanks for sharing your issue in detail 👍

This seems to be related to selenium/chrome/chromedriver versions. Indeed, thanks for bringing the setup.py to my attention, it is outdated. The requirements.txt has the updated packages.

You can see more information about this issue under #140, particularly: #140 (comment)

Let me know if updating the versions helps solving the issue

bellerofonte · 2023-07-15T08:21:16Z

Hi @Voyz! Thank you for your effort!
After some tries I gave up trying to run standalone version due to different issues related to the dependencies.
However, I successfully launched IB GW using docker image 0.5.0-rc2:

docker run \
    --env IBEAM_ACCOUNT=****** \
    --env IBEAM_PASSWORD=****** \
    --env IBEAM_AUTHENTICATION_STRATEGY=B \
    -p 5000:5000 voyz/ibeam:0.5.0-rc2

Voyz · 2023-07-16T15:37:52Z

happy to hear you got it working otherwise @bellerofonte 👍 if you'd like to, feel free to share your troubles with the standalone version.

bellerofonte · 2023-07-20T12:27:51Z

Hi @Voyz !
Sorry for a late response.

Here is the issue I am facing to when I run ibeam standalone:

2023-07-20 12:19:26,201|I| Gateway started with pid: 3604566
2023-07-20 12:19:26,967|I| Gateway connection established
2023-07-20 12:19:27,240|I| No active sessions, logging in...
2023-07-20 12:19:27,241|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
2023-07-20 12:19:28,433|E| Error encountered during authentication
Traceback (most recent call last):
  File "/home/pgm/Projects/ibeam/ibeam/src/authenticate.py", line 253, in authenticate_gateway
    driver = start_driver(base_url, driver_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pgm/Projects/ibeam/ibeam/src/authenticate.py", line 479, in start_driver
    raise e
  File "/home/pgm/Projects/ibeam/ibeam/src/authenticate.py", line 467, in start_driver
    driver = new_chrome_driver(driver_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pgm/Projects/ibeam/ibeam/src/authenticate.py", line 66, in new_chrome_driver
    driver = webdriver.Chrome(driver_path, options=options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
    super().__init__(
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
    super().__init__(
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
    self.start_session(capabilities, browser_profile)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "/home/pgm/.conda/envs/ibkr/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary

Chrome driver is installed at /opt/chromedriver

I run ibeam via following command:

IBEAM_GATEWAY_DIR=/home/pgm/clientportal.gw \
IBEAM_CHROME_DRIVER_PATH=/opt/chromedriver \
IBEAM_OUTPUTS_DIR=. \
IBEAM_ACCOUNT=****** \
IBEAM_PASSWORD=****** \
IBEAM_AUTHENTICATION_STRATEGY=B \
python3 ibeam_starter.py

I have also tried IBEAM_CHROME_DRIVER_PATH=/opt/ but no luck either.
I have tried to download lates release zip and to checkout the repository - same error in both cases.
May be I have to install some package via apt?

Voyz · 2023-07-20T12:52:04Z

Thanks 👍

What's the path to your chrome?
What's the chrome version?
What's the chromedriver version?

See this: https://stackoverflow.com/questions/50138615/webdriverexception-unknown-error-cannot-find-chrome-binary-error-with-selenium

Do you have /usr/bin/google-chrome present?

bellerofonte · 2023-07-20T13:06:57Z

🤦‍♂️

I've just installed chrome via latest stable .deb... and it works!

Thank you!

Voyz · 2023-07-20T13:08:25Z

Way to go! Congrats @bellerofonte 👍 So the problem was that you didn't have chrome installed or that it was outdated?

bellerofonte · 2023-07-20T13:11:02Z

yep, I didn't have chrome installed

Voyz · 2023-07-20T13:12:11Z

Cool, glad that was the problem, and well done figuring it out! 👏

Voyz · 2023-09-07T18:44:51Z

Hey @bellerofonte feel free to close this issue now 👍

bellerofonte added the bug Something isn't working label Jul 14, 2023

Voyz self-assigned this Jul 14, 2023

bellerofonte closed this as completed Sep 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ValueError in urllib3 when authenticating #153

ValueError in urllib3 when authenticating #153

bellerofonte commented Jul 14, 2023 •

edited

Loading

Voyz commented Jul 14, 2023

bellerofonte commented Jul 15, 2023

Voyz commented Jul 16, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

Voyz commented Sep 7, 2023

ValueError in urllib3 when authenticating #153

ValueError in urllib3 when authenticating #153

Comments

bellerofonte commented Jul 14, 2023 • edited Loading

Voyz commented Jul 14, 2023

bellerofonte commented Jul 15, 2023

Voyz commented Jul 16, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

bellerofonte commented Jul 20, 2023

Voyz commented Jul 20, 2023

Voyz commented Sep 7, 2023

bellerofonte commented Jul 14, 2023 •

edited

Loading