New constant occurrence of 'Invalid username password combination'

[zeeb0tt]

Describe the bug
I've been using voyz/ibeam for some time now. Suddenly as of this morning, it reports an invalid username password combination. I can login to IB directly with exactly the credentials provided (copy + paste) but voyz/ibeam will say this at all times now. It had been working flawlessly every day for months.

Although I did notice when I logged in via the web UI @ IB that the URL looked a little different the first time around.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'command prompt'
2. Enter 'docker run --env IBEAM_ACCOUNT=user --env IBEAM_PASSWORD=pass -p 5000:5000 voyz/ibeam'
3. See error

Expected behavior
Typically the authentication is successful and 2FA is triggered.

Environment
IBeam version: 0.4.3
Docker image or standalone: docker
Python version (standalone users only):
OS: Windows 10 Home

[stanj98]

Hi, I tried it two days back and it works. I did come across your issue as well but on 16th Sep. (Friday), it was working again. It could be an issue from IBKR's end. I will try again on Monday and update.

[Voyz]

Hey @zeeb0tt thanks for outlining your issue in detail 👍

Unfortunately, this is a problem that we're aware of and it seems to be on the side of IBKR - you can see similar problems appearing in other issues in this repository. Hence, I'd encourage you to report it to IBKR support and try to solve it with them. There hasn't been any updates to IBeam around the time you're reporting.

Thanks for chipping in @stanj98 👍

[Raulcr93]

Hi, I have the same problem. Regardless of the configuration, it shows me the same "Invalid username password combination" message. The only difference I see in my authentication is that now it asks me to confirm my login from the IBKR app with a floating notification on my mobile. When I run the gateway directly it works for me without any problem, I put in my credentials, I get the mobile notification, I approve it and I access it correctly. For this reason, I think the problem is the notification that reaches the mobile, I don't know if this is contemplated in ibeam. Greetings.

[Voyz]

Hey @Raulcr93 thanks for chipping in. The mobile auth and notification is also handled by IBKR not IBeam, hence once again I'd suggest addressing it with their support.

Other users have also observed that things are different when the Gateway is started locally, however there hasn't been any pattern observed yet that could suggest why there is a difference. Feel free to propose a solution if you can think of something though 👍

[ckcollab]

Also experiencing this problemo -- has anyone resolved this by contacting IBKR? They seemed to be unhelpful, but maybe I'm not "saying the right thing" or "asking the right question?"

Appreciate all the work on this, thank you!

[kadnan]

@Voyz

People saying it is at IBKR's end. I am new and uncertain but just want to tell you that the same user/Pwd works well without any issue if I try on the TWS terminal. Also, not all CSS/Javascript files are loading. Image attached.

[Voyz]

hey @ckcollab I'm sorry, but this has been an ongoing issue since I've started working on IBeam, and it happens with different severity for some users, while not at all for others. I'd love to help more, but IBeam only serves as a thin layer on top of the Gateway. Unless we can show that the credentials are scrambled somehow when they're input - all we do is copy paste them into two fields -, there's little we can do other than trying to address this with IBKR

[Voyz]

@kadnan that same username/password should work again on your IBeam instance on some other attempt. TWS is a separate trading platform altogether, so there it could work when it fails using the Gateway.

Also I've been seeing the issue with the page not loading. IBeam only just loads whatever the Gateway will serve it, if there's some Javascript/HTML/rendering issues then that's again on IBKR side. I'd suggest you highlight this issue with them. I've found this loading issue appear when the connection is particularly slow.

[ckcollab]

@Voyz it's okay! Ty for all the hard work

Things are working ok now, in my case I believe I was IP banned from too many attempts at one point, changing my IP resolved the issue!

[Voyz]

That's very interesting @ckcollab! What makes you think so? This would be very useful for other users to know!

[ckcollab]

When I would go to the IB docs pages they would say "Access Denied" until I turned on my VPN/changed IP :)

[Voyz]

@ckcollab you're right! Using a VPN does help! I changed my VPN to UK.

[ckcollab]

Glad to help! @Voyz

[Wenuka]

Hi,
I am using the IBeam for some time now without any issue. But since a few weeks, I am getting the ######## ATTENTION! ######## Maximum number of failed authentication attempts (IBEAM_MAX_FAILED_AUTH=5) reached. error. Then I have updated the docker image to voyz/ibeam:0.4.5. Then it started working as expected. But, after a few days, I got the same error. When I login manually using https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on it works. Also, when I restart the IBeam, it works again as expected. But after a few days I am getting the same error again.

Here is the most recent logs that I got:

2023-06-17 03:41:11,980|I| Authentication process succeeded
2023-06-17 03:41:15,804|E| Gateway session active but not authenticated
2023-06-17 03:41:15,804|I| Logging out and restarting the Gateway
2023-06-17 03:41:15,939|I| Gateway logout successful
2023-06-17 03:41:16,076|I| Gateway session found but not authenticated, authenticating...
2023-06-17 03:41:16,076|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
2023-06-17 03:41:23,900|I| Gateway auth webpage loaded
2023-06-17 03:41:23,900|I| Login attempt number 1
2023-06-17 03:41:29,278|I| Submitting the form
2023-06-17 03:41:30,486|I| Webpage displayed "Client login succeeds"
2023-06-17 03:41:32,487|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f469236bed0> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="a6608ce90ac7d05b2ff1cb4a617773c8")>
2023-06-17 03:41:32,556|I| Authentication process succeeded
2023-06-17 03:41:35,693|E| Gateway session active but not authenticated
2023-06-17 03:41:35,693|I| Logging out and restarting the Gateway
2023-06-17 03:41:35,827|I| Gateway logout successful
2023-06-17 03:41:35,961|I| Gateway session found but not authenticated, authenticating...
2023-06-17 03:41:35,962|I| Loading auth webpage at https://localhost:5000/sso/Login?forwardTo=22&RL=1&ip2loc=on
2023-06-17 03:41:43,832|I| Gateway auth webpage loaded
2023-06-17 03:41:43,832|I| Login attempt number 1
2023-06-17 03:41:49,233|I| Submitting the form
2023-06-17 03:41:50,441|E| Error displayed by the login webpage: Invalid username password combination
2023-06-17 03:41:51,485|I| Login attempt number 2
Execution of job "GatewayClient._maintenance (trigger: interval[0:01:00], next run at: 2023-06-17 03:41:52 UTC)" skipped: maximum number of running instances reached (1)
2023-06-17 03:41:56,878|I| Submitting the form
2023-06-17 03:41:57,261|E| Error displayed by the login webpage: Invalid username password combination
2023-06-17 03:41:58,306|I| Login attempt number 3
2023-06-17 03:42:03,717|I| Submitting the form
2023-06-17 03:42:04,093|E| Error displayed by the login webpage: Invalid username password combination
2023-06-17 03:42:05,125|I| Login attempt number 4
2023-06-17 03:42:10,485|I| Submitting the form
2023-06-17 03:42:10,911|E| Error displayed by the login webpage: Invalid username password combination
2023-06-17 03:42:11,973|I| Login attempt number 5
2023-06-17 03:42:17,349|I| Submitting the form
2023-06-17 03:42:17,679|E| Error displayed by the login webpage: Invalid username password combination
2023-06-17 03:42:17,720|C| ######## ATTENTION! ######## Maximum number of failed authentication attempts (IBEAM_MAX_FAILED_AUTH=5) reached. IBeam will shut down to prevent an account lock-out. It is recommended you attempt to authenticate manually in order to reset the counter. Read the execution logs and report issues at https://github.com/Voyz/ibeam/issues
2023-06-17 03:42:17,720|I| Cleaning up the resources. Display: <pyvirtualdisplay.display.Display object at 0x7f469239c5d0> | Driver: <selenium.webdriver.chrome.webdriver.WebDriver (session="c486d9a4f9c532bda08ceebb9e7c3199")>
2023-06-17 03:42:17,797|I| Authentication process failed
2023-06-17 03:42:17,797|W| Shutting IBeam down due to critical error.

I also believe this is an issue with the IBKR side.

But, from what I think, if there is a way to automatically restart the docker container after a pre-defined time (say after a few hours), we can bypass this issue. Or as a second suggestion, we can wait longer if we see more and more Invalid username password combination. For example wait 1 minute during the first attempt, 5 minutes in the second and then 30 minutes, 60 minutes and 120 minutes subsequently. With this, if the error happens due to an issue of IBKR server update/ error, we could bypass it by waiting longer.

Or please let me know if there is another way to fix this.

Thank you.

[Voyz]

Hey @Wenuka - thanks for sharing your observations 👍 Honestly though, I really couldn't comment much on this. The errors seem quite sporadic, and there's little to draw conclusions from.

Now as for your points:

1. Restarting Docker can be done - you need to take care of it in your configuration, that is something outside of IBeam itself.
2. The increase interval of 1-5-30-60min is not a bad idea, although it is speculative as to whether it will actually help. I currently don't have the resources to implement it, but feel free to submit a PR if you'd give it a shot.

I honestly wish I had some other ideas on how to handle these issues, but there isn't any indication that could help us tackle it. The best thing we can do is to let IBKR know of these problems, so make sure you highlight it with their support.

[Voyz]

Hey guys, just wanted to share that I've noticed this error happening if the process of logging in happens too fast. I used a selenium script to auto authenticate with the Gateway (a side project, other than IBeam), and if it just fills the creds out and instantly submits, the 'Invalid username password combination' occurs. Something to think about. We already have a 5-second buffer there, but possibly there's something else we could do here:

            ...
            user_name_el.send_keys(account)

            if key is None:
                password_el.send_keys(password)
            else:
                password_el.send_keys(Fernet(key).decrypt(password.encode('utf-8')).decode("utf-8"))

            password_el.send_keys(Keys.TAB)

            # small buffer to prevent race-condition on client side
            time.sleep(5)
            # submit the form
            _LOGGER.info('Submitting the form')
            submit_form_el = driver.find_element(By.CSS_SELECTOR, elements['SUBMIT_EL'])
            submit_form_el.click()
            ...

I tried seeing if it would run some request upon typing the creds, but it doesn't seem to. I wonder if it runs some JavaScript code when you do that, and maybe it doesn't finish by the time we submit.

I decided to add a functionality where we'd increase that buffer timeout on subsequent occurances of this error. There's two new env vars that define the starting and maximum waiting time: IBEAM_MIN_PRESUBMIT_BUFFER and IBEAM_MAX_PRESUBMIT_BUFFER. I've pushed voyz/ibeam:0.4.6-rc5 with this addition.

[Wenuka]

Hey @Wenuka - thanks for sharing your observations 👍 Honestly though, I really couldn't comment much on this. The errors seem quite sporadic, and there's little to draw conclusions from.

Now as for your points:

1. Restarting Docker can be done - you need to take care of it in your configuration, that is something outside of IBeam itself.
2. The increase interval of 1-5-30-60min is not a bad idea, although it is speculative as to whether it will actually help. I currently don't have the resources to implement it, but feel free to submit a PR if you'd give it a shot.

I honestly wish I had some other ideas on how to handle these issues, but there isn't any indication that could help us tackle it. The best thing we can do is to let IBKR know of these problems, so make sure you highlight it with their support.

Thanks @Voyz for your quick response and I agree.

Just now I managed to add the second suggestion (waiting increase interval of 1-5-30-60min).

Incase some one else is interested, all what I did was to add the following in the ibeam/src/authenticate.py in the line 407 (just above _FAILED_ATTEMPTS += 1 inside the function authenticate_gateway),

                    wait_secs = 15 * 60 * (2**_FAILED_ATTEMPTS)
                    _LOGGER.info(f"waiting {wait_secs/60:.0f} minutes before retrying...")
                    time.sleep(wait_secs)

This waits 15-30-60-120 minutes as for me it is okay. But you can modify it as you want.

I will update you incase this solves the problem in a few weeks.

[Voyz]

hey @zeeb0tt @Wenuka @ckcollab and others - there is a new WIP version v0.5.0 of IBeam that introduces a new login logic, which should help reduce some of the issues we've been struggling with.

Run the most recent voyz/ibeam:0.5.0-rc* tag of v0.5.0 and set IBEAM_AUTHENTICATION_STRATEGY to B.

See #147 for more details and report any observations directly there.

Thanks 🙏

[Voyz]

I'm going to close this issue due to inactivity. Thanks for your contribution and please feel free to request a reopen if you'd like to continue the discussion 👍