An idea to support 2FA (Two factor authentication) for Android users

[gmcmicken]

Okay so I was going to hack in a text message 2FA like here, however it would be nice to have this natively supported so I looked into android / google's web messages system messages.google.com/web, and it looks like we could do this not so difficultly by using the chrome driver to load up messages.google.com/web , save the screenshot and output the file location to the user to open, then they would use their phone to "pair" the device (IBeam) with the QR code. This would keep working for quite some time through many logins, each new login attempt will trigger an sms code and the webdriver would load conversations and get the latest one automatically.

My pseudo code would be

 if element exists 2fa:
     if device not paired:
         driver.get(web sms url) driver.save_screenshot
         output log filepath
         wait for pair
         set device paired true
     else
         driver.get(web sms url)
         traverse conversation list, look for "your authentication code"
    set form 2fa = code

[gmcmicken]

even better way to get the QR to the user, on web messages the QR data value is in the html element so can create a URL that goes to a QR generator online with this data.

[Voyz]

Hey @gmcmicken! 👋 Thanks for your contribution of suggesting this workaround for the 2FA.

It's a plausible idea, although as you note it would work for Android devices more than for Apple users. Furthermore, wouldn't it require for the receiving phone to always be active and on network? Eg. if you'd head off camping up in the mountains for a week your system would stop receiving sms' and would not complete 2FA. To be fair though, I never used Google messages so I don't know what are the exact requirements. Do you have any experience with it?

In either case, from what I'm seeing it would be reasonable to start by providing IBeam with a callback that would be called when 2FA would be requested, so that we could all implement the 2FA code acquisition individually depending on our setup and requirements. For instance IBeam could accept a TWO_FA_URL environment variable that each user could set to whatever they would use (eg. a remote server, another docker container, a virtual machine, etc.) and leave the implementation details up to an individual for now. IBeam would then attempt to call that URL with a request for 2FA code and would paste back whatever that request returns. How would you feel about this?

[gmcmicken]

Hey @Voyz I totally get what you're saying and that would be my first reaction if I were the developer of this project - however here's why I think it might be worthwhile considering anyway. (and I submitted a PR for this implementation for you to look at)

1. New IB accounts (such as mine) require 2FA, and it can't be turned off.
2. The IB api gateway should stay logged in for 24 hours, but it doesn't. I did extensive testing overnight each day last week and it always broke after about 3 hours.
3. It's basically impossible to use the IB api gateway without iBeam (and integrated 2FA) unless I want to wake up at 6am every day to log in manually then every 3 hours throughout the day.

#11

[Voyz]

Wow @gmcmicken that's a fantastic contribution! 👏🙌 I greatly appreciate your time you put into developing and submitting this, it puts us all on the right path to solving the 2FA problem.

I'd totally introduce the changes you proposed, although I feel it would be useful to nonetheless abstract the way the 2FA code is being provided. So that everything between lines 201 and 234 in your PR would be abstracted in some way that other users could override with their own method - should they for instance use the method described in #8 using twilio - as well as for potential alternatives natively supported in IBeam in the future.

Making this a passed parameter would ease testing and overriding this method, eg:

def authenticate_gateway(driver_path, account, password, key: str = None, base_url: str = None, accessor_2fa : callable = None) -> bool:
    ...
    if two_factor_el and accessor_2fa is not None:
        code_2fa = accessor_2fa()
        two_factor_el[0].send_keys( code_2fa )

Then we'd define the accessor you propose as ibeam/accessors_2fa/google_msg_accessor.py:

def google_msg_2fa_accessor():
    _LOGGER.debug(f'2FA in use: Loading messages.google.com/web')
    driver_2fa = new_chrome_driver(driver_path)
    ...
    return re.search( r'(\d+)', sms_list_el[0].text ).group(1)

Then we'd call it like this:

import google_msg_accessor
_ACCESSOR_2FA = os.environ.get('IBEAM_ACCESSOR_2FA', 'Two-factor Authentication accessor you want to use.')
...
if _ACCESSOR_2FA  == 'google_msg':
    accessor_2fa = google_msg_accessor.google_msg_2fa_accessor
elif _ACCESSOR_2FA == 'some_other_type':
    accessor_2fa = some_other_accessor.some_other_2fa_accessor
else:
    accessor_2fa = None

authenticate_gateway(self.driver_path, 
    self.account, 
    self.password, 
    self.key, 
    self.base_url, 
    accessor_2fa)

Something along these lines, although I'm just hypothesising here - we could figure out how to make using other some_other_2fa_accessor more reasonable too. What do you recon?

In either case, once again thanks for your input - very useful! 😊

[gmcmicken]

Yea that's a great way to do it, clean and extensible. I only submitted PR so you could see my implementation. I'm not a python developer admittingly! The code does work though and I'm running it now overnight to see if it retains authentication. Fingers crossed!

If you're willing to package this up as you described I would be a very happy camper. And if you need me to write more code I can try doing that too. Thank you.

[Voyz]

Ah sure, great! Well keep me in the loop and let's see how the night goes for your system. If this indeed will have some promising results then we should totally introduce it in some form or another. If you'd feel like contributing more then you can give it a shot yourself although frankly your addition is already a great step forward, so I can totally take care of implementing it.

Also, can you just confirm - does Google Messages require the phone to be active at any point the SMS is to be acquired? Just to see what expectations we could set for the users.

[gmcmicken]

Thanks I think you'd do a better job implementing your suggestion than I would - but I'll keep you posted on my testing and be available for you.

Yes Google Messages requires your phone to actually receive the SMS message, so your edge-case about being off camping without reception would not be covered unfortunately.

[gmcmicken]

I should note there is also an advantage in having the SMS go to your phone - is that you don't need to purchase another number to support 2FA and if you use your credentials to log in manually for whatever things you're doing outside of iBeam, then it's nice for the verification to come to your main phone.

[Voyz]

Sure thing I can take care of it no problem 👍 Thanks for that clarification, it will be important to make this clear in the docs too.

You're making a fair point on why this method has advantages - thanks for listing these too.

Staying tuned for the update tomorrow then. Cheers! 😊

[gmcmicken]

So I found two bugs, first I forgotdriver_2fa.quit() after I was done, second chrome needed a data dir to store cookies, options.add_argument('--user-data-dir=/tmp/ibeam-chrome')

[Voyz]

Thanks for following up with that. Was the system successful otherwise or did these two bugs break it last night?

[gmcmicken]

They broke it, I've been running it today with the fixes, so far so good!

[typeless]

For iOS users, there is a built-in app called Shortcuts which may be of help.

[Voyz]

Thanks for suggesting that app @typeless! Would you know how we could build its usage to IBeam? From what I saw its a task-runner type of an app, so I guess we'd still need some sort of a server that would pass the message?

[typeless]

@Voyz
One can create a shortcut to Run Script Over SSH that is triggered by receiving a message (the 2-factor auth code). I think it can call a webhook as well, but I am not sure how it should be done.

I assume that the authentication can be automated by a curl script. When the trading bot detects 401, it can log in to the gateway with the preset user/password, which would in turn have the IB backend send the security code to the phone.
When the phone receives the SMS, it triggers the shortcut to run the script over SSH to submit the security code.

This approach offers a different trade-off. It doesn't require a separate web service such as Twilio, and most importantly doesn't require me to change the phone number registered at IB. IIRC, IB doesn't allow a 2-factor-auth-dedicated phone number alongside the "real-person" phone number. That would cause potential problems if IB needs to reach me with the number.

[gmcmicken]

Apple has a similar function to Google web messages, it's called text message forwarding, and it sends SMS messages to iCloud for use in the apple messaging app for Macs. Could be a similar implementation although I'm not sure if they have a web client.

[Voyz]

Thanks for suggesting both of these solutions guys! Unfortunately, I don't own an iPhone so I wouldn't be able to lead development on this implementation. If either of you (or anyone in the future) would be interested in introducing a new native 2FA handler for iPhones then I'd be more than happy to help you get there