Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated 1Password Dynamic Folder with support for biometrics #71

Merged

Conversation

joachimtingvold
Copy link
Contributor

@joachimtingvold joachimtingvold commented Aug 7, 2023

Based on my original gist, improved by @jvwam by adding support for filtering multiple vaults.

Further refined by me;

  • Add proper readme/notes in the .rdfx file
  • Fixed default value for OS Paths custom properties (will work by-default on macOS, but since Windows path is dependent on the user installation, a valid default path is not always the case here).
  • Fixed issue if "Vaults" custom properties variable was not defined (solved by adding dummy item to vaults list, which is then ignored during iteration). Blank/undefined Vaults variable = use all vaults.
  • Removed unused/old logic and imports (further slimming down the code)
  • Added the pure Python scripts in a separate folder for easy code revision tracking and modification
  • Updated general readme to reflect .rdfx files as valid dynamic folder import files (Royal TS 7+ and Royal TSX 6+)

This solves issue #46 and #60 and supersedes PR #69 (sorry for "hijacking" your PR, @jvwam (= ).

@joachimtingvold
Copy link
Contributor Author

This has been tested & verified on macOS Ventura with Royal TSX 6.0.0.14 and 1Password for Mac 8.10.9 (81009046).

It has not been tested on Windows, so if any Windows-users would test this, that would be excellent.

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Aug 7, 2023

@lemonmojo, you said the version in #69 did not work for you. Could you please try this version? (edit: also note the Windows op CLI tool version requirement due to a bug that @jvwan mentioned; you have to be on at least 2.19.0+).

All your other issues have been addressed, more or less. There is no check if any of the values are wrong (like the OS Path), but at least they have sane default values (OS Path for Windows is based on the specific example path used by 1Password documentation, and for macOS it's the default path when installing via brew).

@joachimtingvold
Copy link
Contributor Author

I removed the old 1Password script as part of the PR, as it's so unsafe that it should not exist in it's current form IMHO. Also updated the notes to reflect that the Royal TS 7+ or Royal TSX 6+ betas are required (due to the new .rdfx format).

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Aug 7, 2023

@lemonmojo, also, the rebuildindex.py probably needs to be modified to account for .rdfx files (as it's currently only looking at .rdfe files (-: ). Also, if the Scripts dir interferes with rebuildindex.py, maybe it should account for that as well? I really do think more complex scripts like the one for 1Password should be as a separate files, and not just embedded in the .rdfx files. Sure, it's way better now than with the old .rdfe format, but, yeah.

@SixFive7
Copy link

SixFive7 commented Nov 1, 2023

@lemonmojo By the excellent work of all parties involved this PR fixes issues #46, #60 and #92 and improves upon PR #69. Can this be merged? And if not, what is the blocker? I have some time to contribute if need be.

@lemonmojo
Copy link
Member

@SixFive7 I can take another look at this over the next few days.
Before I do, can you please answer a couple of questions regarding the new implementation:

  • Does this support multiple accounts across multiple dynamic folders? Can I, for instance have one dynamic folder configured for my work 1P account and one dynamic folder configured to use my private account? And if so, are you certain that reloading one of the two, but then using a dynamic credential from the other returns the data for the correct account, vault and credential?
  • What does the first use experience on a new machine/new user account look like? So let's say I just finished installing the dependencies, but have not configured or even run "bw". How does the script react to that?
  • How is MFA/SSO (ie. TOTP, Yubikey, Duo, Okta, etc.) handled?

Thx,
Felix

@lemonmojo
Copy link
Member

@SixFive7 @joachimtingvold Please also remove the updated readme file from the PR. I just updated it to reflect your changes but it really is off-topic for this PR.
Additionally, please remove the separate py files. While I understand the reasoning behind including them in the PR, the reality is that it puts additional burden on someone wanting to update the script as we now have three sources of truth instead of one.

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Nov 1, 2023

Changes to readme reverted & separate script folder removed.

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Nov 1, 2023

  • Does this support multiple accounts across multiple dynamic folders? Can I, for instance have one dynamic folder configured for my work 1P account and one dynamic folder configured to use my private account? And if so, are you certain that reloading one of the two, but then using a dynamic credential from the other returns the data for the correct account, vault and credential?

It supports multiple accounts accross multiple dynamic folders (it does not, however, support multiple accounts in the same dynamic folder). I use private and work account in separate dynamic folders without any issues. Reloading of credentials are separate for each dynamic folder. As long as you have set the appropriate account ID in each dynamic folder, that should work just fine.

  • What does the first use experience on a new machine/new user account look like? So let's say I just finished installing the dependencies, but have not configured or even run "bw". How does the script react to that?

I assume you mean "op" here (not "bw", which is BitWarden). As for the "op" CLI tool for 1Password, the only requirement for things to work is that the CLI-integration is enabled in 1Password (which is part of the "Getting started with 1Password CLI" documentation that is linked in the "Notes"-section). There should be no need to do any additional logins or other commands. The only "caveat" is that the last logged in account will be the one used unless an account ID is specified in the configuration (which is also mentioned in the "Notes"-section). The latter is only relevant if you are signed into multiple accounts.

  • How is MFA/SSO (ie. TOTP, Yubikey, Duo, Okta, etc.) handled?

Handled seamlessly and automatically by the 1Password app (as long as the CLI tool has been installed according to the instructions).

@lemonmojo
Copy link
Member

@joachimtingvold Sounds promising!

I wasn't aware that the 1Password app was required for this to work. It wasn't required in the previous version of the script but I guess it's ok to have that as a dependency. Maybe it makes sense to clarify in the notes that the 1Password 8 "app" is required for this to work.

But the most important omission is that nowhere in the notes it's mentioned that "Integrate with 1Password CLI" must be enabled in the 1Password app's "Developer" settings. Without this enabled, a reload of the dynamic folder always yields You are not currently signed in. Please run op signin --help for instructions.
So that definitely has to be mentioned prominently in the notes.

One thing that should be improved is error handling when one is not signed in to any account in the 1Password app. Currently, when that is the case and I reload the dynamic folder I get the following error message which is so large that it doesn't even fit on my screen:

An error occurred while executing the dynamic folder's script: Error while getting items: objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000301c400 of class NSConcreteData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000141c000 of class __NSCFString autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000251c380 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x600001310370 of class __NSXPCInterfaceProxy_RemoteProtocol autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x600001e1c580 of class __NSSetI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x1da510bc0) Object 0x6000013103c0 of class __NSSetI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x137004900 of class _NSInlineData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x600001310460 of class __NSXPCInterfaceProxy_RemoteProtocol autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16db87000) Object 0x137104250 of class _NSInlineData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16db87000) Object 0x600001318000 of class __NSXPCInterfaceProxy_RemoteProtocol autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16db87000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16db87000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16db87000) Object 0x1da529550 of class NSData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x600003218030 of class __NSSetI autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x1371044a0 of class _NSInlineData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x6000013180a0 of class __NSXPCInterfaceProxy_RemoteProtocol autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x60000251c1c0 of class NSXPCInterface autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
objc[15633]: MISSING POOLS: (0x16dfe7000) Object 0x1da529550 of class NSData autoreleased with no pool in place - just leaking - break on objc_autoreleaseNoPool() to debug
[ERROR] 2023/11/01 12:53:21 no accounts configured in 1Password app

The same thing happens when cancelling the "1Password Access Requested" prompt.

I guess one option to handle this is to just filter out any lines that have MISSING POOLS in them. Right now though, a user wouldn't know what went wrong because the actual error message is at the very bottom of the output and, at least on my MacBook below the edge of the screen.

Another way to reproduce the "too long error messages" problem is to specify an account or vault that doesn't exist.

Another issue with this new approach is that when I quit the 1Password app, reloading or resolving dynamic credentials results in the following error: [ERROR] 2023/11/01 13:08:09 native messaging: LostConnectionToApp (35 lines of autorelease pool messages omitted 😉)
Interestingly, that error message appears after the "1Password Access Requested" prompt comes up. And it also happens when I just close 1Password's main window instead of actually quitting the app.
The only way I found to work around this is to keep 1Password in the menu bar.

I guess we cannot do anything about this situation since the new script fully relies on the 1Password app being installed and running. But, in that case I suggest at least making sure that any error messages are readable by the user.

I did my tests only on a Mac but all of this (mainly error handling) should be tested on Windows as well.

I'll play with the new version of the script some more and let you know if I find anything else.

@lemonmojo
Copy link
Member

Regarding the "MISSING POOLS" problem: You can work around this by setting the environment variable OBJC_DEBUG_MISSING_POOLS to NO.

Here's an example on how to do this in the get_items method:

env = os.environ.copy()
env["OBJC_DEBUG_MISSING_POOLS"] = "NO"

op = subprocess.Popen(cmd_list_items, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Nov 2, 2023

I wasn't aware that the 1Password app was required for this to work. It wasn't required in the previous version of the script but I guess it's ok to have that as a dependency. Maybe it makes sense to clarify in the notes that the 1Password 8 "app" is required for this to work.

The notes already state that it "Requires 1Password version 8 or above.".

But the most important omission is that nowhere in the notes it's mentioned that "Integrate with 1Password CLI" must be enabled in the 1Password app's "Developer" settings. Without this enabled, a reload of the dynamic folder always yields You are not currently signed in. Please run op signin --help for instructions. So that definitely has to be mentioned prominently in the notes.

The notes mentions that "1Password CLI tool" is required, and links to this url, where the second step of the official CLI tool installation guide is to enable that setting.

One thing that should be improved is error handling when one is not signed in to any account in the 1Password app. Currently, when that is the case and I reload the dynamic folder I get the following error message which is so large that it doesn't even fit on my screen:

While I agree that error handling could probably be improved, the main goal of the rewrite was to address the huge security issues of the old implementation (and secondly to add biometrics).

The only way I found to work around this is to keep 1Password in the menu bar. I guess we cannot do anything about this situation since the new script fully relies on the 1Password app being installed and running.

Yes, the nature of the MFA/Biometrics introduces that dependency. The security benefits far outweigh that constraint IMHO.

@joachimtingvold
Copy link
Contributor Author

Regarding the "MISSING POOLS" problem: You can work around this by setting the environment variable OBJC_DEBUG_MISSING_POOLS to NO.

Thanks. Added to PR.

@lemonmojo
Copy link
Member

@joachimtingvold I'm aware that the notes already say 1Password version 8. I just suggest adding the word "app" so that it's more clear. The same applies to the setting. It's good that it's in 1Password's documentation but it doesn't hurt to also mention it in our documentation since it's so vital for the script to work.

@joachimtingvold
Copy link
Contributor Author

Should be pretty explicitly specified in the notes now (-:

Copy link
Member

@lemonmojo lemonmojo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thx!

@lemonmojo
Copy link
Member

@st9rm1337 Please take a look at this on the Windows side of things. Thx!

@st9rm1337
Copy link
Collaborator

hi @joachimtingvold
I've tried testing the script on a Windows system but am unable to activate the Command-Line Interface (CLI) setting within the 1Password application. I've added the folder containing the op.exe to the PATH system variable, but the CLI option is still greyed out. (https://developer.1password.com/docs/cli/get-started/?utm_medium=organic&utm_source=oph&utm_campaign=windows)
Would you be able to assist in this matter?

@joachimtingvold
Copy link
Contributor Author

joachimtingvold commented Nov 5, 2023

I'm traveling right now, so I don't have any Windows machine readily available at the moment. However, it seems like Windows Hello is required to enable the CLI integration (as per the requirements at the top of the CLI tool install guide);

image

It links to this page for instructions of how to enable that within 1Password. This is also further confirmed by this forum post.

@lemonmojo
Copy link
Member

@joachimtingvold So does that mean that only computers compatible with Windows Hello would be able to use the new version of the 1Password Dynamic Folder?

If that's the case I don't think we can get rid of the current version as it would probably prevent a lot of users from being able to integrate 1Password into Royal TS.

@StefanKoell What's your take on this?

@SixFive7
Copy link

SixFive7 commented Nov 5, 2023

@lemonmojo No it doesn't mean you need a full Windows Hello compatible computer. I understand the confusion given the wording of the ambiguous 1Password documentation. But what they mean to say is you need a non-ancient Windows version. See, they leverage the Windows Hello security subsystem to offload some security and authentication to Windows its secure desktop (secure popups an application cannot interact with). But there is no requirements for the additional login functionality provided by Windows Hello like signing in with biometrics or IR camera's.
I myself use the latest 1Password and CLI without any finger print scanner, camera's or whatnot. Enabling Windows Hello in 1Password just means I authenticate using the standard secure Windows popup (asking for my PIN or whatever I configure for Windows login) instead of 1Password's own login popup.

@lemonmojo
Copy link
Member

Interesting! Thx for the insights @SixFive7!

Now we still have to figure out why the setting is greyed out for @st9rm1337.

@joachimtingvold
Copy link
Contributor Author

The greyed out option should be due to not enabling Windows Hello, as already explained above? Unless @st9rm1337 already tried that to no avail?

@lemonmojo
Copy link
Member

@st9rm1337 Is on vacation this week. We'll continue looking into this when he returns.

@tezgno
Copy link

tezgno commented Nov 20, 2023

Just tried this version. Previous version (2.0.5) works with SSH keys. This version does not appear to work with SSH keys. Also, both versions still (new and old) still generate the following error when trying to load a "Connection with Options" SFTP:

An error occurred while executing the action 'Authenticate'. Details: String cannot have zero length.
Parameter name: userName

@tezgno
Copy link

tezgno commented Nov 20, 2023

I was able to modify this script to work with SSH keys. Not sure if you want me to make changes to this file and commit for review or pass it up, but it a simple add in on that. Still having the string cannot have zero length issue with Parameter name: userName. Not sure what's driving that error.

@tezgno
Copy link

tezgno commented Nov 20, 2023

Update, the "Parameter userName" issue appears to be a bug in Royal TSX. If I turn on "Prompt for Credentials" on the Connect Using menu, and select the credential, it works as normal. It only fails if that option isn't selected. So likely not a issue with the dynamic folders but how Royal TSX handles that option.

@lemonmojo
Copy link
Member

@tezgno There's currently no specific handling for private key files in this version of the 1Password Script. The issue here is likely caused by the fact that your private key item in 1Password has either no username specified or that the username field is not named in a way that we expect.
Since the whole topic is not related to the discussion in this thread, please open either a new support ticket or a new Github issue for this. Thx!

@st9rm1337
Copy link
Collaborator

joachimtingvold

Hi, just tested the script whilst also activating Windows Hello within the Security settings of the 1Password application. After doing so, I was also able to activate the 1PW CLI Integration, which also let me reload the script.
Thanks for the heads-up, I think we're good here!

@tezgno
Copy link

tezgno commented Nov 21, 2023

@tezgno There's currently no specific handling for private key files in this version of the 1Password Script. The issue here is likely caused by the fact that your private key item in 1Password has either no username specified or that the username field is not named in a way that we expect. Since the whole topic is not related to the discussion in this thread, please open either a new support ticket or a new Github issue for this. Thx!

I fixed it myself by updating the script here. Unlike the existing 1password script, this version doesn’t pass the private key at all (those lines of code are completely missing). Once I added in to pass the private key, it worked just as the previous version of the script. I can share the copy I made.

@joachimtingvold
Copy link
Contributor Author

I see what you're talking about. Not sure why I initially removed those lines of code (it was done at the very beginning when I was slimming the old code, and I probably confused it with something else). I've re-added them now, should probably work again now?

@tezgno
Copy link

tezgno commented Nov 22, 2023

Yep. Working again.

@lemonmojo lemonmojo merged commit 3a6af8b into royalapplications:master Nov 28, 2023
@lemonmojo
Copy link
Member

This has been merged now. Thx for everyone involved!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants