-
-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't fetch password from system #219
Comments
Exactly the same error here on El Capitan (10.11.4). Python 2.7 running in a virtualenv. What does "enable all applications to use the password from the keychain" mean (from @rgardam comment)? Config root and data root are both reported as Any suggestions? [Edit:] When I tried running keyring outside the virtualenv, it works as expected. This is a virtualenv issue. However, the config and data roots are still being reported as above, and still contain nothing from keyring. |
Keyring doesn't store any passwords in the config or data root for the OS X backend, but instead stores the passwords in the default keychain. After adding a password, you should see changes in the As for the error message reported or the issues with virtualenv, I'm unable to replicate them.
Before deleting the password, I could find it in Keychain Access. I did not have to do anything special like /enable all applications to use the password from the keychain/. I'm using OS X El Capitan and Keyring 9.0 on Python 2.7.10. Can you provide any additional information on what's different about your environment that's different from mine? |
Until I have more information, I don't know what else I can do to help resolve the issue. I do have a clean Mac OS X VM, so if there are steps I can take in that VM to replicate the issue, I'm willing to try. I'm closing the issue for now, but I'm happy to re-open it as new information comes to light. |
I am getting this error with Python2.7 when using a virtualenv (tox). It works okay with Python2.7 if the venv is deactivated. I am not getting the error in Python3.4 or Python3.5 in the venv (again, tox) -- they seem to work okay. Strange. EDIT: Also on OS X 10.11(.5) EDIT: For each password request, I'm seeing in Console.app (no such message for python3):
EDIT: I'm getting it with my pyenv installed 2.7.11: mkdir tmp
cd tmp
virtualenv -p python2 venv
venv/bin/pip install keyring
venv/bin/python2 -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))' As well as OS X's built-in 2.7.10: mkdir tmp
cd tmp
virtualenv -p /usr/bin/python venv
venv/bin/pip install keyring
venv/bin/python -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))' But running fine with my pyenv 3.5.1: mkdir tmp
cd tmp
virtualenv -p python3 venv
venv/bin/pip install keyring
venv/bin/python -c 'import keyring; print(keyring.get_password("Some Service", "Some Account"))' |
Data point: this is affecting our program too (https://github.com/grow/grow) and I can confirm it occurs when run in a virtualenv. I can also confirm that downgrading keyring to 8.7 seems to have fixed the issue. Here's the relevant traceback:
|
Sorry, but I'm still unable to replicate the issue. I created a fresh install of OS X in a virtual machine. Python didn't even have pip installed. This machine is stock. Yet, when I install all the prereqs, keyring works as advertised without any issue:
It sure seems like whatever is triggering this API error is not universal (and is specific to only some users' systems). Perhaps there's a security setting enabled or a legacy settings lingering from an older build of the OS. I still am aware of no actions that keyring can take to work around the issue, short of bringing back the command-based invocation. I'd consider that, if it's a separate package and not in the default implementation. I'd prefer, though, if we could understand the underlying cause of the error and correct for it, so any details you might be able to provide as to why your systems are failing and a stock system is not would be most helpful in advancing this issue. |
Slight update: If I request a password that doesn't exist (by changing a letter of either the service or account), I get |
@jaraco thanks for working through this with us. I just reproduced the error inside virtualenv following your commands exactly (minus the initial ensurepip). See below. Let me know if there's anything else you'd like me to try. Here's my OS version, if it helps:
|
Worked with
|
I get this error too after I upgraded to OS X 10.11.5 today, which may or may not have to do with the error. I am not sure. Quoting @n8henrie :
And I will not get an error (exception) if I "allow all application to access this item" in OS X Keychain Access.app |
I second @jeremydw . Version 8.7 works. Version 8.7 invokes the shell command |
I ran into this as well on 10.11.5 and keyring 9.0, the system's python.app would pop up the dialog asking for permission for the keyring item correctly, but a python in a virtualenv would not. Adding the allow-access for the virtualenv python also did not help. I can set new keys with the virtualenv binary but not read them back out. Moving to a 'security' exec'ing version works as expected, in this case 5.4. |
I'm still working to get this figured out. To that end, I've made a test keychain credential I'm still getting error 25293 / errSecAuthFailed under Python 3.5.2, but I'm able to retrieve the test password under 2.7.12. I've simplified the new ctypes OSX API to the following test script: import ctypes
sec_path = "/System/Library/Frameworks/Security.framework/Versions/A/Security"
kc = ctypes.CDLL(sec_path).SecKeychainFindGenericPassword
kc.argtypes = (
ctypes.c_void_p,
ctypes.c_uint32,
ctypes.c_char_p,
ctypes.c_uint32,
ctypes.c_char_p,
ctypes.POINTER(ctypes.c_uint32), # passwordLength
ctypes.POINTER(ctypes.c_void_p), # passwordData
ctypes.POINTER(ctypes.c_void_p), # itemRef
)
length = ctypes.c_uint32()
data = ctypes.c_void_p()
kc(None, len("keyring_test"), "keyring_test".encode("utf-8"), len("foo"),
"foo".encode("utf-8"), length, data, None,)
password = ctypes.create_string_buffer(length.value)
ctypes.memmove(password, data.value, length.value)
print(password.raw.decode("utf-8")) Interestingly, I found that I do get the popup to "allow" and subsequently can access it under the following Python versions (with keyring 9.3.1):
This is really confusing at this point. Specifically,
It clearly doesn't seem to be a problem with Python, since 3.5.2 from pyenv works, as well as 3.5.2 from homebrew in a virtualenv installed virtualenv (but not a pyvenv installed virtualenv). Any other ideas? |
Perhaps there's something about the security settings on the Python 3.5.2 installed by homebrew, something that might be getting copied to the pyvenv. That's quite interesting, though. It sounds like you're on to something, and this really gives us an opportunity to perhaps reproduce the issue, so I'll give it another go. |
I've installed Python on homebrew:
And added the credential as you did:
I then created a virtualenv.
I then saved the script you created as test-keychain.py and ran it with success:
It did prompt me to Always Allow, Allow, or Deny the request, and I clicked allow. I wiped and recreated
So it seems that there's still something unique about your environment that I can't replicate. I haven't done anything special with this machine - it's the same stock machine I referenced above, and I installed homebrew from https://brew.sh today with no special steps. May I suggest you try the same? Create an OS X virtual machine using Virtualbox and an installer image. Then, see if you can replicate the failure in that clean environment. I suspect you cannot, at which point you might then compare the security settings and file system permissions against that machine and the failing system to perhaps suss out the cause. |
I give up finding out why somehow my environment (the python that comes along with Xcode, I guess) complains However, I have got a
... and yes, it works:
My guess is that Keychain Access.app identifies and distinguishes apps by "runtime libraries" --- the output of
My hack (and virtual environments, as mentioned above) works because I effectively used another runtime library to get passwords from Keychain Access.app (after clicking "Allow" or "Always Allow"). So, all in all, IMHO this is not a bug per se, but as a result of (unintendedly) having messed up system-wise configurations and OS X no longer pops the "Do you want xxx to access your keychains?" dialog. |
Thanks @jialongl . Sorry to hear you didn't have a proper fix, but you seem to have gotten awfully close to the underlying cause. If anyone else is able to trace a solution to the underlying problem, perhaps by clearing a .properties file or similar, do please report back here. It would be great to be able to provide a better error message. In the meantime, I don't see any pending action, so I'm closing this ticket, but anyone is welcome to revive the conversation. |
I got access to another Macbook Air on El Capitan and installed 3.5.2 with homebrew -- keyring worked appropriately with or without a venv, no issues. So it doesn't look like a homebrew problem necessarily either. On my personal MBA I've tried uninstalling, cleaning the homebrew cache, updating to the latest formula, and reinstalling, and still no dice. ^^ Sorry, thought I had submitted that a few days ago before I left on a little camping trip, came back to find it still in the comment box, unsubmitted. Sounds fair, I'll come back to the thread if I ever figure this out. Thanks for the consideration and recommendations. |
It's also working for me with a fresh Virtualbox installation of homebrew's 3.5.2. Still trying to figure out what's going on in my system (I have a number of python3 scripts that rely on keyring, which is why I'm being so persistent about this). I found that homebrew was leaving Selecting the option to allow access to all apps does let keyring access the password. Once it's changed back, it's back to the same error. I created a new keychain So looks my problem isn't intrinsic to the keychain, either. |
Something seems to have fixed it in in Homebrew/homebrew-core@84897ff . |
I use python 2.7.10 (provided with mac) and the problem persists |
@tommyblue If you get the error message Python wants to use your confidential information stored in "KEYRING_NAME" in your keychain The authenticity of "Python" cannot be verified. Do you want to allow access to this item? and clicked Deny Thus, my point is, your system is somehow configured to deny this request automatically for you (though without your awareness / intention). |
I also got this problem with a fresh install of keyring on High Sierra today. Only in the venv.
So I tried checking out the signature of the venv python:
Re-signing it fixed it:
|
I encountered this issue today. At one moment, keyring was working just fine. Less than two hours later, it spontaneously started failing with
I was able to add a signature for it:
Yet somehow still the keyring continues to fail with the same error. |
@jaraco Do you intend to investigate this issue? |
I was facing the same issue on macOS Mojave after changing password, I fixed it by doing by following this jdamata/k8vault#3 (comment) Not sure why this fixed it though |
@rohithrpai: I did investigate, but what I found was that the problem went away on its own. I was working with my internal IT department as the issue was isolated to my work macbook. The behavior was very strange. I installed Python 3.8b4 and the keyring seemed to be working there, but not on 3.7.4. Here are some excerpts from my interaction then. 2019-10-07Okay, this is bizarre. Today, I installed Python 3.7.5rc1 over 3.7.4 and I made one other change--for the foo/bar test credential, I with to Get Info and Access Control and configured it to "Allow all applications to access this item", and I was able to query it. I then went back to the original credential that was giving me trouble and I was able to access it without the error. It seems that mysteriously, whatever was causing the problem the week before last has gone away as spontaneously as it occurred. 2019-10-10Even more bizarre, I'm finding now that the Python 3.8rc1 is no longer signed and is now being rejected by the keychain calls but Python 3.7 (still unsigned) is not rejected by the keychain. 2019-10-11And as I encounter even more weirdness, I realize I'm simply out of my depth. After struggling for most of a day with the issue, I made several surprising findings:
So I'm very confused. I don't understand what caused the access to stop nor why reinstalling a later Python allowed the access to be restored. I'm beginning to think there's some executable whitelist and that executable whitelist only includes a few select recent Python versions. StatusI'm currently not experiencing the issue in any Python environment on any system, but when I was, the issue was elusive and inscrutable. The behavior would seemingly change without any explicit alteration to the system. I just don't know what else to do. |
Note, I continue to use keyring on my macOS systems without any signed Python executable. |
@jaraco : This issue is happing on my mac, I am using python 3.7, keyring==18.0.1, with same setup in other Mac this is working fine.
|
I've just hit that. After removing the profile, I still had to delete my login keychain and then reset it (which would create a new login keychain). (Thankfully, I didn't have anything important in my keychain). Now it's working again. |
Yesterday, this problem mysteriously and spontaneously re-emerged on my workstation. Steps I tried unsuccessfully:
|
I've confirmed that while keyring is failing under Python 3.8.1, it's succeeding under Python 3.7.5rc1, and neither are signed:
|
Since Python 3.7.5 was an RC, I decided to update it to 3.7.6. After upgrading it to 3.7.6, the issue now manifests in Python 3.7 as well, leading me to believe there's some setting in the system that's preventing new apps from being authorized to the keychain. |
I should have mentioned - this environment is using a keychain that's not the default keychain. It's using a separate file and setting I've also observed that when logging in, I was previously prompted to enter the password to unlock the keychain (for some macOS application), but macOS has not been prompting me there either. This leads me to believe that applications other than keyring are affected (and suppressing the error when querying the keyring). |
@satyagv557 The error you're getting is different from mine. I encountered a similar error |
I've found some references to the |
Issue fixed, sometime even your keychain is unlocked, keyring is not able to grab the password, I did added a shell build in my pipeline to unlock in runtime, that did solve this issue, thanks for your comments and suggestions. |
A few other things I tried today to no avail:
I did find that while Python 3.8 and 3.7 from python.org are currently failing, Python 3.7.6 from homebrew is able to query the keychain. |
So here's something interesting: I rebooted the system. After logging in, when fmfd and Reminders prompted for the password on this password file, instead of entering the password, I went to keyring to query the keychain. When I did, the Python routine blocked. So I entered the password for fmfd, then was prompted for the password for Reminders, then unexpectedly I was prompted for the password for Python. After entering the password for that keychain, Python was unblocked but still emitted the -25293 Security Auth Failure. |
I extracted some "python" logs from the Console when the error occurs and here's what I see:
|
Okay, so I've found something that seems to have worked:
(no output as the error did not occur) And yet the
It seems that What I really don't understand is why these binaries are apparently signed but not with a valid signature and only sometimes. |
We had a user receive The application is using python 3.7 installed from homebrew. Locking and unlocking the login keychain solved the issue. This leads me to believe the codesigning error is a complete red herring. |
Just some more info that might help, I've discovered that if there is no entry in MacOS Keychain for my app then I do not receive the Update: Discovered that forcing a resign of the PyInstaller created executable fixes the Keychain codesign issue: |
It isn't immediately clear to me if this issue has been fixed. But we are still experiencing this crash with keyring version 23.0.1 We have included your keyring library in Cura 4.9.0 (https://github.com/Ultimaker/Cura/blob/master/cura/OAuth2/KeyringAttribute.py) to store our auth tokens. But now that Cura 4.9.0 is released in the wild, I noticed multiple crashes coming in on our Sentry server. With users that are experiencing this crash: This only on occurs for the Mac's Darwin OS versions: 17.7.0, 18.7.0, 19.6.0, 20.2.0, 20.3.0, 20.4.0 |
@jellespijker From keyring's perspective, there's little that can be done. The keyring library just invokes Apple's APIs under whatever security regime that application (be it Python or some bundler like cx_freeze or pyinstaller) was built. Apple then sometimes rejects requests with opaque error codes. Of one thing you can be pretty confident, if you see that error message, then Apple rejected the security API call with one of two codes: keyring/keyring/backends/macOS/api.py Lines 42 to 47 in bfa0148
The best recommendation I can make is to ensure the executable image is signed and if you've already done that, maybe consider breaking down the problem into smaller pieces:
To be sure, though, I use macOS daily and haven't encountered this issue in a long time, so I consider it solved. If you have concerns that involve new and different factors, please consider distilling the concerns and then opening a new issue. |
@jaraco thanks for the reply. Our MacOS executable is signed by Apple. Our developers and testers did not encounter the issue running from source or image. Luckily there aren't any issues reported by our users on GitHub. Just multiple Sentry crash reports. Which could be an indication that it is a single, non-recuring event for an individual user. We will keep an eye out and if we find the root-cause report it here. |
Here is a stack trace, although it probably doesn't give much more information. A bunch of it is filtered out as personally identifiable information of our users: https://sentry.io/share/issue/cadfe0c6dee14b5d8e5dd119e22407a8/ It's occurred for several hundred people, but not for any of our developers or testers so I think it's safe to say that it's computer-dependent. On average each user has encountered it 3-4 times as far as we can see (we can't identify individual users) so it's recurring for them too. We can't reproduce the issue locally, so we can't verify the other two questions. It really seems to be depending on the configuration on the computer. A configuration that we don't have. We're seeing it in all MacOS versions Cura supports (at least most of them from what I can see) so it's not a specific MacOS version. It must be something else in the configuration. Our application is signed by Apple with an enterprise account, and also notarized (which unpacks and repacks the binaries). |
That's consistent with what I've seen in some situations too - that the behavior is system-dependent and somewhat transient. One thing I'd like to do is provide more context in the error so that it's apparent which error code is actually returned. |
Oh, I see the actual code is already included in the error message, and it's -25293 ( So unfortunately, that's the full context that Apple gives us in this situation. Perhaps you could reach out to Apple (or its community) to see if they can shed any light onto what factors might cause this error or what more diagnoses could be performed to gather more insight. Perhaps the systems in question somehow have corrupt signing keys (so even though the apps are properly signed, they appear unsigned on those systems). |
I am seeing this error on Yosemite (10.10.5 (14F27))
keyring.backends._OS_X_API.Error: (-25293, "Can't fetch password from system")
edit: it works if I enable all applications to use the password from the keychain.
The text was updated successfully, but these errors were encountered: