Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test_datalad_credential_helper test wants interaction #6623

Closed
yarikoptic opened this issue Apr 9, 2022 · 4 comments · Fixed by #7209
Closed

test_datalad_credential_helper test wants interaction #6623

yarikoptic opened this issue Apr 9, 2022 · 4 comments · Fixed by #7209
Labels
DX developer experience

Comments

@yarikoptic
Copy link
Member

while building debian package (unattended), next day spotted that it wants to talk to me

datalad.local.tests.test_gitcredential.test_datalad_credential_helper ... Please set a password for your new keyring: 
Please confirm the password: 
Error: blank passwords aren't allowed.
Please set a password for your new keyring:

shouldn't happen, and any needed for that settings (if any) changes should be done in setup_package fixture or prevented via mocking in the test.
@yarikoptic yarikoptic added the DX developer experience label Apr 9, 2022
@yarikoptic yarikoptic mentioned this issue Apr 9, 2022
Merged
3 tasks
@TheChymera
Copy link
Contributor

Just ran into the exact same issue running the tests for 0.16.1

datalad.local.tests.test_gitcredential.test_datalad_credential_helper ... Please set a password for your new keyring:

@bpoldrack
Copy link
Member

FTR: Didn't look closely yet, but it's likely GIT_ASKPASS (or the config core.askpass) that need to be set to not let Git ask interactively. Which is currently done in our CI setups. I see the argument to move this to setup_package.

@yarikoptic yarikoptic mentioned this issue Jul 8, 2022
Merged
@yarikoptic
Copy link
Member Author

seting GIT_ASKPASS to true is of no effect:

root@smaug:/build/datalad-0.17.0/.pybuild/cpython3_3.9_datalad/build# GIT_ASKPASS=true PYTHONPATH=$PWD PATH=/build/datalad-0.17.0/bin:$PATH python3 -m pytest -s -v datalad/local/tests/test_gitcredential.py::test_datalad_credential_helper
It is highly recommended to configure Git before using DataLad. Set both 'user.name' and 'user.email' configuration variables.
======================================================================================================================= test session starts =======================================================================================================================
platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /build/datalad-0.17.0, configfile: tox.ini
collected 1 item                                                                                                                                                                                                                                                  

datalad/local/tests/test_gitcredential.py::test_datalad_credential_helper create(ok): . (dataset)
Please set a password for your new keyring:

and we seems to set it already in fixtures:

$> git grep ASKPASS
datalad/conftest.py:            # variable GIT_ASKPASS overwrites this, but neither env var nor this config
datalad/conftest.py:        m.setenv('GIT_ASKPASS', 'true')
datalad/local/gitcredential.py:        # TODO we could prevent prompting by setting GIT_ASKPASS=true
datalad/local/gitcredential.py:        # with GIT_ASKPASS=true would just be an empty string
datalad/tests/utils.py:        # variable GIT_ASKPASS overwrites this, but neither env var nor this config
datalad/tests/utils.py:    set_envvar('GIT_ASKPASS', 'true')

since it is asking to set password for a new keyring it is really just trying to initiate the entire credentials keyring...

debug level output 
[DEBUG  ] Run ['git', 'credential', 'approve'] (cwd=/tmp/datalad_temp_test_datalad_credential_helperibyppzvj) 
[Level 8] Process 40635 started 
[Level 5] STDIN: git credential appro (Thread<(STDIN: git credential appro, 7)>) started 
[Level 5] STDIN: git credential appro (Thread<(STDIN: git credential appro, 7)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] Instantiating ssh manager 
[Level 5] Done importing main __init__ 
[Level 5] Importing support.network 
[Level 5] Done importing support.network 
[Level 5] Requested to provide version for cmd:git 
[DEBUG] Run ['git', 'version'] (cwd=None) 
[Level 8] Process 40698 started 
[Level 5] STDERR: git version (Thread<(STDERR: git version, 5)>) started 
[Level 5] STDERR: git version (Thread<(STDERR: git version, 5)>) exiting (exit_requested: False, last data: None) 
[Level 5] STDOUT: git version (Thread<(STDOUT: git version, 3)>) started 
[Level 5] Read 19 bytes from 40698[stdout] 
[Level 5] STDOUT: git version (Thread<(STDOUT: git version, 3)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] process_waiter (Thread<process_waiter>) exiting 
[Level 8] Process 40698 exited with return code 0 
[DEBUG] Finished ['git', 'version'] with status 0 
[Level 8] No module named 'datalad_deprecated' [gitrepo.py:<module>:3744] 
[DEBUG] Not retro-fitting GitRepo with deprecated symbols, datalad-deprecated package not found 
[Level 5] Importing dataset 
[Level 5] Done importing dataset 
[Level 8] No module named 'requests_ftp' [http.py:<module>:62] 
[DEBUG] Failed to import requests_ftp, thus no ftp support: ModuleNotFoundError(No module named 'requests_ftp') 
[DEBUG] Run ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] (cwd=None) 
[Level 8] Process 40720 started 
[Level 5] STDERR: git --git-dir= confi (Thread<(STDERR: git --git-dir= confi, 5)>) started 
[Level 5] STDERR: git --git-dir= confi (Thread<(STDERR: git --git-dir= confi, 5)>) exiting (exit_requested: False, last data: None) 
[Level 5] STDOUT: git --git-dir= confi (Thread<(STDOUT: git --git-dir= confi, 3)>) started 
[Level 5] STDOUT: git --git-dir= confi (Thread<(STDOUT: git --git-dir= confi, 3)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] Read 630 bytes from 40720[stdout] 
[Level 5] process_waiter (Thread<process_waiter>) exiting 
[Level 8] Process 40720 exited with return code 0 
[DEBUG] Finished ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0 
[DEBUG] Assigning credentials into 21 providers 
[DEBUG] Assigning credentials into 22 providers 
[DEBUG] Returning provider Provider(authenticator=<<HTTPBasicAuthA++74 chars++one)>>, credential=<<UserPassword(d++38 chars++one)>>, name='datalad-test.org', url_res=['https://datalad-test.org.*']) for url https://datalad-test.org 
[DEBUG] Importing keyring 
[DEBUG] Loading Gnome 
[DEBUG] Loading Google 
[DEBUG] Loading Windows (alt) 
[DEBUG] Loading file 
[DEBUG] Loading keyczar 
[DEBUG] Loading multi 
[DEBUG] Loading pyfs 
[DEBUG] Loading KWallet 
[DEBUG] Loading SecretService 
[DEBUG] Loading Windows 
[DEBUG] Loading chainer 
[DEBUG] Loading libsecret 
[DEBUG] Loading macOS 
Please set a password for your new keyring:
indeed leads us into `keyring` module...
tail of it with traceback on log lines 
[DEBUG] git-credential-datalad:33>gitcredential_datalad:89,131>providers:424,356  Returning provider Provider(authenticator=<<HTTPBasicAuthA++74 chars++one)>>, credential=<<UserPassword(d++38 chars++one)>>, name='datalad-test.org', url_res=['https://datalad-test.org.*']) for url https://datalad-test.org 
[DEBUG] git-credential-datalad:33>gitcredential_datalad:89,161>credentials:211>keyring_:81,46  Importing keyring 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Gnome 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Google 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Windows (alt) 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading file 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading keyczar 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading multi 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading pyfs 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading KWallet 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading SecretService 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Windows 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading chainer 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading libsecret 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading macOS 
Please set a password for your new keyring:

and that gitcredential_datalad:161 is https://github.com/datalad/datalad/blob/HEAD/datalad/local/gitcredential_datalad.py#L161

    saved_provider.credential.set(user=attrs['username'],
                                  password=attrs['password'])

so all makes sense in that regard (keyring tries to save and for that needs to initiate keyring)... so the most reasonable resolution would probably be to craft some helper/fixture/decorator to establish a fake keyring for the duration of the test (we do not want that generally since some tests do use real credentials to test against S3 if such credentials available).

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Jul 8, 2022
@yarikoptic
Unsucesfull attempt to provide fixture creating a plain text keyring
b22c26e 
does not work for some reason on debian... easiest is to just set

PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring

for the entire process and then nothing is asked...

Ref: datalad#6623
yarikoptic added a commit to yarikoptic/datalad that referenced this issue Jul 8, 2022
@yarikoptic
Unsucesfull attempt to provide fixture creating a plain text keyring
3ef6a74 
does not work for some reason on debian... easiest is to just set

PYTHON_KEYRING_BACKEND=keyrings.alt.file.PlaintextKeyring

for the entire process and then nothing is asked...

Ref: datalad#6623
@yarikoptic yarikoptic mentioned this issue Jul 8, 2022
Closed
@yarikoptic yarikoptic mentioned this issue Jul 23, 2022
Closed
@yarikoptic yarikoptic mentioned this issue Nov 9, 2022
Closed
@bpoldrack bpoldrack mentioned this issue Nov 30, 2022
Merged
bpoldrack added a commit to bpoldrack/datalad that referenced this issue Dec 8, 2022
@bpoldrack
Use Plaintext keyring backend in tests
d8d538a 
by patching `os.environ` from within `conftest.py:setup_package` in
order to not have the keyring asking for a password in order to be
created in non-interactive (CI-) test runs.

(Closes datalad#6623)
@yarikoptic-gitmate
Copy link
Collaborator

Issue fixed in 0.17.10

yarikoptic added a commit to yarikoptic/datalad that referenced this issue Mar 28, 2023
@yarikoptic
ENH(TST): add DATALAD_TESTS_CREDENTIALS=system for known credentials …
f6b074f 
…to be used during tests

We hardcoded to set it to a plain text keyring in
d8d538a of
datalad#7209 to completely disable possible
attempt of keyring to interact with user e.g. to establish a new keyring.
That was done to address datalad#6623 which came up during debian package building.

But the need for older behavior was realized while approaching datalad#7340
where interactions with s3 were replaced to use boto3 but to test I needed
credentials, but we were just skipping the tests since credentials were not
present in that fake keyring.

With this patch I could run

  DATALAD_TESTS_CREDENTIALS=system python3 -m pytest -s -v datalad/downloaders/tests/test_s3.py

and see tests to interact with S3.  Test relating to NDA were failing
since most likely NDA broke the "flow" of how to access data there.
@yarikoptic yarikoptic mentioned this issue Mar 28, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
DX developer experience
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use plaintext keyring backend in test bpoldrack/datalad
4 participants
@yarikoptic @TheChymera @bpoldrack @yarikoptic-gitmate