test_datalad_credential_helper test wants interaction

[yarikoptic]

while building debian package (unattended), next day spotted that it wants to talk to me

datalad.local.tests.test_gitcredential.test_datalad_credential_helper ... Please set a password for your new keyring: 
Please confirm the password: 
Error: blank passwords aren't allowed.
Please set a password for your new keyring: 

shouldn't happen, and any needed for that settings (if any) changes should be done in setup_package fixture or prevented via mocking in the test.

[TheChymera]

Just ran into the exact same issue running the tests for 0.16.1

datalad.local.tests.test_gitcredential.test_datalad_credential_helper ... Please set a password for your new keyring:

[bpoldrack]

FTR: Didn't look closely yet, but it's likely GIT_ASKPASS (or the config core.askpass) that need to be set to not let Git ask interactively. Which is currently done in our CI setups. I see the argument to move this to setup_package.

[yarikoptic]

seting GIT_ASKPASS to true is of no effect:

root@smaug:/build/datalad-0.17.0/.pybuild/cpython3_3.9_datalad/build# GIT_ASKPASS=true PYTHONPATH=$PWD PATH=/build/datalad-0.17.0/bin:$PATH python3 -m pytest -s -v datalad/local/tests/test_gitcredential.py::test_datalad_credential_helper
It is highly recommended to configure Git before using DataLad. Set both 'user.name' and 'user.email' configuration variables.
======================================================================================================================= test session starts =======================================================================================================================
platform linux -- Python 3.10.5, pytest-7.1.2, pluggy-1.0.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /build/datalad-0.17.0, configfile: tox.ini
collected 1 item                                                                                                                                                                                                                                                  

datalad/local/tests/test_gitcredential.py::test_datalad_credential_helper create(ok): . (dataset)
Please set a password for your new keyring: 

and we seems to set it already in fixtures:

$> git grep ASKPASS
datalad/conftest.py:            # variable GIT_ASKPASS overwrites this, but neither env var nor this config
datalad/conftest.py:        m.setenv('GIT_ASKPASS', 'true')
datalad/local/gitcredential.py:        # TODO we could prevent prompting by setting GIT_ASKPASS=true
datalad/local/gitcredential.py:        # with GIT_ASKPASS=true would just be an empty string
datalad/tests/utils.py:        # variable GIT_ASKPASS overwrites this, but neither env var nor this config
datalad/tests/utils.py:    set_envvar('GIT_ASKPASS', 'true')

since it is asking to set password for a new keyring it is really just trying to initiate the entire credentials keyring...

debug level output

[DEBUG  ] Run ['git', 'credential', 'approve'] (cwd=/tmp/datalad_temp_test_datalad_credential_helperibyppzvj) 
[Level 8] Process 40635 started 
[Level 5] STDIN: git credential appro (Thread<(STDIN: git credential appro, 7)>) started 
[Level 5] STDIN: git credential appro (Thread<(STDIN: git credential appro, 7)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] Instantiating ssh manager 
[Level 5] Done importing main __init__ 
[Level 5] Importing support.network 
[Level 5] Done importing support.network 
[Level 5] Requested to provide version for cmd:git 
[DEBUG] Run ['git', 'version'] (cwd=None) 
[Level 8] Process 40698 started 
[Level 5] STDERR: git version (Thread<(STDERR: git version, 5)>) started 
[Level 5] STDERR: git version (Thread<(STDERR: git version, 5)>) exiting (exit_requested: False, last data: None) 
[Level 5] STDOUT: git version (Thread<(STDOUT: git version, 3)>) started 
[Level 5] Read 19 bytes from 40698[stdout] 
[Level 5] STDOUT: git version (Thread<(STDOUT: git version, 3)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] process_waiter (Thread<process_waiter>) exiting 
[Level 8] Process 40698 exited with return code 0 
[DEBUG] Finished ['git', 'version'] with status 0 
[Level 8] No module named 'datalad_deprecated' [gitrepo.py:<module>:3744] 
[DEBUG] Not retro-fitting GitRepo with deprecated symbols, datalad-deprecated package not found 
[Level 5] Importing dataset 
[Level 5] Done importing dataset 
[Level 8] No module named 'requests_ftp' [http.py:<module>:62] 
[DEBUG] Failed to import requests_ftp, thus no ftp support: ModuleNotFoundError(No module named 'requests_ftp') 
[DEBUG] Run ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] (cwd=None) 
[Level 8] Process 40720 started 
[Level 5] STDERR: git --git-dir= confi (Thread<(STDERR: git --git-dir= confi, 5)>) started 
[Level 5] STDERR: git --git-dir= confi (Thread<(STDERR: git --git-dir= confi, 5)>) exiting (exit_requested: False, last data: None) 
[Level 5] STDOUT: git --git-dir= confi (Thread<(STDOUT: git --git-dir= confi, 3)>) started 
[Level 5] STDOUT: git --git-dir= confi (Thread<(STDOUT: git --git-dir= confi, 3)>) exiting (exit_requested: False, last data: None) 
[Level 5] process_waiter (Thread<process_waiter>) started 
[Level 5] Read 630 bytes from 40720[stdout] 
[Level 5] process_waiter (Thread<process_waiter>) exiting 
[Level 8] Process 40720 exited with return code 0 
[DEBUG] Finished ['git', '--git-dir=', 'config', '-z', '-l', '--show-origin'] with status 0 
[DEBUG] Assigning credentials into 21 providers 
[DEBUG] Assigning credentials into 22 providers 
[DEBUG] Returning provider Provider(authenticator=<<HTTPBasicAuthA++74 chars++one)>>, credential=<<UserPassword(d++38 chars++one)>>, name='datalad-test.org', url_res=['https://datalad-test.org.*']) for url https://datalad-test.org 
[DEBUG] Importing keyring 
[DEBUG] Loading Gnome 
[DEBUG] Loading Google 
[DEBUG] Loading Windows (alt) 
[DEBUG] Loading file 
[DEBUG] Loading keyczar 
[DEBUG] Loading multi 
[DEBUG] Loading pyfs 
[DEBUG] Loading KWallet 
[DEBUG] Loading SecretService 
[DEBUG] Loading Windows 
[DEBUG] Loading chainer 
[DEBUG] Loading libsecret 
[DEBUG] Loading macOS 
Please set a password for your new keyring: 

indeed leads us into `keyring` module...

tail of it with traceback on log lines

[DEBUG] git-credential-datalad:33>gitcredential_datalad:89,131>providers:424,356  Returning provider Provider(authenticator=<<HTTPBasicAuthA++74 chars++one)>>, credential=<<UserPassword(d++38 chars++one)>>, name='datalad-test.org', url_res=['https://datalad-test.org.*']) for url https://datalad-test.org 
[DEBUG] git-credential-datalad:33>gitcredential_datalad:89,161>credentials:211>keyring_:81,46  Importing keyring 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Gnome 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Google 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Windows (alt) 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading file 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading keyczar 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading multi 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading pyfs 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading KWallet 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading SecretService 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading Windows 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading chainer 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading libsecret 
[DEBUG] gitcredential_datalad:89,161>credentials:211>keyring_:81,55>core:32,83,101>util.__init__:22>backend:212,198  Loading macOS 
Please set a password for your new keyring: 

and that gitcredential_datalad:161 is https://github.com/datalad/datalad/blob/HEAD/datalad/local/gitcredential_datalad.py#L161

    saved_provider.credential.set(user=attrs['username'],
                                  password=attrs['password'])

so all makes sense in that regard (keyring tries to save and for that needs to initiate keyring)... so the most reasonable resolution would probably be to craft some helper/fixture/decorator to establish a fake keyring for the duration of the test (we do not want that generally since some tests do use real credentials to test against S3 if such credentials available).

[yarikoptic-gitmate]

Issue fixed in 0.17.10