[BUG] git.latest fails with permission error on macOS when using identity file and non-standard user

[taranlu-houzz]

Description
When using git.latest I get a permission error if using the user parameter combined with the identity parameter.

Edit: Looks like this only happens when I am trying to run this state against a minion that is running on the same machine as the master. This works fine if used on a minion running on another machine...

----:
----------
          ID: states.git.clone_---- - Clone ---- repo or pull latest
    Function: git.latest
        Name: git@github.com:----
      Result: False
     Comment: Failed to check remote refs: Unable to authenticate using identity file:

              git@github.com: Permission denied (publickey).
              fatal: Could not read from remote repository.

              Please make sure you have the correct access rights
              and the repository exists.
     Started: 17:22:17.809552
    Duration: 1664.535 ms
     Changes:

Setup
sls file that fails:

{{ sls }} - Clone ---- repo or pull latest:
  git.latest:
    - name: git@github.com:----
    - target: {{ pillar['minion_home'] }}/git/----
    - user: {{ pillar['minion_user'] }}
    - branch: master
    - identity: salt://data/ssh/----_deploy_key
    - force_checkout: True
    - force_fetch: True
    - force_reset: remote-changes

sls file that works (but clones/updates repo as root):

{{ sls }} - Clone ---- repo or pull latest:
  git.latest:
    - name: git@github.com:----
    - target: {{ pillar['minion_home'] }}/git/----
    - branch: master
    - identity: salt://data/ssh/----_deploy_key
    - force_checkout: True
    - force_fetch: True
    - force_reset: remote-changes

• minion_user is just a different user than root in this case
• minion_home is the above user's home directory

Expected behavior
The repo should be cloned/updated to latest correctly by the user specified using the user parameter.

Versions Report

salt --versions-report

Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.12.2
       cherrypy: unknown
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.6
      gitpython: 2.1.15
         Jinja2: 2.10.1
        libgit2: 1.0.0
       M2Crypto: 0.35.2
           Mako: 1.0.7
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.8.1
   pycryptodome: Not Installed
         pygit2: 1.2.0
         Python: 3.7.7 (default, Mar 10 2020, 15:43:27)
   python-gnupg: 0.4.4
         PyYAML: 5.1.2
          PyZMQ: 18.0.1
          smmap: 3.0.2
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.1

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 17.7.0
         system: Darwin
        version: 10.13.6 x86_64

Additional context
This seems like it might be related to: #42550

[DmitryKuzmenko]

Right now I have no chance to reproduce this on MacOS. The described configuration works good for me in Linux. Probably it's something specific for Mac. @tam-core could anyone try this?

[taranlu-houzz]

I've just found that it actually isn't an issue with the machine running both the master and a minion, it seems to be some other identity/ssh/permission issue. I disabled the ssh config and other old keys for the user on a different machine running a minion and also cleared out the stored keys using ssh-add -D and now it is exhibiting the same behavior. Still works with identity and no user param, but fails when I try to use the user.

[taranlu-houzz]

So, I just ran this command on the machine running the minion that exhibits this behavior as mentioned above: sudo salt-call -l debug state.single git.latest "git@github.com:<repo>.git" target="<path>/test" identity="salt://data/ssh/<key>" user="<user>" which gives me this output:

...
[INFO    ] Checking remote revision for git@github.com:<repo>.git
[DEBUG   ] In saltenv 'base', ** considering ** path '/tmp/__salt.tmp.5n5z96o9' to resolve 'salt://data/ssh/<key>'
[DEBUG   ] Fetching file from saltenv 'base', ** attempting ** 'salt://data/ssh/<key>'
[INFO    ] Fetching file from saltenv 'base', ** done ** 'data/ssh/<key>'
[WARNING ] /usr/local/Cellar/salt/3000.1/libexec/lib/python3.7/site-packages/salt/modules/file.py:32: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Iterable, Mapping, namedtuple

[DEBUG   ] LazyLoaded file.user_to_uid
[INFO    ] Attempting git authentication using identity file /tmp/__salt.tmp.5n5z96o9
[INFO    ] Executing command ['git', 'ls-remote', 'git@github.com:<repo>.git'] as user '<user>' in directory '/Users/<user>'
[WARNING ] /usr/local/Cellar/salt/3000.1/libexec/lib/python3.7/site-packages/salt/utils/decorators/signature.py:31: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
  *salt.utils.args.get_function_argspec(original_function)

[DEBUG   ] LazyLoaded user.info
[ERROR   ] Command '['su', '-l', '<user>', '-c', "/usr/local/bin/bash -l -c 'cd -- /Users/<user> && { git ls-remote git@github.com:<repo>.git\n }'"]' failed with return code: 128
[ERROR   ] stderr: git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
[ERROR   ] retcode: 128
[DEBUG   ] Removed identity file /tmp/__salt.tmp.5n5z96o9
[ERROR   ] Failed to check remote refs: Unable to authenticate using identity file:

git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
...

The above command executes successfully when I remove the user parameter.