Unable to check out SSH submodule using deploy key

[daveespo]

Moving this thread from #116 over here

@ericsciple -- just tried the new ssh-key param but no joy .. (the SSH private key that I'm passing in via Secret is one that works with the ssh-agent hack I described earlier in this thread)

2020-03-11T20:08:31.0562996Z Current runner version: '2.165.2'
2020-03-11T20:08:31.0564725Z Prepare workflow directory
2020-03-11T20:08:31.0791957Z Prepare all required actions
2020-03-11T20:08:31.0806683Z Download action repository 'actions/checkout@master'
2020-03-11T20:08:33.0116518Z Download action repository 'sfdx-actions/setup-sfdx@v1'
2020-03-11T20:08:33.2496238Z Download action repository 'nanasess/setup-chromedriver@master'
2020-03-11T20:08:40.5559823Z Download action repository 'actions/upload-artifact@v1'
2020-03-11T20:08:41.1828303Z ##[group]Run actions/checkout@master
2020-03-11T20:08:41.1828907Z with:
2020-03-11T20:08:41.1829353Z   lfs: true
2020-03-11T20:08:41.1831532Z   ssh-key: ***
2020-03-11T20:08:41.1831924Z   submodules: true
2020-03-11T20:08:41.1832300Z   repository: <redacted>
2020-03-11T20:08:41.1832778Z   token: ***
2020-03-11T20:08:41.1833150Z   ssh-strict: true
2020-03-11T20:08:41.1833521Z   persist-credentials: true
2020-03-11T20:08:41.1833910Z   clean: true
2020-03-11T20:08:41.1834281Z   fetch-depth: 1
2020-03-11T20:08:41.1834679Z ##[endgroup]
2020-03-11T20:08:41.6235786Z Added matchers: 'checkout-git'. Problem matchers scan action output for known warning or error strings and report these inline.
2020-03-11T20:08:41.6237082Z Syncing repository: <redacted>/patronsignup
2020-03-11T20:08:41.6237958Z Working directory is '/home/runner/work/patronsignup/patronsignup'
2020-03-11T20:08:41.6310463Z [command]/usr/bin/git version
2020-03-11T20:08:41.6458784Z git version 2.25.1
2020-03-11T20:08:41.6491554Z [command]/usr/bin/git lfs version
2020-03-11T20:08:41.7979462Z git-lfs/2.10.0 (GitHub; linux amd64; go 1.13.4)
2020-03-11T20:08:41.8001760Z Deleting the contents of '/home/runner/work/patronsignup/patronsignup'
2020-03-11T20:08:41.8013376Z [command]/usr/bin/git init /home/runner/work/patronsignup/patronsignup
2020-03-11T20:08:41.8095844Z Initialized empty Git repository in /home/runner/work/patronsignup/patronsignup/.git/
2020-03-11T20:08:41.8104612Z [command]/usr/bin/git remote add origin git@github.com:<redacted>/patronsignup.git
2020-03-11T20:08:41.8155136Z [command]/usr/bin/git config --local gc.auto 0
2020-03-11T20:08:41.8200346Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2020-03-11T20:08:41.8240012Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2020-03-11T20:08:41.8606381Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2020-03-11T20:08:41.8641874Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
2020-03-11T20:08:41.8916066Z Temporarily overriding GIT_SSH_COMMAND="/usr/bin/ssh" -i "$RUNNER_TEMP/3e0953a3-c730-442d-9693-4efc5a4164fb" -o StrictHostKeyChecking=yes -o CheckHostIP=no -o "UserKnownHostsFile=$RUNNER_TEMP/3e0953a3-c730-442d-9693-4efc5a4164fb_known_hosts"
2020-03-11T20:08:41.8921024Z [command]/usr/bin/git config --local core.sshCommand "/usr/bin/ssh" -i "$RUNNER_TEMP/3e0953a3-c730-442d-9693-4efc5a4164fb" -o StrictHostKeyChecking=yes -o CheckHostIP=no -o "UserKnownHostsFile=$RUNNER_TEMP/3e0953a3-c730-442d-9693-4efc5a4164fb_known_hosts"
2020-03-11T20:08:41.8960017Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2020-03-11T20:08:41.9006512Z [command]/usr/bin/git lfs install --local
2020-03-11T20:08:41.9487647Z Updated git hooks.
2020-03-11T20:08:41.9488142Z Git LFS initialized.
2020-03-11T20:08:41.9510631Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +c485eed0d330b13b19d081fb186d2f35836986ae:refs/remotes/origin/master
2020-03-11T20:08:42.3528109Z ERROR: Repository not found.
2020-03-11T20:08:42.3535717Z ##[error]fatal: Could not read from remote repository.
2020-03-11T20:08:42.3539415Z 
2020-03-11T20:08:42.3540237Z Please make sure you have the correct access rights
2020-03-11T20:08:42.3540662Z and the repository exists.
2020-03-11T20:08:42.3541669Z The process '/usr/bin/git' failed with exit code 128

And my workflow YAML looks like

    steps:
      - uses: actions/checkout@master
        with:
          lfs: true
          ssh-key: ${{ secrets.SFDX_COMMON_SSH_KEY }}
          submodules: true

And my .gitmodules is

[submodule "sfdx-common"]
	path = sfdx-common
	url = git@github.com:<redacted>/sfdx-common.git

[ericsciple]

Does the ssh-key have access to both repos? Or are you using a deploy key (access to only one repo)?

[daveespo]

The ssh-key that I'm supplying to the action/checkout action is only authorized for the submodule (sfdx-common). I presumed that the repo that the workflow is running in would already be implicitly authenticated. Do I have that part wrong?

[ericsciple]

Oh I see. Currently if you supply an SSH key then it's used to checkout the main repo and submodules.

[ericsciple]

Since one of the features of checkout@v2 is to persist the creds on disk to enable easier scripting authenticated git commands, I think using the provided sshkey for the main repo makes sense (remote url is configured with the ssh url and core.sshCommand is setup for auth).

Looking into multiple keys...

[ericsciple]

It looks like ssh allows multiple -i identity_file args.

What do you think about being able to specify multiple ssh keys. For example:

ssh-key: |
  ${{ secrets.my_main_repo_deploy_key }}
  ${{ secrets.my_other_repo_deploy_key }}

I think if one doesnt work, it will fallback and try the next. To be clear, this currently won't work - would need to update the action to support it.

Would that be a good solution?

[daveespo]

I have to back up a second and ask how the checkout action works today ... Is it the case that whatever implicit authentication scheme that is used provides only read-only access to the repo it's running against? i.e. if I were to switch over to using deploy keys for this, would I lose the ability to commit back to the repo? (this isn't a use case I have today, but have been asked by a few team members if we could commit build artifacts back to the repo or to tag a release .. long story)

Second, by switching over to require SSH, it increases the burden of configuration. I need to set up deploy keys for both repos. Not a huge burden, but it does make things more complex.

And then there's that rule about not being able to use the same private key for more than one repo's deploy-key, right? "Key is already in use" if you try to use the same private/pub key pair as a deploy key? That's a huge drag that we've had to work around by creating a dedicated paid Github user (we're on a paid Teams plan) to act as a service user just so we can bless that user with readonly access to a bunch of repositories.

So, yes, what you propose would work, but I think it's going to frustrate a lot of people.

You may need to support multiple SSH keys anyway if a repo has multiple submodules .. so I can see why you're thinking along those lines.

[ericsciple]

how the checkout action works today ... Is it the case that whatever implicit authentication scheme that is used provides only read-only access to the repo it's running against?

The default token - i.e. ${{ github.token }} - token has read/write permission to the main repo and read permission to public repos.

if I were to switch over to using deploy keys for this, would I lose the ability to commit back to the repo?

No. When creating a deploy key, there is a checkbox to Allow write access.

increases the burden of configuration. I need to set up deploy keys for both repos

Have you considered granting the service account read access to the main repo also?

If in the future you want to commit back, the default auth token is still embedded in the config too (http.https://github.com/.extraheader). So if you switch the URL back to HTTPS, then push will use the embedded token. For example: git remote set-url origin https://github.com/my-org/my-repo.git

[daveespo]

It sounds like the only real option is to use the service account to grant access to both repos .. and yes, we can do this.

But again, it sounds more convoluted than dealing with the submodule checkout myself (with the ssh-agent action)

I would definitely prefer for the repo that the workflow is running in to be implicitly authorized and only have to deal with SSH authorization for submodules. But so far it sounds like I'm the only one that's given this a whirl :-)

[ericsciple]

Proposed here: #190

[daveespo]

Great! Though I looked at the PR and it looks just like copy changes? I don't see code changes

[ericsciple]

It was a proposed change to the spec, but prevented other scenarios. As part of the discussion we figured out separate inputs submodules-ssh-key and submodules-ssh-token would be better.

[martinitus]

I would definitely prefer for the repo that the workflow is running in to be implicitly authorized and only have to deal with SSH authorization for submodules. But so far it sounds like I'm the only one that's given this a whirl :-)

Nope, you are not ;-)

I already use a deploy key to pull in a python dependency from another private repository 'B' into the build of our repository 'A'. For this to work, i set up a private key via a configured secret in 'A' and the respective public key in 'B' and use the following step:

      - name: Setup access via public/private key.
        # Below command requires the FOOBAR_PRIVATE_KEY to be configured via github repository secrets.
        # Also the key's public part must be added to the foobar repository deploy keys.
        # A private/public key pair without password (required in this case) can be generated with ssh-keygen.
        # This part is used for the git/foobar part in requirements.txt
        run: |
          mkdir ~/.ssh
          echo "${{ secrets.FOOBAR_PRIVATE_KEY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa

I am tempted to simply clone the submodule in another custom step until the checkout action officially supports it again :)

[paresy]

It looks like ssh allows multiple -i identity_file args.

What do you think about being able to specify multiple ssh keys. For example:

ssh-key: |
  ${{ secrets.my_main_repo_deploy_key }}
  ${{ secrets.my_other_repo_deploy_key }}

I think if one doesnt work, it will fallback and try the next. To be clear, this currently won't work - would need to update the action to support it.

Would that be a good solution?

I like that idea (if it works technically).

On GitHub the problem is, that we need one key for each submodule. AFAIK the idea in PR #190 will only allow one key for all submodules. But we need multiple if we have multiple private submodules. Therefore +1 for this idea.

Thanks!