Error cloning with private SSH key

[austenadler]

I am always getting the error:

Load key "/home/.../ssh-key": invalid format

When using keys generated on linux with ssh-keygen. I believe it is due to the newline being trimmed when the key is sent to the worker.

Steps taken:

I added:

import subprocess
print(subprocess.run(["cp", keyPath, "/tmp/copied"])

After this line: https://github.com/buildbot/buildbot/blob/master/master/buildbot/util/git.py#L33. The copied file has no trailing newline. I tried a git clone with this keyfile and it failed. If I manually added a newline to /tmp/copied, clone worked.

I added a second trailing newline to the keyfile but it still copied with no newlines. Finally, I added a space to the end of the file and it copied with the space and the clone worked after restarting the master.

Possible fix:
Figure out why this key doesn't have a trailing newline.

Version:

Buildbot version: 2.7.0
Twisted version: 20.3.0

[tardyp]

Hello,
The code for downloading the key is here:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/util/git.py#L264

There might be some kind of strip() hidden somewhere.

You could print(repr(self.sshPrivateKey[-3:])) to figure out where it is. actual print will be stored in the twisted.log file

Pierre

[austenadler]

Okay I added it:

print("XXXDEBUG: ", [ord(i) for i in private_key[-3:]])
# XXXDEBUG:  [45, 45, 45]
# The dash character from ---OPENSSL PRIVATE KEY---

So p.render(self.sshPrivateKey) is stripping the newline. I'm having trouble tracking all the renderers in secret. I can look a little but I do not know where the keyfile is converted into a string.

Edit: I see now buildbot/steps/download_secret_to_worker.py. Will add a comment if I find anything

[austenadler]

I see. I had configured strip=True in my build master. I did not realize this would trim newlines from private keys. This is not an issue.

Edit: I got this configuration option from this issue: #4681. It was never resolved, but I assumed the only issue there was hostkey format

[tardyp]

Hey,

For the record, where exactly did you have the strip=True?

[austenadler]

Same as in #4681, I put it in SecretInAFile: ```python c['secretsProviders'] = [ secrets.SecretInAFile(dirname="/var/lib/buildbot/secrets", strip=True), ] ``` Apr 21, 2020 2:26:11 AM Pierre Tardy <notifications@github.com>:

…

Hey, For the record, where exactly did you have the strip=True? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub [#5264 (comment)] , or unsubscribe [https://github.com/notifications/unsubscribe-auth/ALFAXOQ23AZGAFW4XXXMTALRNU4ADANCNFSM4MMDN2AQ] . [https://github.com/notifications/beacon/ALFAXOVFIT443WTU4UK24WDRNU4ADA5CNFSM4MMDN2A2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOETDFVHA.gif]

[razrdog0]

I've come across this same issue trying to pass ssh keys in git steps while using the SecretInAFile provider. setting strip=False fixes the ssh key but striping newlines from secrets is preferred in my case.

I've created a helper function and test case for downloading ssh keys which ensures there is a newline before sending ssh keys without affecting other secrets. I'll be creating a merge request in a moment referencing this issue with the fix.