Clone other private repositories using Netrc file

[hmarr]

It would be great to be able to inject a pre-defined SSH key into containers.

As part of our build process, we pull down dependencies that are in private GitHub repos. Currently there's no (easy) way of accessing these repos from Drone. This could be resolved by adding the ability to generate an SSH key that's used by containers (copied to ~/.ssh/id_rsa), which we could then add as the deploy key for our dependencies.

[afex]

have you looked into the -identity flag for the drone command?

[hmarr]

No, I hadn't see that - it looks like it'll solve my issue perfectly, thanks!

It's still worth documenting that somewhere, and ideally adding the functionality to the web UI.

I'll close this for now, and leave it up to you whether you want to open a new issue for the web UI.

Thanks again.

[skorfmann]

I tried to find the right place to add a global SSH key for my Drone instance, however I couldn't find it.

How and where do you provide that identity flag?

[bradrydzewski]

SSH keys are automatically generated for each repository and stored in the database. For private projects, the public key is automatically registered with GitHub. The private key gets injected into the build container at $HOME/.ssh/id_rsa

[skorfmann]

SSH keys are automatically generated for each repository and stored in the database. For private projects, the public key is automatically registered with GitHub. The private key gets injected into the build container at $HOME/.ssh/id_rsa

Yes, I took that generated key, removed it from the 'deployment keys' on Github and added it to our 'automation github user'. However, as we have various projects with private dependencies, it gets quite cumbersome to repeat this for every new project. Hence I thought, it would be nice to add a global key that gets injected to every build.

The identity option for the drone command sounds pretty much like the solution I was looking for, but I wasn't able to wrap my head around it. I found this reference in the code, but I just couldn't figure out how to utilize it.

[bradrydzewski]

The --identity option is only available to the drone client. The drone client is used to execute builds in virtual machines locally, on your personal computer, as opposed to on the server.

The drone server does not provide a global ssh key. We can definitely come up with a solution to this, however, in the short term you'll need to add the keys to your automation user account.

[skorfmann]

Cool, thanks for explaining.

[thenewguy]

Is there a solution to this now? The issue is closed and I didn't see anything referencing it. This would be very helpful.

[bradrydzewski]

I have a solution that I'd like to implement to solve this in our 0.3 branch (labeled exp). It should be a relatively simple change, with just a few modifications to the codebase

Step 1: alter the REST handler that updates the repo to accept a key pair
https://github.com/drone/drone/blob/exp/server/handler/repo.go#L180

Step 2: add a command to the CLI to upload a key pair
https://github.com/drone/drone/tree/exp/cmd

The command would look something like drone key set github.com/foo/bar /path/to/private/key. The repo name and private key would be provide as command line args. The public key is assumed to reside in the same directory as the private key, but with a ".pub" extension.

Ideally we could verify the following in the command line utility, prior to uploading:

• the public / private key are valid using Go's RSA package
• the key pair does not require a password

NOTE this would give you the ability to change key pairs from the command line only (or via the RESTful API). You would not be able to update the key pair via the website. We are trying to keep the website as minimalist as possible, and are trying to push all advanced features to the command line utility / REST API.

[gdborton]

@skorfmann We've had some success using personal access tokens on our 'machine user', and then providing them in the url for the repository.

Similar to this

git clone https://<token>:x-oauth-basic@github.com/owner/repo.git

It's still cumbersome, but slightly less so.

[bradrydzewski]

Just pushed 96e6603 which allows you to update the SSH keys via REST API. Next step will be to add this functionality to the command line utility.

[bradrydzewski]

Official support for updating your repository keys via rest:
http://readme.drone.io/api/repos/put/

Along with support for updating keys via the command line utility:

drone set-key github.com/foo/bar ~/.ssh/id_rsa

[bradrydzewski]

we are going to experiment with a new idea here. We are going to add a .netrc file to the build environment with the access token, similar to the comment by @gdborton

this would not apply to public (open source) repositories for security reasons.

this should work well for GitHub and GitLab. I'm sure we can get something working with Gogs as well. Bitbucket seems like a lost cause, but you never know ...

[hd-deman]

take a look https://github.com/blog/1957-git-2-3-has-been-released for new GIT_SSH_COMMAND

[MattyKuzyk]

I am having issues with Github SSH and Drone.

Originally, I was using a deploy key from Drone on my repo and everything was peachy. However, we soon ran into a problem where we needed to clone a private repo in our npm install. I made a machine user on Github that I added the keys for on my machine and on the machine user, and added that machine user to our orgs on GH. I could clone everything just fine from the machine, all the access rights were correct.

When drone would build, however, it wouldn't pull in this key and use another key (presumably the deploy key) to try and clone this repo we were using in npm.

So I used the REST API to update the public and private keys on Drone to be that of our user, it worked properly and on the web interface it showed my repo as having that public key.

Now I can't even get Drone to clone my repo in the first place since the keys for that are now wrong, and I basically took two steps backward.

@bradrydzewski am I doing something completely wrong? I now have a broken Drone instance with no way to reset the keys. Are there any workarounds here?

[shawnzhu]

@MattyKuzyk you can reset key pair of any of your repo via command drone set-key. see this comment. it works for me when using latest drone cli.

[bradrydzewski]

closing in favor of #1090 which is implemented in 0.4 branch (upcoming release). We use a .netrc file to authorize cloning the https url. This means you can go get or npm install other private repos

[jgermade]

Hi @bradrydzewski , could you show an example of use with the .netrc

In the proyect I'm working in, I have a public repo that uses a private for build. So I'm not sure I can use this approach

also, can not find an example of use for the identity flag

thanks in advance!

[bradrydzewski]

@jgermade the netrc file is only injected into private build environments for security reasons. Exposing a netrc file or ssh key for a public repository would make it susceptible to theft by issuing a malicious pull request.

You are more than welcome to use secrets to provide a GitHub token or credentials to your build environment, but secrets won't be injected into your build environment for pull requests for the same security reasons mentioned above.

You can read more about secrets at http://readme.drone.io/usage/secrets/

[jgermade]

thanks @bradrydzewski for support!!

finally got it working combining token in secret and sed replace the bower.json dependence