Generate repository and Upload on successful check

[lordfolken]

Once a build passes the continous integration stage, the repository should be generated with nodejs and automatically uploaded to download.xcsoar.org/repository.

[Turbo87]

@lordfolken that used to work fine with the old download server, but I guess the upload instructions need to be updated. do you have the necessary credentials to upload it? I suppose it works via scp now instead of ftp?

[lordfolken]

Yes, but that is my personal public key. So I need to discuss with cirrus of how we proceed.

[Turbo87]

I think it would be easiest to create a user account on that download server that is only able to modify this one file, generate a SSH key, authorize it to ssh into that account and add it to travis as an encrypted file. I could do the whole ssh key and travis thing, but you would need to take care of the user account part on the server.

[Turbo87]

@lordfolken any news on this?

[lordfolken]

I talked to cirrus and he is willing to have a ssh keypair with limited access to the repository file. So there should be soon a solution.

[lordfolken]

I have the ssh pair for the repository file. How do we proceed?

[lordfolken]

I saw this in the travis-ci config

after_success:
- test $TRAVIS_PULL_REQUEST == 'false' &&
test $TRAVIS_BRANCH == 'master' &&
test $FTP_CREDENTIALS &&
npm run convert &&
curl -T repository -u $FTP_CREDENTIALS --disable-epsv ftp://download.xcsoar.org/

I would suggest to paste the private key into an env variable in the travis-ci configuration, and write that into a file in the build environment at runtime. This file can then be handed to scp with the -i switch as a private key.

This seems to be the way to do it as I assertain it. Do you think this is solid and secure?

[Turbo87]

@lordfolken there is a better way. I'll write up the necessary steps tomorrow.

[Turbo87]

have a look at https://docs.travis-ci.com/user/encrypting-files and https://github.com/Turbo87/sidebar-v2/tree/master/.travis for an example.

you basically use the travis CLI tool to encrypt the file and save the decryption keys into env vars. then you commit the encrypted file and add the encryption command to some deploy step. in our case we won't use git to push anywhere on deploy but rather use scp to transfer the generated file to the download server. I'd suggest testing this locally first before pushing it to the repo.

[Turbo87]

we might also want to use something like https://github.com/ember-cli/ember-cli/blob/master/.travis/deploy.sh#L5-L18 to make sure we don't leak the key

[Turbo87]

@lordfolken any progress on this? can I help in some way?

[Turbo87]

I assume with 2676521 and it's previous commits this can be closed now.

[lordfolken]

Not perfectly happy with the solution, but it works.

[Turbo87]

@lordfolken any specific issues with it?

[lordfolken]

i cannot check the host fingerprint, as the ssh_known_host declarative does not allow to specify a port. In the background it uses ssh-keyscan which in theory supports the -p flag, but that isn't exposed via the travis interface. The whole mechanism is actually stupid, as it scans the ssh-fingerprint during the build. Correct would be to store the hostkey in a secure in travis.yml and then pipe that into the .ssh/known_hosts file. Only that way we can guarantee that we are actually deploying to the correct server. (aside from the public/private) I might implement this, but i'd rather focus on content checking now.

[Turbo87]

I see, thanks for the clarification!

[Turbo87]

@lordfolken it seems that the current approach is breaking PR builds due to the missing secrets in those builds. we should only try to decrypt the key of we actually deploy from master.