Automatically deploy PKGBUILD to AUR

[alexander-bauer]

The AUR accepts package updates via Git, so updating an AUR package is as simple as pushing a git repository to their server. This repository must have the PKGBUILD at its root, however. One way I think we can accomplish this is to extract the dist/arch/ directory as a subtree on-the-fly, and push that to the AUR as one step in a continuous deployment pipeline. If the subtree is extracted in the same way every time, it will keep the same history, with just the commits relevant to that directory, regardless of when or where it is generated.

Doing this would mean collaborating with @jtojnar, who owns the package on the AUR.

[jtojnar]

I think the most reasonable would be creating a new AUR account and adding it as co-maintainer. Then adding the following script into deploy section of .travis.yml.

git clone ssh://aur@aur.archlinux.org/tectonic.git staging/arch
cp dist/arch/PKGBUILD staging/arch
cd staging/arch
mksrcinfo
git add PKGBUILD .SRCINFO
git commit -m "Release $TRAVIS_TAG"
git push origin master

For generating .SRCINFO we can either use makepkg or probably easier pkgbuild-introspection (only depends on bash).

We would also need to add a private key for deploying. This looks like another area where Travis is not optimal but we could always try to store it in an environment variable and then save it to a proper file in the script.

[pkgw]

I will leave it to you two (and other Arch Linux users) to think about how you'd like to do this.

Am I understanding correctly that in this model, every push to master would result in an update of the AUR git repo and a new version appearing on the AUR? Is that standard practice? If it is also common to only push updates for new tagged upstream releases, I might prefer that model — life as a maintainer is a bit easier if your users aren't running ever-changing unreleased versions of the package.

[jtojnar]

In AUR, there are two kinds of packages: Normal packages that get updated when a new version is tagged, and VCS packages that always fetch the current HEAD and need to be updated only when dependencies change. The tectonic package is the former one, since it lacks -git suffix.

Travis allows you to mark deploy scenarios to run only when a tag is added, see conditional releases.

[alexander-bauer]

@jtojnar It looks like the way to accomplish deployment with Travis will be to store the ssh key in the repository, but keep it encrypted with Travis's file encrypting capabilities. We'll have to collaborate with @pkgw to generate a "deploy" keypair, and to store the encrypted one in the repository with the Travis key. After that, we can create an AUR user as a co-maintainer and install the deploy public key.

[pkgw]

OK, I am happy to generate a keypair and get an encrypted private key into .travis.yml if you can outline any unusual settings for the SSH keys and what bits need to go into the Travis file.

[alexander-bauer]

@pkgw If you are willing, I think these are the steps that you'll need to do, as someone with Travis access:

1. Generate a standard ssh keypair without a password using ssh-keygen and put the results where in the repository (I suggest dist/deploy_key, but you may want a separate directory)
2. Check in the public key (dist/deploy_key.pub), but not the private one (dist/deploy_key)
3. Install and log into the Travis CLI client gem install travis && travis login
4. Encrypt the secret key, adding the associated command to the Travis file: travis encrypt-file dist/deploy_key --add
5. Add the resulting dist/deploy_key.enc and changes to .travis.yml to the repository

Afterward, if you're so inclined, you can remove or shred the unencrypted private key. Once this public key is available, @jtojnar and I can collaborate to figure out the rest.

[pkgw]

OK. CC #65. I think that I had to monkey with the file path in the .travis.yml (its suggested change named the file as just deploy_key not dist/deploy_key) but wanted to put it through a CI run to double-check that.

[alexander-bauer]

With the key now extant (thanks @pkgw!), I think the next step is to create an automated tectonic-deploy user on the AUR with that public key installed. It would be much nicer to add the key directly to the AUR repository as a deploy key, like GitHub does, but the AUR doesn't have that feature.

I've gone ahead and created this user (tectonic-deploy), and think the following steps are next:

1. Change the email address to one associated with the Tectonic project, if one exists. Otherwise, I can keep hold of it.
2. Add it as a co-maintainer on the AUR project. This will require @jtojnar's doing.
3. Add the deploy step of the Travis build.

[jtojnar]

I added the user as co-maintainer, will try to create the deploy script later.

[pkgw]

@alexander-bauer For the time being, I guess you can/should keep the account associated with your email address. If more of these accounts need to be created I guess I will make a GMail account for the project leadership — this is also something that comes up in the Code of Conduct.

[pkgw]

It seems as if the autodeploy didn't quite work: see the Travis build log.

[alexander-bauer]

I'm on my phone at the moment, but from what I can see it looks like there's an erroneous `git commit` when there isn't a branch checked out. Should be an easy fix?

…

On Jul 8, 2017 11:10 AM, "Peter Williams" ***@***.***> wrote: It seems as if the autodeploy didn't quite work: see the Travis build log <https://travis-ci.org/tectonic-typesetting/tectonic/jobs/251498381>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfHPrKy9R52s_tkieTvgGvYnCC8WZzyks5sL5vGgaJpZM4Nticc> .

[jtojnar]

I do not see why a branch would not be checked out. Cloning it locally chooses master.

On a side note, I switched my Arch machine to NixOS so AUR packaging is no longer a priority for me. I can transfer the ownership to you, if you wish.

[pkgw]

I think I might want to tag another release shortly to update the Documentation link on crates.io. So if anyone has a chance to try to solve this in the near future, there can be a chance to test the fix :-)

[alexander-bauer]

I'm happy to take ownership of the aur package; I'll hopefully have some time to resolve this tonight or tomorrow.

…

On Jul 9, 2017 10:01 AM, "Peter Williams" ***@***.***> wrote: I think I might want to tag another release shortly to update the Documentation link on crates.io. So if anyone has a chance to try to solve this in the near future, there can be a chance to test the fix :-) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfHPn8dbcm2T2U8gAogyl5JpBeAD53dks5sMN0dgaJpZM4Nticc> .

[jtojnar]

@alexander-bauer I transferred the package to you.

[pkgw]

Hello, folks that have previously been interested in packaging Tectonic on Arch Linux! It's been a while — apologies for not being super attentive to this aspect of things.

I put out a 0.1.10 release today, and I decided to spend a bit of time trying to figure out the status of automatic deployment to the AUR. There were definitely a few problems. I've fixed some of the more superficial ones, but it's still not quite working.

Here's the latest build log. As far as I can tell, the SSH key that we're trying to use to deploy just isn't working. Would that have changed in the past year or so?

Edit: and in case they come in handy, here's the current deploy script and here's the relevant piece of the .travis.yml that also does some stuff.

[pkgw]

@jtojnar Looking at the AUR page, perhaps the problem is that the tectonic-deploy user is not a co-maintainer? Are you able to alter that?

I realized that the past few releases wouldn't have worked anyway because I wasn't updating the PKGBUILD file — I didn't realize it needed maintenance. I will try to keep it updated going forward.

[jtojnar]

@pkgw I do not see a link for managing co-maintainers any more, perhaps @polyzen needs to do it.

[polyzen]

Edit: and in case they come in handy, here's the current deploy script and here's the relevant piece of the .travis.yml that also does some stuff.

Seems it would be better to use mksrcinfo available in the pkgbuild-introspection package. Save some bandwidth, at least.

@pkgw I do not see a link for managing co-maintainers any more, perhaps @polyzen needs to do it.

Done, and you are also now (again :p) the maintainer of the package. Tried to switch myself to a co-maintainer, but something went wrong there. I will just send a PR or something if need be. :)

[jtojnar]

@polyzen I do not use Arch any more, so I transferred it back to you.

[pkgw]

It worked! Thanks all for assisting.

@polyzen It appears the pkgbuild-introspection has been removed — see the comment at the bottom of its package page. That's why I switched away from it.