debian/ubuntu packaging in saltstack repo needs refresh

[joehealy]

pinging @seanchannel and @KB1JWQ for discussion and comments.

This is just a brain dump of the things we could (or should?) do in a ubuntu packaging refresh/refactor.

Items:

1. switch to quilt format rather than native
2. remove unused bits from debian/rules (dh_override_auto_build typo)
3. build docs at the right time rather than every time (move from % chunk)
4. investigate using dh_python2
5. version numbers on python-zmq to ensure right dependencies installed
6. investigate getting pkg dir included in released .tar.gz
7. start building official uploads from released tar.gz rather than git
8. decide (and document) purpose of (and how to use) debian dir in git (daily builds, peoples custom builds, etc...)
9. warning about editing in config files
10. ensure no compatibility issues between debian derived packages synced to ubuntu and saltstack ubuntu packages

I'm sure there are others and maybe some of these don't need to be done, but I thought I would get them down while they are fresh in my mind.

[ghost]

These are excellent points that have always been on my wish list. Most of the hold-back has been related to keeping Ubuntu's aging LTS release happy while using the same package sources for all newer versions as well, and my Debian mojo is quite limited so I never migrated packages from older to newer deb tools. I agree with all this. Otherwise misc. thoughts:

re #7 when did that start happening? Git should only be used for the 'daily' builds. I thought I wrote something up about it, but may have left of that particular. Please do build release packages only from the tar ball on PyPi or I'm telling. ;)

re #6, #8 Saltstack has a general policy of not putting the OS-specific and packaging stuff in the release. Everything Salt is just a little bit different like that. I would expect this to remain for a while more.

re: #9 if you mean adding warning comments to the config file.. yeah, sure (previously it was installed as e.g. "minion.template", and that's not a terrible thing to go back to either)

[QuinnyPig]

Yes, with #7 we have been using the pypi tarball, but the debian directory from the git tag.

The LTS release from 2010 is probably the single biggest albatross we have at this point. I'm wondering how much longer it makes sense to support via the PPA... It hits EOL in April of 2015.

And I would love to refactor this stuff!

[joehealy]

On Thu, Nov 21, 2013 at 6:21 AM, Corey Quinn notifications@github.comwrote:

Yes, with #7 #7 we have been
using the pypi tarball, but the debian directory from the git tag.

Cool - shows I've never done this part, only "adding" to packaging once you
have done the major release.

The LTS release from 2010 is probably the single biggest albatross we have
at this point. I'm wondering how much longer it makes sense to support via
the PPA... It hits EOL in April of 2015.

yeah - makes it tricky.

And I would love to refactor this stuff!

I'll take this as a statement of intention and back off a little.

Cheers,

Joe

[QuinnyPig]

Oh, no-- if you want to take a stab at it, go nuts-- just let me know what I can do to help with it!

[joehealy]

@boltronics I/we are looking to update some of the packaging in git. As part of this I'm wishing to find out how it is used currently.

I gather you build your own packages, as such I have a few questions:

1. which distros/releases do you build for?
2. do you do any manual patching?
3. what is your experience with debian/ubuntu packaging? ie where are you in a range from debian project leader to maintained packages before to build salt packages because you have to and just follow a recipe?
4. What is your aim in building packages? Is it a reliability and control thing? Is it because you occasionally need your own patches?
5. You've asked about updating a tag, which implies you build against tags. Do you often/ever build against develop?

Hopefully answering these will give us a better understanding of how the repo is currently used and I don't cut people off from current usages.

[boltronics]

No worries.

1. Just Debian GNU/Linux, wheezy amd64
2. Not if I can help it, so generally no.
3. I know enough to be dangerous. :) I have to build packages sometimes for work. Sometimes I take what I can from repo sources like sid and try to backport when necessary to avoid doing everything from scratch. I'm not a developer or an official package maintainer of any particular project.
4. We host our own repo managed with (apt-utils/apt-ftparchive), and sign everything that's put into it. We know everything there has been built and tested to work specifically with Wheezy. We can also trust the repo's uptime, since it's ours. If the salt repo went down or was left in a broken state, there's probably no other mirror anywhere that we can point our production environment to (unlike the official Debian repos, etc). We also know that we can easily roll back to a specific version, since we don't need to delete anything when hosting ourselves.
5. Sometimes. We've been trying to make use of docker states in one of our testing environments. But in general we'd prefer not to.

Cheers.

[joehealy]

Thanks adam, that was quite different to what I expected and good to know.

Looks like we've got a slightly wider set of targets than I thought.

Thanks.

Joe

On Thu, Nov 21, 2013 at 12:57 PM, Adam Bolte notifications@github.comwrote:

No worries.

1. Just Debian GNU/Linux, wheezy amd64
2. Not if I can help it, so generally no.
3. I know enough to be dangerous. :) I have to build packages
   sometimes for work. Sometimes I take what I can from repo sources like sid
   and try to backport when necessary to avoid doing everything from scratch.
   I'm not a developer or an official package maintainer of any particular
   project.
4. We host our own repo managed with (apt-utils/apt-ftparchive), and
   sign everything that's put into it. We know everything there has been built
   and tested to work specifically with Wheezy. We can also trust the repo's
   uptime, since it's ours. If the salt repo went down or was left in a broken
   state, there's probably no other mirror anywhere that we can point our
   production environment to (unlike the official Debian repos, etc). We also
   know that we can easily roll back to a specific version, since we don't
   need to delete anything when hosting ourselves.
5. Sometimes. We've been trying to make use of docker states in one of
   our testing environments. But in general we'd prefer not to.

Cheers.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8675#issuecomment-28951830
.

[ghost]

+1: you guys should totally re-template this baby if you can, fwiw! I also had a goal of harmonizing with what the DD's are doing so as Salt stabilizes it will become unnecessary for anything but daily build packages in the PPA. That day.. is not yet in sight, so we are still saddled with Lucid Lynx (Ubuntu 12.04 LTS). One slight work-around is to allow the updated and proposed repos in the package builds, which I have done previously to sneak in newer versions of deps, but may also mean the users may need those enabled to install Salt (it did not require that so far in the past).

[joehealy]

based on #8833 @t0rrant, we should consider updating changelog entries in the branches at appropriate times...

Need to consider how this integrates with nightly builds.

[t0rrant]

re 10:

The default behaviour when building the salt deb's from git is to add the dependency "upstart-job" which is not widely used outside Ubuntu (to my knowledge).

The classic init scripts are already included within the debian directory after building, so we just remove the dependency from the rules file in "debian/rules".

re @joehealy: it would be nice to have the changelog updated according to the git tagging (v0.16.4,...., v0.17.2)
based on

[joehealy]

This can be closed now, all relevant items have been done for a few releases now