Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Please use dpkg tools to build .deb when available #409

Closed
bigon opened this Issue Apr 11, 2013 · 15 comments

Comments

Projects
None yet
7 participants

bigon commented Apr 11, 2013

Hello,

I'm currently thinking about package fpm for debian and include it in the official archive.

I got one objection about the fact that it's completely bypassing dpkg to generate the package (see: http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=22;bug=688896)

Do you think it could be possible to call the native dpkg (dpkg-deb -c) when it's installed on the machine?

Owner

jordansissel commented Apr 11, 2013

I appreciate your efforts here :)

The request made by the debian person (Wouter?) on that ticket seems laced with personal bias and not real-world data. Patching fpm to do something completely different, functionally, than it does today? That seems like a weird request.

Basically, fpm avoids dpkg tools explicitly for two reasons:

  • because not all systems have dpkg and friends
  • because deb's tooling mandates policies with no way to disable

Wouter's suggestion to patch to make it use dpkg/etc would make fpm suffer because part of the functional feature set of fpm is to be able to build debs and avoid any conflicts your universe might have with the debian policies and universe.

Lastly, fpm is a plugin-based system. If "use dpkg/debuild/etc" is a requirement, anyone is welcome to contribute a package plugin that only works on Debian systems that also uses the Debian packaging tools.

Owner

jordansissel commented Apr 11, 2013

Specific responses to Wouter:

While this happens to work

This inflamatory judgement is offensive and misinformed. fpm has a pretty awesome test suite to make sure it actually works and does things correctly.

Owner

jordansissel commented Apr 11, 2013

What if I provided an apt repo with fpm in it, so folks could 'apt-get install fpm' ?

Contributor

kwilczynski commented May 6, 2013

👍 to the Apt repository. 👎 goes to Debian trying to hack shit up again. Anyway, perhaps an omnibus package for FPM in the future? :)

because deb's tooling mandates policies with no way to disable

I think you are mixing up the several layers of .deb tools that we have. First there's dpkg-deb, used to handle .deb files, this tool has no polices by design (for example related to paths and similar) beside what dpkg itself would accept as a validly formed .deb file (because it uses dpkg-deb as a backend), and some of the checks it performs at build time can even be disabled (at the peril of producing broken packages with --nocheck).

Then there's the package build scripts (found in dpkg-dev on Debian-based systems), stuff like dpkg-gencontrol, dpkg-genchanges, dpkg-source or dpkg-vendor. These might not make sense when not using Debian source packages, like for example in Fink. In any case, the very small amount of "policy" contained in those scripts is dependent on the vendor, and can be selected easily by passing a specific one (via the DEB_VENDOR environment variable, see the Dpkg::Vendor::* submodules for example policies). Among those, there are few other tools that are pretty much universally useful when creating .deb binary packages, and not using them is prone to create possibly buggy packages; these would be dpkg-architecture to map compiler GNU triplets to valid dpkg architecture names instead of having to hard-code the mappings in 3rd party code; dpkg-shlibdeps and dpkg-gensymbols to generate correct dependency information for executable, shared libraries, plugins, etc, because the installed library packages contain ABI information, that if ignored could produce too loose dependencies that could for example crash a program.

Then there's packaging helper tools, like debhelper, cdbs, dh_python and similar. These might be heavy on policy, stuff like what files to expect where, injection of required maintainer script snippets (like ldconfig, or python module handling), when to compress what, etc. But in any case these are also dependent on using a Debian source package, so are mostly not usable on other environments, although some of their actions might need to be replicated elsewhere to be able to correctly integrate within that specific distribution.

Coming back to dpkg-deb, yes, creating .deb files from scratch should be considered supported, and that's one of the reasons the format is documented in detail, for example, in the deb(5) man page, but on Debian-based systems that should be considered discouraged, because it's not possible to spot warnings from dpkg-deb for old stuff being deprecated, or failing to pick up new .deb package defaults (like compression parameters), or similar. So, yes, not using at least dpkg-deb, dpkg-architecture, dpkg-shlibdeps and dpkg-gensymbols on Debian-based systems, is prone to create problems, with different degrees of severity.

As long as fpm does not use these native tools on Debian-based systems to create packages targeted at those same systems, I think it should pretty much be discouraged, and I don't think packages for it should be included in those distributions, because it does not integrate nicely with the native environment. In any case, but related, the comments on the slides and on the front-page regarding the distribution policies do not inspire much confidence, because those policies are there for a reason, and most of the time not following them will cause packages that are alien to the system w/o proper integration or are of low-quality, like most packages created by 3rd party software vendors.

Owner

jordansissel commented Jun 10, 2013

@guillemj I'll try to address all your points, but the general summary is that I believe you are addressing fpm from the perspective of someone outside the target audience of fpm. FPM is aimed at regular folk who have things they need to get done, not official upstream debian package maintainers.

prone to create possibly buggy packages

Further, fear of bugs is not resolved by using debian's own tools. You can create invalid or incorrect packages with debian's own tools.

Bugs don't go away simply by changing the implementation.

This is why the fpm test suite exists: to ensure verify package creation is successful and correct.

I don't think packages for it should be included in those distributions

I don't know what you are referring to, but I"ll answer both possibilities:

  • If you mean fpm should not be included in official debian distributions, I agree. Debian moves at a glacier pace. An 'official debian' fpm package in debian today would be available to users in 2-4 years, at which point the latest fpm package would be far ahead of it. Please do not attempt to include fpm in debian. Doing so would actually hurt the fpm project because today's bugs, fixed tomorrow, would still be reported for another 6 years.
  • If you mean packages produced by fpm should not be included in official debian distributions, I agree again. The target audience for fpm is not Debian package maintainers of official packages. The target audience is everyone else.

alien to the system w/o proper integration or are of low-quality

I suspect you're simply just not in the target audience (as mentioned above a few times) for fpm. This is OK, not all tools are for all people.

Many folks, myself included, use fpm in the wild to generate package deployed to production. These packages are happily accepted by both dpkg and apt-get and work correctly when installed.

In conclusion, fear of bugs is not a strong reason to change fpm's implementation. Further, being able to generate packages without guidance of the debian policy manual is an explicit feature of fpm, and to change that would be to remove a well-used feature of fpm.

Do let me know if I can answer any more questions.

Owner

jordansissel commented Jun 10, 2013

Also! Probably worth mentioning. RPM support in fpm will soon be avoiding rpmbuild just like deb support in fpm avoids debian tooling. This is partly due to the success of the deb support in fpm and partly due to bugs and misfeatures in rpmbuild.

This is the way forward.

bigon commented Jun 14, 2013

Hi,

If you mean fpm should not be included in official debian distributions, I agree. Debian moves at a glacier pace. An 'official debian' fpm package in debian today would be available to users in 2-4 years, at which point the latest fpm package would be far ahead of it. Please do not attempt to include fpm in debian. Doing so would actually hurt the fpm project because today's bugs, fixed tomorrow, would still be reported for another 6 years.

I'll close my ITP then

Cheers

Owner

jordansissel commented Jun 14, 2013

@bigon I do appreciate your efforts though, so thank you for trying to help make the project more awesome :)

calston commented Jul 6, 2013

So Debians stance is "how dare you build a competing product to dpkg"? That's rather bizarre, if something can produce .deb's in line with whatever standards surround the format then anything else is a non-issue.

Collaborator

r4um commented Jul 30, 2013

Closing. FYI the specs use the dpkg-deb program for testing.

@r4um r4um closed this Jul 30, 2013

Contributor

vincentbernat commented Mar 19, 2014

@jordansissel The "Debian is too old" is a common complain from upstreams not wanting to be packaged in Debian. However, being in official Debian repositories (and therefore in Ubuntu) would allow users to just get it with apt-get install, which is quite convenient (and more convenient than the current setup, and more convenient than adding an unofficial repository). So, you'll get more users.

Concerning bugs in stable versions, we don't really handle them unless they are critical. If we get a bug report (through our bug report system) and the bug is already fixed upstream, we mark the bug as is and we move on. If the bug is critical (unlikely), it can be fixed on the next stable release (every 2-3 months). If not, most users can live with it. Consider that we apply the same rules for every package. This includes critical packages like the libc, ssh, etc. Nobody complains they are too buggy. FPM is now mature enough for a release to be usable for several years, isn't it?

Also, we have backports. This allow users not satisfied with the version in stable to get a more recent version. This is an official service we provide and we can get the best of the two worlds: users wanting a "stable" (i.e not moving) package can keep the one in stable, those wanting a more up-to-date version can use the backported one.

Also consider a Debian release is done every 2 years (and Ubuntu every 6 months). And this doesn't prevent users to grab the latest release from GitHub. Maybe most users will be happy with a 2-year old package.

(cc @bigon)

calston commented Mar 19, 2014

@vincentbernat I think 'gem install fpm' is convenient enough for everyone anyway

Contributor

vincentbernat commented Mar 19, 2014

@calston If this was the case, nobody would use the Ruby packages present in Debian. Like puppet, chef and others. Some people like to have the latest version, some people like to have the exact same FPM (dependencies included) than they had 6 months ago.

Owner

jordansissel commented Oct 23, 2014

Some people like to have the latest version, some people like to have the exact same FPM

Nothing preventing you from achieving this. We can make computers solve this for us :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment