New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Pacman (ArchLinux package) support #916

Merged
merged 42 commits into from Nov 7, 2015

Conversation

Projects
None yet
8 participants
@djhaskin987
Contributor

djhaskin987 commented May 9, 2015

I finished it! Addresses #231.

Features

  • ArchLinux packaging support (pacman packages)
  • Lots of rspecs to make sure it works 馃憤 Everytime I found an error, I wrote an rspec and coded to it. Broke none, made more.
  • Both input() (conversion from pacman) and output() (conversion to pacman) works.
  • Implementation is completely independent of makepkg. It doesn't call it, it doesn't use it.
    • That makes fpm the first pacman implementation (that I know of) that is MIT licensed
    • That also means you can create a pacman package without being on ArchLinux.
    • All you need to have on your machine is tar, du, and bsdtar

Known issue(s)

  • The input() function uses a method that sort-of parses bash so that it can slurp the maintainer scripts into FPM format. It's not perfect -- If it sees an unbalanced curly brace in valid bash, like echo "{", it will choke. However, this is a small corner case and not common, and unless I made the function nearly a full bash parser itself, I couldn't get around this. However, for 98% of the cases out there, it should work.
  • With other package types, the --after-install and --before-install , as well as the --after-remove and --before-remove scripts run regardless of whether it's an upgrade case or not. With pacman, if the installation is an upgrade, only the --after-upgrade and --before-upgrade scripts are run. This is more of a pacman nuance, but it should be noted. If those scripts are not specified, pacman will die

Other notes

I had to move the copy_metadata() method from the FPM::Package::Dir class to the FPM::Util module. It's included in all the classes, so the Dir class still works, but pacman is really picky with permissions. To ensure that the permissions of the files in the pacman package match those of the files the packager gave FPM, I needed to use copy_metadata in FPM::Package::Pacman as well as FPM::Package::Dir, so I made it available to everyone by putting it in the FPM::Util module. Hope that's alright :) I tried to make sure I made as few disruptive changes as possible, so I hope this change is not unwelcome :)

djhaskin987 added some commits Dec 26, 2014

begin
while true
line = lines.next
m = /^\s*(\w+)\s*\(\s*\)\s*\{\s*([^}]+?)?\s*(\})?\s*$/.match(

This comment has been minimized.

@jordansissel

jordansissel Oct 7, 2015

Owner

maybe put a comment above showing an example line entry for easier eyeballing of this to see if it's right

while true
line = lines.next
m = /^\s*(\w+)\s*\(\s*\)\s*\{\s*([^}]+?)?\s*(\})?\s*$/.match(
line)

This comment has been minimized.

@jordansissel

jordansissel Oct 7, 2015

Owner

put this on the line above?

Maybe put the regexp into a constant and then have m = INSTALL_SCRIPT_LINE_REGEX.match(line) or something?

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Oct 7, 2015

Lots of specs failing when I run them on OSX:

Example - all are related to tar invocations, it seems:

  1) FPM::Package::Pacman#output Test that empty epoch is tested properly should have the correct name
     Failure/Error: original.output(target)
     FPM::Util::ProcessFailed:
       tar failed (exit code 1). Full command was:["tar", "cJf", "/var/folders/4r/4fpyt_2d1qvfrgb2fnngrqm40000gn/T/studtmp-19648f37-87eb-4ec8-84fe-888b422caad2.pkg.tar.xz", "--numeric-owner", "--owner", "0", "--numeric-owner", "--group", "0", ".MTREE", ".PKGINFO"]
     # ./lib/fpm/util.rb:83:in `safesystem'
     # ./lib/fpm/package/pacman.rb:273:in `block (2 levels) in output'
     # ./lib/fpm/package/pacman.rb:272:in `chdir'
     # ./lib/fpm/package/pacman.rb:272:in `block in output'
     # ./lib/fpm/util.rb:146:in `call'
     # ./lib/fpm/util.rb:146:in `with'
     # ./lib/fpm/package/pacman.rb:271:in `output'
     # ./spec/fpm/package/pacman_spec.rb:89:in `block (4 levels) in <top (required)>'
@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 9, 2015

Yes, the tar invocations --

The only way to make all the options to work is to use bsdtar . Normally on systems, when you call bsdtar, it forwards the call to GNU tar, not the actual bsdtar . But they're actually different. You'll find that the tar spec problems disappear when you install bsdtar instead of just the usual tar. It's installed by default on arch linux systems. Arch's makepkg makes use of bsdtar's unique options and functionalities to make its package.

I use libarchive's bsdtar to make the .MTREE file just like makepkg does. I use the "normal" tar command to make the actual archive.

bsdtar is a prerequisite for packaging arch linux, the tar format is unique to bsdtar. I ran rspec, it works (except for an unrelated python rspec 馃憤 )

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 9, 2015

I have looked at all of your suggestions, thanks for them! Let me know if you would like to see anything else.

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 9, 2015

Actually, looking at the error, I don't know if this is a bsdtar thing. But my specs run fine? I'm confused.

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 9, 2015

FYI

[ djhaskin987@djhaskin987-S301LA:~ ]$ tar --version
tar (GNU tar) 1.27.1
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by John Gilmore and Jay Fenlason.
[ djhaskin987@djhaskin987-S301LA:~ ]$ bsdtar --version
bsdtar 3.1.2 - libarchive 3.1.2
[ djhaskin987@djhaskin987-S301LA:~ ]$
@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 10, 2015

Tested on CentOS 6 and Debian 6. As long as bsdtar was installed, I got no rspec pacman errors.

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 10, 2015

It looks like XZ compression may not be entirely supported on OS X. I'll try using the --xz on tar instead of the -J option I use above and see what happens. The document in that link says it already works on OS X :)

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 10, 2015

I changed the code to use an option that is supposed to work better is OS X, But I don't have access to OS X. The change works in Ubuntu, Debian and CentOS. Would you mind testing it again, @jordansissel?

@flaccid

This comment has been minimized.

flaccid commented Oct 11, 2015

For OS X El Capitan users they will probably run into ffi/ffi#461 / #1010 which is what I'm running into.

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 11, 2015

In addition, OS X users may need to install xz, although supposedly with the tar --xz option OS X can create (if not decompress) tar.xz archives with no additional installation. I appreciate your help tracking OS X bugs down, @flaccid , and anyone else who would care to look at my stuff for that OS. I just don't have a mac available to me, I'm on Ubuntu 15.04 .

I do have access via vagrant to a FreeBSD 10.2 machine I can test it on, I'll look into making sure it runs on that. Would that help?

@hatt

This comment has been minimized.

Contributor

hatt commented Oct 14, 2015

The OS X bundled version of tar supports --xz by default anyway, assuming the user is on 10.7 or later. The -J flag seems to be arbitrary between gnutar and bsdtar

@flaccid

This comment has been minimized.

flaccid commented Oct 19, 2015

Testing with fpm --debug -s dir -t pacman -n "slashbin" -v 1.0 /bin /sbin.
...

Process is running {:pid=>15023, :level=>:debug, :file=>"fpm/util.rb", :line=>"74", :method=>"safesystem"}
tar: Option --owner is not supported {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Usage: {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
  List:    tar -tf <archive-filename> {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
  Extract: tar -xf <archive-filename> {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
  Create:  tar -cf <archive-filename> [filenames...] {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
  Help:    tar --help {:level=>:info, :file=>"cabin/mixins/pipe.rb", :line=>"47", :method=>"block in pipe"}
Process failed: tar failed (exit code 1). Full command was:["tar", "--xz", "-cf", "/private/tmp/slashbin-1.0-1-x86_64.pkg.tar.xz", "--numeric-owner", "--owner", "0", "--numeric-owner", "--group", "0", ".MTREE", ".PKGINFO", "bin", "sbin"] {:level=>:error, :file=>"fpm/command.rb", :line=>"492", :method=>"rescue in execute"}
Cleaning up staging path {:path=>"/var/folders/sx/ht6gznq979v5r725sqt7_p617p38sz/T/package-dir-staging20151020-15018-clex4o", :level=>:debug, :file=>"fpm/package.rb", :line=>"279", :method=>"cleanup_staging"}
Cleaning up build path {:path=>"/var/folders/sx/ht6gznq979v5r725sqt7_p617p38sz/T/package-dir-build20151020-15018-yq29wo", :level=>:debug, :file=>"fpm/package.rb", :line=>"286", :method=>"cleanup_build"}
Cleaning up build path {:path=>"/var/folders/sx/ht6gznq979v5r725sqt7_p617p38sz/T/package-pacman-build20151020-15018-1hud8fy", :level=>:debug, :file=>"fpm/package.rb", :line=>"286", :method=>"cleanup_build"}
@jordansissel

This comment has been minimized.

Owner

jordansissel commented Oct 21, 2015

Just FYI, I'm OK if this doesn't work on OSX for the first version of it. (We can merge if folks agree it's ready)

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Oct 21, 2015

You will hear no dissent from me 馃憦

@hatt

This comment has been minimized.

Contributor

hatt commented Oct 23, 2015

馃憤

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Nov 7, 2015

damn you merge conflicts! I will try to fix and merge.

@jordansissel jordansissel merged commit 0654b94 into jordansissel:master Nov 7, 2015

jordansissel added a commit that referenced this pull request Nov 7, 2015

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Nov 7, 2015

Merged <3

@djhaskin987

This comment has been minimized.

Contributor

djhaskin987 commented Nov 7, 2015

YAY!!!!! Thanks!

@coxley

This comment has been minimized.

coxley commented Nov 9, 2015

馃憤

@flaccid

This comment has been minimized.

flaccid commented Jan 19, 2016

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment