fpm creates debs with underscores, but cannot consume them #340

Closed
ToddG opened this Issue Jan 15, 2013 · 10 comments

Projects

None yet

3 participants

@ToddG
ToddG commented Jan 15, 2013
fpm -s dir -t deb -a all -n 'jdk6u' -v 1.6.0.37 /opt/local/java/java6/jdk1.6.0_37
fpm -s dir -t deb -a all -n 'product' -v 1.0 -d ./jdk6u_1.6.0.37_all.deb product

Where:
/opt/local/java/java6/... is a java6 jdk installation
and ./product is a dummy application src tree

Error:
Created deb package {"path":"/home/todd/releases/jdk6u_1.6.0.37_all.deb"}
Replacing underscores with dashes in './jdk6u_1.6.0.37_all.deb' because debs don't like underscores {"level":"warn"}
/var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:339:in `gsub!': can't modify frozen string (TypeError)
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:339:in `fix_dependency'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:304:in `converted_from'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:303:in `collect'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:303:in `converted_from'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package.rb:208:in `convert'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/command.rb:396:in `execute'
    from /var/lib/gems/1.8/gems/clamp-0.3.1/lib/clamp/command.rb:64:in `run'
    from /var/lib/gems/1.8/gems/clamp-0.3.1/lib/clamp/command.rb:126:in `run'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/bin/fpm:8
    from /usr/local/bin/fpm:19:in `load'
    from /usr/local/bin/fpm:19

Q: Why is fpm creating a debian package that it cannot consume? Is there a workaround?

@jordansissel
Owner

I'm confused. The error "Replacing underscores ..." shows up after the "Created deb package" one?

@ToddG
ToddG commented Jan 15, 2013
  1. This line executes fine
    fpm -s dir -t deb -a all -n 'jdk6u' -v 1.6.0.37 /opt/local/java/java6/jdk1.6.0_37
  2. It generates this artifact
    /home/todd/releases/jdk6u_1.6.0.37_all.deb
  3. But then, this line fails
    fpm -s dir -t deb -a all -n 'product' -v 1.0 -d ./jdk6u_1.6.0.37_all.deb product
  4. And returns this error:
    Created deb package {"path":"/home/todd/releases/jdk6u_1.6.0.37_all.deb"}
    Replacing underscores with dashes in './jdk6u_1.6.0.37_all.deb' because debs don't like underscores {"level":"warn"}
    /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:339:in gsub!': can't modify frozen string (TypeError) from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:339:infix_dependency'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:304:in converted_from' from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:303:incollect'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package/deb.rb:303:in converted_from' from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/package.rb:208:inconvert'
    from /var/lib/gems/1.8/gems/fpm-0.4.26/lib/fpm/command.rb:396:in execute' from /var/lib/gems/1.8/gems/clamp-0.3.1/lib/clamp/command.rb:64:inrun'
    from /var/lib/gems/1.8/gems/clamp-0.3.1/lib/clamp/command.rb:126:in run' from /var/lib/gems/1.8/gems/fpm-0.4.26/bin/fpm:8 from /usr/local/bin/fpm:19:inload'
    from /usr/local/bin/fpm:19
  5. So basically, the deb created in line #1 cannot be used in line #3.
@jordansissel
Owner

Oooh, I see now.

Ok, so, I'll explain the intended operation first - The '-d' flag is meant to specify a dependency, such as -d 'jdk6u = 1.6.0.37 with a 'name operator version' style, where name is the name of the package to depend on, operator is "=", ">" etc, and version is the version of that package.

You're using -d in a strange (but perhaps a fair) way.

First thing, let's fix your command line so you can move on with your work:

fpm -s dir -t deb -a all -n 'product' -v 1.0 -d "jdk6u = 1.6.0.37" product

Second, I'm open to supporting your use case (specifying a dependency by giving the path to a package file), but only if the solution above isn't acceptable to you.

Let me know :)

@ToddG
ToddG commented Jan 16, 2013

Jordan -

I'm creating a package for an application with numerous dependencies,
some already exist as packages (mysql, for example), some do not (java6
or java7, for example).

I'll back up a bit and see if my approach is correct.

Approach 1: Break out deps into separate packages

My original thought was that I should use fpm to package up all the
dependencies that did not exist in a packaged form, and archive those
somewhere...say a private artifact repo. {I guess I think this way b/c
this is a pretty common thing when working on java apps. Pack up
dependencies into separate jars, stuff them into an artifact repo, and
reference them.}

Then, when I need to cut an application release, I would simply point at
my application directories, and reference the nec. packages (both well
known, like mysql, and private packages that I created). This seemed
logical, because in reality, I have several applications, with
overlapping dependencies (apps A,B,C all depend on java6, for example).

However, I'm getting the feeling that's not the intended use case for
fpm. Correct me if I'm wrong, but it seems the expected use case is more
like below:

Approach 2: KISS - just create one monolithic package

Create one single package for your app, but instead of creating packages
for private dependencies...just include those artifacts directly in this
package. So, if I need java6, for example, and that's located at
/opt/java/java6...then just add that to the package with the
--directories flag. So if my app looked like:

/path/to/app
/path/to/private/dep1
/path/to/private/dep2

I'd package it with something like:

fpm -s dir -t deb -a all -n 'app_name' -v 1.0 -d "some_public_dep =
some_version" --directories /path/to/private/dep1 --directories
/path/to/private/dep2 /path/to/app

I'll try out approach #2, and see how far I get.

With respect to your proposed solution...

I'm not sure what I'm doing wrong, but the private dependency is not
getting packaged with the app...which it would have to, b/c it's a
private package and not publicly available.

[releases]$ /usr/local/bin/fpm -s dir -t deb -a all -n 'product' -v 1.0
-d "jdk6u = 1.6.0.37" product

Created deb package {:path=>"/home/todd/releases/product_1.0_all.deb"}
[releases]$ h
/home/todd/releases
total 161844
58203198 -rw-rw-r-- 1 todd 165711854 Jan 14 18:18 jdk6u-1.6.0.37-all.deb
59639372 drwxrwxr-x 4 todd 4096 Jan 14 15:37 product
58203199 -rw-rw-r-- 1 todd 812 Jan 16 08:58 product_1.0_all.deb
[releases]$ mkdir temp
[releases]$ cd temp/
[temp]$ dpkg-deb -R ../product_1.0_all.deb .
[temp]$ h
/home/todd/releases/temp
total 8
58459850 drwxr-xr-x 2 todd 4096 Jan 16 08:58 DEBIAN
58459844 drwxrwxr-x 4 todd 4096 Jan 16 08:58 product
[temp]$ tree
.
├── DEBIAN
│ ├── conffiles
│ └── control
└── product
├── bin
│ └── binary.txt
└── src
└── source.txt

4 directories, 4 files
[temp]$ cat DEBIAN/con
cat: DEBIAN/con: No such file or directory
[temp]$ cat DEBIAN/control
Package: product
Version: 1.0
License: unknown
Vendor: todd@foo
Architecture: all
Maintainer: todd@foo
Installed-Size: 0
Depends: jdk6u (= 1.6.0.37)
Section: default
Priority: extra
Homepage: http://example.com/no-uri-given
Description: no description given

Thanks again!
-Todd

On 01/15/2013 04:27 PM, Jordan Sissel wrote:

Oooh, I see now.

Ok, so, I'll explain the intended operation first - The '-d' flag is
meant to specify a dependency, such as |-d 'jdk6u = 1.6.0.37| with a
'name operator version' style, where name is the name of the package
to depend on, operator is "=", ">" etc, and version is the version of
that package.

You're using -d in a strange (but perhaps a fair) way.

First thing, let's fix your command line so you can move on with your
work:

|fpm -s dir -t deb -a all -n 'product' -v 1.0 -d "jdk6u = 1.6.0.37"
product|

Second, I'm open to supporting your use case (specifying a dependency
by giving the path to a package file), but only if the solution above
isn't acceptable to you.

Let me know :)

— Reply to this email directly or view it on GitHub
#340 (comment).

Todd Greenwood-Geer

@ToddG
ToddG commented Jan 29, 2013

Jordan... I'm proceeding with Approach #2 below.

I have a couple questions:

Q: I have a dependency on mysql. Assuming I'm installing onto a clean
ubuntu/debian instance...is it possible to script the installation of a
dependency, such as mysql? In this example, I'd like to provide my own
password for the root mysql account.

Q: What happens for upgrades? Are there any special hooks that are
invoked in the upgrade case? For example, if I write A1.deb, and then
write the upgrade, A2.deb... what happens then?

Example:
A1 : installed --> user invokes 'apt-get upgrade' or tries to apt-get
install A2

-t

On 01/16/2013 09:08 AM, Todd Greenwood-Geer wrote:

Jordan -

I'm creating a package for an application with numerous dependencies,
some already exist as packages (mysql, for example), some do not (java6
or java7, for example).

I'll back up a bit and see if my approach is correct.

Approach 1: Break out deps into separate packages

My original thought was that I should use fpm to package up all the
dependencies that did not exist in a packaged form, and archive those
somewhere...say a private artifact repo. {I guess I think this way b/c
this is a pretty common thing when working on java apps. Pack up
dependencies into separate jars, stuff them into an artifact repo, and
reference them.}

Then, when I need to cut an application release, I would simply point at
my application directories, and reference the nec. packages (both well
known, like mysql, and private packages that I created). This seemed
logical, because in reality, I have several applications, with
overlapping dependencies (apps A,B,C all depend on java6, for example).

However, I'm getting the feeling that's not the intended use case for
fpm. Correct me if I'm wrong, but it seems the expected use case is more
like below:

Approach 2: KISS - just create one monolithic package

Create one single package for your app, but instead of creating packages
for private dependencies...just include those artifacts directly in this
package. So, if I need java6, for example, and that's located at
/opt/java/java6...then just add that to the package with the
--directories flag. So if my app looked like:

/path/to/app
/path/to/private/dep1
/path/to/private/dep2

I'd package it with something like:

fpm -s dir -t deb -a all -n 'app_name' -v 1.0 -d "some_public_dep =
some_version" --directories /path/to/private/dep1 --directories
/path/to/private/dep2 /path/to/app

I'll try out approach #2, and see how far I get.

With respect to your proposed solution...

I'm not sure what I'm doing wrong, but the private dependency is not
getting packaged with the app...which it would have to, b/c it's a
private package and not publicly available.

[releases]$ /usr/local/bin/fpm -s dir -t deb -a all -n 'product' -v 1.0
-d "jdk6u = 1.6.0.37" product

Created deb package {:path=>"/home/todd/releases/product_1.0_all.deb"}
[releases]$ h
/home/todd/releases
total 161844
58203198 -rw-rw-r-- 1 todd 165711854 Jan 14 18:18 jdk6u-1.6.0.37-all.deb
59639372 drwxrwxr-x 4 todd 4096 Jan 14 15:37 product
58203199 -rw-rw-r-- 1 todd 812 Jan 16 08:58 product_1.0_all.deb
[releases]$ mkdir temp
[releases]$ cd temp/
[temp]$ dpkg-deb -R ../product_1.0_all.deb .
[temp]$ h
/home/todd/releases/temp
total 8
58459850 drwxr-xr-x 2 todd 4096 Jan 16 08:58 DEBIAN
58459844 drwxrwxr-x 4 todd 4096 Jan 16 08:58 product
[temp]$ tree
.
├── DEBIAN
│ ├── conffiles
│ └── control
└── product
├── bin
│ └── binary.txt
└── src
└── source.txt

4 directories, 4 files
[temp]$ cat DEBIAN/con
cat: DEBIAN/con: No such file or directory
[temp]$ cat DEBIAN/control
Package: product
Version: 1.0
License: unknown
Vendor: todd@foo
Architecture: all
Maintainer: todd@foo
Installed-Size: 0
Depends: jdk6u (= 1.6.0.37)
Section: default
Priority: extra
Homepage: http://example.com/no-uri-given
Description: no description given

Thanks again!
-Todd

On 01/15/2013 04:27 PM, Jordan Sissel wrote:

Oooh, I see now.

Ok, so, I'll explain the intended operation first - The '-d' flag is
meant to specify a dependency, such as |-d 'jdk6u = 1.6.0.37| with a
'name operator version' style, where name is the name of the package
to depend on, operator is "=", ">" etc, and version is the version of
that package.

You're using -d in a strange (but perhaps a fair) way.

First thing, let's fix your command line so you can move on with your
work:

|fpm -s dir -t deb -a all -n 'product' -v 1.0 -d "jdk6u = 1.6.0.37"
product|

Second, I'm open to supporting your use case (specifying a dependency
by giving the path to a package file), but only if the solution above
isn't acceptable to you.

Let me know :)

— Reply to this email directly or view it on GitHub
#340 (comment).

Todd Greenwood-Geer

@jordansissel
Owner

debian/ubuntu have "preseed" feature that lets you answer questions (like the mysql root password) about a package's installation before you install it. I don't know how to use this system since I do not use it, but I know it exists.

As for what happens during upgrades, I don't know and it may be implementation dependent (apt-get upgrade may do different things than yum, in different orders, etc) - no idea ;)

@jordansissel
Owner

debian/ubuntu have "preseed" feature that lets you answer questions (like the mysql root password) about a package's installation before you install it. I don't know how to use this system since I do not use it, but I know it exists.

As for what happens during upgrades, I don't know and it may be implementation dependent (apt-get upgrade may do different things than yum, in different orders, etc) - no idea ;)

@ToddG
ToddG commented Jan 29, 2013

Cool, thanks!

On 01/29/2013 04:20 PM, Jordan Sissel wrote:

debian/ubuntu have "preseed" feature that lets you answer questions
(like the mysql root password) about a package's installation before you
install it. I don't know how to use this system since I do not use it,
but I know it exists.

As for what happens during upgrades, I don't know and it may be
implementation dependent (apt-get upgrade may do different things than
yum, in different orders, etc) - no idea ;)


Reply to this email directly or view it on GitHub
#340 (comment).

Todd Greenwood-Geer

@ToddG
ToddG commented Jan 31, 2013

Jordan -
How do I cause dependencies to be installed?

Say I used fpm to create foo.deb, which depends on mysql-server-5.5:

$ fpm -s dir -t deb -a all -n 'foo' -v 1.0 -d mysql-server (= 5.5)
./some/source/dir

Then when I attempt to install foo.deb on my target machine, I was
expecting that foo.deb would cause the system to install mysql first.
Not shown, but I even pre-seeded my debconf with the values for the
mysql-server root password.

Instead, I got:

dpkg: dependency problems prevent configuration of foo:
foo depends on mysql-server (= 5.5); however:
Package mysql-server is not installed.
foo depends on mysql-client (= 5.5); however:
Package mysql-client is not installed.
dpkg: error processing foo (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
foo

What is the correct way to both specify the dependency and have it
installed? Should I just invoke my package manager in the pre-install
script?

-Todd

On 01/29/2013 04:20 PM, Jordan Sissel wrote:

debian/ubuntu have "preseed" feature that lets you answer questions
(like the mysql root password) about a package's installation before you
install it. I don't know how to use this system since I do not use it,
but I know it exists.

As for what happens during upgrades, I don't know and it may be
implementation dependent (apt-get upgrade may do different things than
yum, in different orders, etc) - no idea ;)


Reply to this email directly or view it on GitHub
#340 (comment).

Todd Greenwood-Geer

@r4um
Collaborator
r4um commented Apr 9, 2013

@ToddG best way would be host your own repository of packages and then install from there, invoking package
manager in pre-install is a bad idea. For hosting/building your own repository consider

@r4um r4um closed this Jun 5, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment