A task to package mruby-cli as deb or rpm #10

toch · 2015-08-02T12:48:51Z

Following discussion with @hone, here a first version allowing to package mruby-cli as a deb or rpm.

Some comments:

It depend on the fpm Gem. As there is no Gemfile, I check if it is installed in the rake task.
@hone suggested me to make it a mruby-gem so it could be reused. As I do not have yet found how to do it, I wanted already submit it in order to already collect your feedback. have you any ref about making a mruby gem?

Is there anything I do not do right?

zzak · 2015-08-02T15:04:49Z

I think this should be a separate gem, but we can agree on the design/implementation here.

Since there are some dependencies (fpm, rpmbuild) they would also have to be added to the stack image.

hone · 2015-08-03T23:28:56Z

I'm not sure a separate gem is worth the effort tbh. There aren't great ways in the MRI Rakefile to detect if a mrbgem is installed and then to pull it in. We could hack stuff and then require stuff in there, but that seems SUPER hacky.

zzak · 2015-08-04T13:44:39Z

Let's see if it's possible to build the packaging service we need before worry about doing any hacks.

This seems like we can do rpm and deb, what about the other packages: dmg and windows?

I think it should be all or nothing.

toch · 2015-08-04T14:39:38Z

Sorry for the delay (holidays :p)

Since there are some dependencies (fpm, rpmbuild) they would also have to be added to the stack image.

done there toch/mruby-cli-docker@8d2d65d but as you refer into the Dockerfile to a public image (hone/mruby-cli), toch/mruby-cli-docker should be merged into it, right?

(zzak) I think this should be a separate gem, but we can agree on the design/implementation here.
(hone) I'm not sure a separate gem is worth the effort tbh. There aren't great ways in the MRI Rakefile to detect if a mrbgem is installed and then to pull it in. We could hack stuff and then require stuff in there, but that seems SUPER hacky.

I've just pushed my solution:

What do you think?

This seems like we can do rpm and deb, what about the other packages: dmg and windows?
I think it should be all or nothing.

fpm can also generate osx package, I'll git it a try.
for msi, I'll try https://wiki.gnome.org/msitools

zzak · 2015-08-07T15:42:00Z

Sounds good to me, let's try to get osx and msi working before we ship this.

toch · 2015-08-11T21:43:02Z

A quick update about my investigation.

msitools do the job, but there are no package for 14.04, I'm struggling with compilation of the package,
fpm won't work to build a osx package as it requires pkgbuild tool which is a osx tool.

For osx, I have two options, build a simple dmg package, or a brew package. It seems that dmg is more ubiquitous, but brew easier. what is your preference?

zzak · 2015-08-12T12:07:12Z

Perhaps pkgbuild is something we can get out of osxcross?

Myself, I'd prefer to build a package for OSX as that is more easily distributable (I don't also need homebrew installed). Although, I don't see why we can't offer both as ultimately all of this should be configurable [which] packages we actually output.

Where are you stuck on msitools? I found a package for Wily Werewolf (whatever that is), so maybe we can just backport that? https://launchpad.net/ubuntu/+source/msitools

toch · 2015-08-15T13:44:35Z

Perhaps pkgbuild is something we can get out of osxcross?

I do not know, I have to check.
I found that discussion about building .dmg on the osxcross. It refers two other tools (https://pypi.python.org/pypi/dmgbuild and https://github.com/LinusU/node-appdmg)

Myself, I'd prefer to build a package for OSX as that is more easily distributable (I don't also need homebrew installed). Although, I don't see why we can't offer both as ultimately all of this should be configurable [which] packages we actually output.

👍 let's start first with dmg which is the most ubiquitous.

Where are you stuck on msitools? I found a package for Wily Werewolf (whatever that is), so maybe we can just backport that? https://launchpad.net/ubuntu/+source/msitools

With the upstream, configure failed even if all the dependencies were provided.
With the ubuntu package 15.10, I can go farther but compilation fails with VALAC with the following error:

error: Package `config' not found in specified Vala API directories or GObject-Introspection GIR directories

I'll try first with the one you provided me before digging more the Vala error.

Thanks a lot for your pointers.

zzak · 2015-08-16T13:38:09Z

I found that discussion about building .dmg on the osxcross. It refers two other tools (https://pypi.python.org/pypi/dmgbuild and https://github.com/LinusU/node-appdmg)

I guess it doesn't really matter which tools we use, so long as it works.

With the ubuntu package 15.10, I can go farther but compilation fails with VALAC with the following error:

It looks like that msitools is just a meta package, maybe try finding the equivalent in the 14.04 repos, otherwise installing the newer packages manually until we find which one breaks.

toch · 2015-08-24T15:31:03Z

quick update. I've managed to install msitools into our docker and started to build msi package.

toch · 2015-08-24T16:37:34Z

@zzak
I've updated the gems mruby-cli-distpackage. I can now generate msi package that embeds the exe file mruby-cli.exe. \o/

I still need to try it on a real windows machine. Then, I'll work on dmg package.

toch · 2015-08-24T17:24:57Z

I just tried the x86_64 msi package on a windows 8. It works! (I can install and uninstall it, and I can run mruby-cli)

I cannot easily access a win 32 install. Would you have one?

zzak · 2015-08-24T17:42:03Z

Woot!

I cannot easily access a win 32 install. Would you have one?

I don't but would compatibility mode work?

toch · 2015-08-24T18:32:33Z

I don't but would compatibility mode work?

I've tried to install the 32 msi package on the windows 8 machine and got a cryptic message informing me I can only this for package already installed ?!

zzak · 2015-08-24T18:43:26Z

@toch Can you try installing the 32bit version of Windows 10?
https://www.microsoft.com/en-us/software-download/windows10ISO

This link is also valid for 24 hours.

toch · 2015-08-24T22:10:15Z

Just downloaded it, I'll try it now.

I've just pushed the dmg packaging. My OSX VM crashes I do not know why. Could you try the dmg packages?

zzak · 2015-08-24T22:50:29Z

Sure! I will try out the OSX packages <3

On Monday, August 24, 2015, Christophe Philemotte notifications@github.com
wrote:

Just downloaded it, I'll try it now.

I've just pushed the dmg packaging
https://github.com/toch/mruby-cli-distpackage/blob/master/tasks/package.rake#L141.
My OSX VM crashes I do not know why. Could you try the dmg packages?

—
Reply to this email directly or view it on GitHub
#10 (comment).

toch · 2015-08-24T23:24:05Z

It would be great if we can automatically test the packages on respective platforms. Even after some google-fu, I do not find any hosted CI service for win32, win64 (maybe AppVeyor), OSx32, or OSx64. Do you know one or another doing that?

toch · 2015-08-24T23:47:56Z

Found the problem for the i686 msi package, fixed. it works now.

zzak · 2015-08-24T23:57:41Z

If the container will run on Windows we can use that to test the compiled
binary.

For OS X we should be able to use Travis still, I think.

On Monday, August 24, 2015, Christophe Philemotte notifications@github.com
wrote:

Found the problem for the i686 msi package, fixed. it works now.

—
Reply to this email directly or view it on GitHub
#10 (comment).

toch · 2015-08-25T00:14:24Z

I've fixed my VirtualBox setup, OSX VM works again. I've tried the dmg packages. It nearly works:

dmg is correctly mounted OK
the bundle .app is recognized and can be moved to the Applications folder OK
the execution permissions are not kept, I had to set it again on the binary KO

For both dmg and msi packages, the binary is not available in the PATH. Should it be?

zzak · 2015-08-25T01:44:29Z

It would be nice to get CI hooked up for this, but I realize that's probably a little out of scope for this feature.

the execution permissions are not kept, I had to set it again on the binary KO

Perhaps this is a bug with the packager?

How did you transfer the binary? (I know google drive will remove the executable bit)

For both dmg and msi packages, the binary is not available in the PATH. Should it be?

For most installers, don't you have to put the thing someplace that already exists in your path?
Like the users Application directory, or Program Files on windows?

toch · 2015-09-05T23:34:35Z

@zzak here an update:

for dmg,
- permission problem is fixed
- for the path, after having tried several options, I've added a simple setup bash script that the user can run to add mruby-cli to his environment PATH.
for msi, msitools doesn't support the Environment property, I've ended to find a way which consists in using BATCH command called by a CustomAction ... when it's called, I can't make it work. I'll try to find someone who can help.

The last changes are available in https://github.com/toch/mruby-cli-distpackage/blob/master/tasks/package.rake

zzak · 2015-09-06T13:57:22Z

@toch Does dmg work then?

I can help test osx and 64bit windows, but I'd have to reinstall for 32bit (I think).

toch · 2015-09-07T08:19:04Z

@zzak yes, dmg works.

msi works also. I'm just looking for a way to set the environment path. It's a nice to have to help the user to set it. A simpler way to do that is to provide the instruction to set the env path.

toch · 2015-09-12T18:08:12Z

@zzak I've rebased the PR onto #13.

Quick recap, in order to testdocker-compose run package you need the following:

zzak · 2015-09-17T16:47:31Z

Also found this blog post, that might be interesting:
http://blog.codeship.com/using-docker-build-debian-packages/

zzak · 2015-09-17T17:59:22Z

So here's the output I see:

Writing releases/v0.0.2/mruby-cli-0.0.2-x86_64-pc-linux-gnu.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2-i686-pc-linux-gnu.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2-x86_64-apple-darwin14.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2-i386-apple-darwin14.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2-x86_64-w64-mingw32.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2-i686-w64-mingw32.tgz
Writing releases/v0.0.2/mruby-cli-0.0.2.tgz
Packaging deb for x86_64 into packages/v0.0.2
Packaging deb for i686 into packages/v0.0.2
Packaging rpm for x86_64 into packages/v0.0.2
Packaging rpm for i686 into packages/v0.0.2
Packaging msi for x86_64 into packages/v0.0.2

(wixl:1742): wixl-CRITICAL **: wixl_msi_table_custom_action_add: assertion 'Source != NULL' failed

(wixl:1742): wixl-CRITICAL **: wixl_msi_summary_info_set_author: assertion 'value != NULL' failed

(wixl:1742): wixl-CRITICAL **: wixl_msi_table_property_add: assertion 'value != NULL' failed

** (wixl:1742): CRITICAL **: wixl_add_braces: assertion 'str != NULL' failed

(wixl:1742): wixl-CRITICAL **: wixl_msi_table_property_add: assertion 'value != NULL' failed
Packaging msi for i686 into packages/v0.0.2

(wixl:1748): wixl-CRITICAL **: wixl_msi_table_custom_action_add: assertion 'Source != NULL' failed

(wixl:1748): wixl-CRITICAL **: wixl_msi_summary_info_set_author: assertion 'value != NULL' failed

(wixl:1748): wixl-CRITICAL **: wixl_msi_table_property_add: assertion 'value != NULL' failed

** (wixl:1748): CRITICAL **: wixl_add_braces: assertion 'str != NULL' failed

(wixl:1748): wixl-CRITICAL **: wixl_msi_table_property_add: assertion 'value != NULL' failed
Packaging msi for x86_64 into packages/v0.0.2
genisoimage: Warning: no Apple/Unix files will be decoded/mapped
Total translation table size: 0
Total rockridge attributes bytes: 1117
Total directory bytes: 6144
Path table size(bytes): 60
Max brk space used 22000
245 extents written (0 MB)
Packaging msi for i386 into packages/v0.0.2
genisoimage: Warning: no Apple/Unix files will be decoded/mapped

@toch It'd be nice to fix these warnings.

Also, I think we should use the same output format as release so instead of:

Packaging deb for x86_64 into packages/v0.0.2

We'd have:

Writing packages/v0.0.2/mruby-cli-0.0.2-ARCH.deb

It also seems like the ARCH is missing from the output, so I'd like to fix that as well.

toch · 2015-09-17T21:25:08Z

@zzak

Also found this blog post, that might be interesting:
http://blog.codeship.com/using-docker-build-debian-packages/

I'll read the blog post.

It'd be nice to fix these warnings.

do you mean to no show them?

for genisoimage, this is just information.
for msitools, the software is not yet very mature, it seems that those warnings could happen and still you have a msi generated (as in our case).

Also, I think we should use the same output format as release

I'll fix that.

toch · 2015-09-19T12:15:52Z

@zzak update: better output is available toch/mruby-cli-distpackage@b99cd46

toch · 2015-09-19T13:13:51Z

@zzak I've rebased onto subdirs and I've fixed the warning cf toch/mruby-cli-distpackage@b0801be

Do you see anything else to do?

toch · 2015-10-01T09:19:56Z

@hone @zzak

I've rebased onto the PR #13 and squashed the commits. Before merging it, that PR hone/mruby-cli-docker#1 should be merged first into hone/mruby-cli-docker

Two questions:

1/ should I put the code of toch/mruby-cli-distpackage here?

2/ According to mrblib/mruby-cli/setup.rb, the task release is not yet available when you setup your project with mruby-cli. Without that, I can't right now make the package task available neither. I suggest to implement that separately. What do you think?

zzak · 2015-10-20T06:28:23Z

the task release is not yet available when you setup your project with mruby-cli

Right, we can keep this task private as well, until we're happy with it.

hone · 2015-10-21T23:36:13Z

We can make packaging available to users of mruby-cli vs devs a separate PR for sure.

toch · 2015-11-04T15:25:00Z

following a call I had with @zzak, here a summary of the things to decide and some other informations

current situation:

deb and rpm work
dmg works but a script should be run via shell to add the binary to the PATH
msi works but nothing to set up the PATH

dmg

I do not know how to include a install action into the bundle such as linking the binary into /usr/local/bin what we would like to do.

some apps do that when you run it the first time (e.g. atom), we should find someone who knows it

msi

with the current msitools, the directive Environment is not recognized. It's then impossible to set it up during the installation.

I've contacted the maintainer to understand how to support it

end goal:

the user shouldn't do something more than just install the package

Tasks to do

merge https://github.com/toch/mruby-cli-docker/blob/master/Dockerfile into upstream and build the stack image
bring back the code of https://github.com/toch/mruby-cli-distpackage

To decide

we should decide if:

we already merge it as it is and agree to postpone the automated installation of the binary in the path environment for osx and windows, or
we wait for having fix it the

hone · 2015-11-04T20:51:20Z

talked with @toch. I'm fine merging as is for mruby-cli itself. Let's make sure to make issues for fixing the MSI/DMG path stuff so it's tracked. We should probably wait on making all of this generally available (at least MSI/DMG) downstream of mruby-cli.

zzak · 2015-11-05T02:56:23Z

Ok, let's clean this up and get it merged/shipped so we can start working on the next steps.

toch · 2015-11-17T23:24:31Z

done

* release task depends on compile task * check the presence of toolchain before running the compile task * remove superfluoous load of mrbtest.rake

* abort the task if not run in-docker * env var MRUBY_CLI_LOCAL allows to run the rake task locally * local task can be added * refactoring of the check is-in-a-docker-container?

it seems that the options weren't correctly passed to the command.

* documentation about package task * it adds a packaging task * it builds package for deb, rpm, msi, and dmg

toch · 2015-11-18T18:55:41Z

rebased onto master to resolve conflict

hone · 2015-11-20T20:45:08Z

merged here c2ef3f2

zzak mentioned this pull request Aug 21, 2015

Installer packages #17

Open

toch force-pushed the create-deb-rpm-packages branch from 4cc2880 to 10a14f8 Compare September 12, 2015 18:05

toch force-pushed the create-deb-rpm-packages branch from a134078 to b9898f6 Compare September 19, 2015 13:12

toch force-pushed the create-deb-rpm-packages branch from b9898f6 to cc1d33a Compare October 1, 2015 09:07

toch force-pushed the create-deb-rpm-packages branch 2 times, most recently from a45019a to 71430f3 Compare November 17, 2015 23:20

This was referenced Nov 17, 2015

osx package should make available mruby-cli in user PATH #28

Open

windows package should make available mruby-cli in user PATH #29

Open

toch added 5 commits November 18, 2015 19:45

Improvement of the rake task compile

6ef51e5

* release task depends on compile task * check the presence of toolchain before running the compile task * remove superfluoous load of mrbtest.rake

By default, rake task has to be run in-docker

a4d2651

* abort the task if not run in-docker * env var MRUBY_CLI_LOCAL allows to run the rake task locally * local task can be added * refactoring of the check is-in-a-docker-container?

Fix broken bintests

b949478

it seems that the options weren't correctly passed to the command.

Fix buggy path on windows when running bintest

f240c09

A task package mruby-cli

1c365be

* documentation about package task * it adds a packaging task * it builds package for deb, rpm, msi, and dmg

toch force-pushed the create-deb-rpm-packages branch from 71430f3 to 1c365be Compare November 18, 2015 18:54

hone added a commit that referenced this pull request Nov 20, 2015

Merge pull request #10 from toch/create-deb-rpm-packages

c2ef3f2

hone closed this Nov 20, 2015

A task to package mruby-cli as deb or rpm #10

A task to package mruby-cli as deb or rpm #10

Conversation

toch commented Aug 2, 2015

zzak commented Aug 2, 2015

hone commented Aug 3, 2015

zzak commented Aug 4, 2015

toch commented Aug 4, 2015

zzak commented Aug 7, 2015

toch commented Aug 11, 2015

zzak commented Aug 12, 2015

toch commented Aug 15, 2015

zzak commented Aug 16, 2015

toch commented Aug 24, 2015

toch commented Aug 24, 2015

toch commented Aug 24, 2015

zzak commented Aug 24, 2015

toch commented Aug 24, 2015

zzak commented Aug 24, 2015

toch commented Aug 24, 2015

zzak commented Aug 24, 2015

toch commented Aug 24, 2015

toch commented Aug 24, 2015

zzak commented Aug 24, 2015

toch commented Aug 25, 2015

zzak commented Aug 25, 2015

toch commented Sep 5, 2015

zzak commented Sep 6, 2015

toch commented Sep 7, 2015

toch commented Sep 12, 2015

zzak commented Sep 17, 2015

zzak commented Sep 17, 2015

toch commented Sep 17, 2015

toch commented Sep 19, 2015

toch commented Sep 19, 2015

toch commented Oct 1, 2015

zzak commented Oct 20, 2015

hone commented Oct 21, 2015

toch commented Nov 4, 2015

current situation:

dmg

msi

end goal:

Tasks to do

To decide

hone commented Nov 4, 2015

zzak commented Nov 5, 2015

toch commented Nov 17, 2015

toch commented Nov 18, 2015

hone commented Nov 20, 2015