opam-mkbundle

[UnixJunkie]

This is a request from @edwintorok that landed in my inbox.
Please ask him for more details.

https://github.com/edwintorok

opam-mkbundle package_name
would create a bundle of source archives and a script to build it all, depending only on the standard OCaml compiler, and then the user could get a source package with everything in it.

He his willing to distribute some software in source code without
the need to ask users to install OPAM.

[edwintorok]

In the C/C++ world there is a tendency to ship applications with embedded copies of libraries (just see Qt, Firefox, and so on). While that is in general frowned upon (those embedded libraries are often out-of-date, make it hard to do security updates, etc.) they do provide one advantage to the end-user: convenience.
They don't need to learn how to download/compile and install all the dependencies of the application, they can just build/install the application directly.

Of course relying only on system packages has its downsides too: it might be the system package that is out of date (zlib on old Solaris for example), or there might be no system package available, or there is a patch needed and upstream hasn't made a new release yet, etc.

My opam-mkbundle idea tries to provide just the advantages (convenience, installation without downloading packages from the internet), without the disadvantages:

• distributions can still package only the standalone version of the application, and use their own packages as dependencies, they don't have to use the bundle
• end-users who know nothing about OCaml and OPAM can still be given simple instructions that they are used to for other software: ./configure && make && make install
• it could be that the OS/arch combination of the user is not yet supported by OPAM, but would be supported by the bundle
  • if there is an issue with a package dependency a patch can easily be provided to the user for testing, without complicated manual patching/reinstallation of dependencies
  • the bundle should check for presence of C library dependencies (via pkg-config, or by trying to run an autoconf-like test)

The part about relying just on the OCaml compiler is because its quite easy to seriously mess up an OCaml installation, beyond what an end user is expected to debug.
For example: old CentOS/RHEL provides OCaml 3.09 + findlib package. There is a 3rd-party repository which provides a newer version of OCaml (IIRC 3.12 or 4.00), but not findlib. So you end up with OCaml 3.12 and findlib 3.09, which obviously won't work.
If you have a bundle, then user only needs to ensure one thing: install a recent enough OCaml. The bundle's configure script can detect if that wasn't done and give a helpful error message.

[AltGr]

Interesting idea. This is related to #1059 (actually a large superset of it)

[UnixJunkie]

Is there some progress on this feature?
I know some authors of ocaml code that are interested in creating automatically self-contained installers of their software without tying their users to opam.

[AltGr]

Not exactly the same (it doesn't include build scripts), and still a bit complex, but #2098 may be a step in this direction.
Note that you'll need to assume the target architecture is the same (or all packages are equally available); also, extracting portable build scripts may be difficult when OPAM variables are used -- we would need to write (shell?) code to dynamically evaluate them.
Another, easier solution would be an embeddable OPAM , tied to a local, included, minimal repository. But that would require including the build of OPAM itself...

I would gladly give pointers to anyone willing to contribute on this: that would be a nice feature to have, but I don't have time at the moment -- the priority for 1.3 having been set to signing our repository.

[UnixJunkie]

Apparently there is a tool in the Haskell ecosystem doing this, it is called stack:
https://github.com/commercialhaskell/stack.
Maybe it can be used as a source of inspiration.

[UnixJunkie]

Would it be possible to set a bounty on this feature?
Maybe many users are interested and would contribute.

[UnixJunkie]

Dear Santa, for Christmas, I would like this feature. :-)

[ygrek]

ftr https://github.com/ygrek/opam-bundle

[samoht]

Tentative doc: https://gist.github.com/samoht/54c5aaa7429a68d27fbe66515bea1c7b

In that doc, we still need opam to import/build bundles (even if they are self-contained) but maybe we can also distribute a small script which just build a precise bundle.

[UnixJunkie]

Is this implemented?
That would be a nice way to ship ocaml software software to clients.
Should we setup some crowdfunding to get this feature sponsored?

[AltGr]

This is not yet implemented, although the new opam admin command offers some of the required pieces (e.g. opam admin filter --resolve foo.1 && opam admin cache will filter a repository to keep only what is required to install foo.1, and put all archives in a local cache).

The feature is already planned and funded, but thank you!

[dra27]

Addressed (at least in part) by #2944

[AltGr]

Addressed by https://github.com/AltGr/opam-bundle
This is slightly different in that even ocaml is not required and will be bootstrapped, as well as opam. So while the end goal is achieved, the bundle may be a little bit heavier than what was initially requested.