OPAM in a box #1035

Open
avsm opened this Issue Dec 13, 2013 · 27 comments

6 participants

@avsm
OCaml member

This is a tracker thread to pull together the various scripts and tools required to generate an "OPAM in a box" suitable for teaching purposes. It should result in:

  • A base OCaml installation of 4.01.0 (available via Ubuntu PPA in avsm/ppa)
  • an OPAM installation with a user-configured set of packages all pre-installed, including any external library dependencies (e.g. libpcre). (see https://github.com/AltGr/docker-opam)
  • An .ocamlinit file preconfigured to require all the libraries.
  • The utop toplevel.
  • The merlin tool configured to work with Emacs and Vim.
  • An opam-doc-generated set of cross-referenced HTML docs for all the installed packages, served via a localhost webserver.
  • A local HTTP remote of all the archives, so that it can be used without requiring external net access. (OPAM 1.1.1dev has opam-admin make --resolve <pkgs> to generate this)
  • an iPython/OCaml console running (ocaml/opam-repository#1690)

For a first milestone, we're going to pull this together using Ubuntu and shell scripting, and wrap it for release using Docker containers (see http://anil.recoil.org/2013/11/15/docker-and-opam.html for an overview of this). When this works satisfactorily, we can distribute the Ubuntu+Docker binaries in a VM for those folk using Windows as a second step.

This thread arose from a discussion with various profs and @yminsky, and a followup meeting with @AltGr and @samoht. We'll use this issue to track progress of the various steps above.

@avsm
OCaml member

For opam-doc, see https://github.com/ocamllabs/opam-doc -- that repository takes a package set and generates documentation suitable for serving from a localhost webserver. Cohttp-0.9.14 installs a cohttp-server which simply serves local files over HTTP, so that can be used to serve the documentation too.

@samoht
OCaml member

A minor remark: using 4.01.0 as system compiler will not work with opam-doc (see ocamllabs/opam-doc#68)

@avsm
OCaml member

Good point; since we're going to be redistributing binaries, we might as well just compile up a custom switch for 4.01.0 anyway. This could also include the frame-pointer to have better debugging, since this particular one wouldn't be for performance-critical code.

@AltGr
OCaml member

The latest version of https://github.com/AltGr/docker-opam provides a straight-forward make.sh script that takes a list of packages and:

  • builds a base docker image with OPAM (note, we could export that to the docker repo)
  • creates a git mirror of the opam repository containing the needed package archives (so that no further access to the individual packages URLs is needed from the docker container ; that step could be skipped though). Note that we don't mirror the depexts (yet).
  • installs the required OPAM packages (together with their depexts)
@AltGr
OCaml member

The second point above uses the new opam-admin make --resolve <packages> to mirror the selected packages and all their dependencies, at the best version possible.

@jnfoster

[Cc @bennn @mdgeorge]

So is this happening? I'm at the point where I need to finalize the VM for spring 2013 CS 3110 at Cornell very soon.

Thanks,
Nate

@avsm
OCaml member

It is -- I did some cleanups of the opam-doc code over the vacation, and one version is up here: http://mirage.github.io. There are still a number of layout issues, and the front page needs to be templated (so that you don't just get a raw dump of modules, but a better front page).

So, if you don't mind being on the bleeding edge of some of this, I can create an Ubuntu VM and create the Docker images. Your best VM choice is likely to be Ubuntu Saucy x86_64, as that'll run Docker out of the box.

@avsm
OCaml member

Actually, I'm going to give Vagrant a try after seeing @reitblatt's commit in frenetic-lang/frenetic#202. It seems pretty portable and easy to script too, whereas Docker is Linux-only at present.

@jnfoster
@avsm avsm referenced this issue in ocamllabs/opam-doc Dec 30, 2013
Merged

script improvements #90

@avsm
OCaml member

I've got a Vagrant repository at https://github.com/avsm/vagrant-opam that serves docs, installs Vim and utop/merlin.

Can any Emacs users help me out with a script for that too?

@samoht
OCaml member

Hadn't had much time to test (the compilation time of the whole thing is still a bit high), but I think avsm/vagrant-opam#1 is working (at least for my settings).

Feel free to send a PR on https://github.com/samoht/ocaml-emacs-settings to improve the emacs mode.

@avsm
OCaml member

We have everything except a local remote now, which requires OPAM 1.1.1 -- I'll create some PPA snapshots when I get back to Cambridge later in the week.

@jnfoster
@avsm
OCaml member

I reckon this should be the kitchen sink edition with everything in. If you could send a pull req with a setup-eclipse.sh script, I'm happy to try it out. I'd like to publish this one on the RWO site as well, and several people asked about Eclipse.

@avsm
OCaml member

@samoht, I see lots of errors on the Vagrant build with the emacs script. It completes successfully, so just checking if they're important or not.

In toplevel form:
auto-complete-config.el:31:1:Error: Cannot open load file: popup
Compiling /home/vagrant/.emacs.d/elpa/auto-complete-20131128.233/auto-complete-pkg.el...
Wrote /home/vagrant/.emacs.d/elpa/auto-complete-20131128.233/auto-complete-pkg.elc
Compiling /home/vagrant/.emacs.d/elpa/auto-complete-20131128.233/auto-complete.el...

In toplevel form:
auto-complete.el:51:1:Error: Cannot open load file: popup
Checking /home/vagrant/.emacs.d/elpa/auto-complete-20131128.233/dict...
Done (Total of 1 file compiled, 2 failed, 1 skipped)
@samoht
OCaml member

Hum, seems that popup is not correctly installed. I'll add it to the package list.

@AltGr
OCaml member

@samoht hmm yes, if you're using apt-get, popup.el is part of auto-complete-el (http://packages.ubuntu.com/saucy/all/auto-complete-el/filelist) -- so I think this should be alright ; but using the new emacs package system that may not be the case.
I am now testing using apt-get. Any reason why we may prefer one way or the other ?

@yminsky

I'm curious what the status of this is? Is there something in a good enough state for Cornell to use? Has anyone reached out to the other schools on the email thread?

@AltGr
OCaml member

The general setup is working, but the editor configuration isn't very nice yet. I am trying to have better emacs support, and ocp-indent support in vim.

I also found that you can run into trouble with the vagrant VM when you have a locale set, which is just a matter of running 'LC_ALL=C vagrant ssh'.

@avsm
OCaml member
@lefessan

Do you have a deadline from Cornell, or other universities, to know at least when the first VM should be shipped ? It would allow us to schedule the important work first, leaving polishing for later.

@jnfoster
@avsm
OCaml member

@jnfoster, I'll start publishing binary images before Eclipse is in; it can just be rebuilt when the pull request shows up. @AltGr, if you can branch OPAM 1.1.1, then I can add support for a local package repository, which needs the latest OPAM I believe.

@avsm
OCaml member

I've pushed a number of cleanups and will start publishing binary packages of the boxes tomorrow. @jnfoster, any luck with the Eclipse pull?

The most important part has also been done: the ascii art! avsm/vagrant-opam@0ba2974

@jnfoster
@avsm
OCaml member
@avsm avsm referenced this issue in mirage/mirage Feb 17, 2014
Closed

publish binary vagrant images #224

@avsm
OCaml member

I've also updated the checklist to note that we have an awesome IPython/OCaml interactive console now, so we could include that in the default binary image when it's merged into OPAM trunk.

@AltGr AltGr modified the milestone: outgoing, 1.2 Jun 27, 2014
@AltGr AltGr modified the milestone: 1.3, Next Nov 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment