Issues in trying to make a Scala environment

[andreaferretti]

I am trying to make a simple container for Scala development. The usual process of installing Scala consists of

• having a JVM installed
• downloading a script, such as https://github.com/paulp/sbt-extras that in turn
• downloads the Jar of SBT (the Scala package manager), that in turn
• downloads the Scala compiler and all the dependencies

This seems a little contrived, but allows to change everything (libraries, the version of Scala, the version of SBT itself) from a simple configuration file per project. The only thing that is needed to bootstrap everything is a small script.

So, here is my tentative vagga.yaml:

containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, curl]
  sbt:
    setup:
    - !Container jvm
    - !Sh "curl -s https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt > sbt"
    - !Sh "chmod +x sbt"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
commands:
  scala: !Command
    container: sbt
    run: ./sbt -sbt-create console
  sh: !Command
    container: sbt
    run: bash

When I try to run sbt -sbt-create or anything that starts the bootstrap process, the sbt launcher tries to download a few jars inside /tmp, which is why I have the temporary volume. Unfortunately, I get the following error:

mkdir: cannot create directory '/.sbt': Read-only file system
Downloading sbt launcher for 0.13.9:
  From  http://repo.typesafe.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.9/sbt-launch.jar
    To  /tmp/sbt_extras_launchers.dzPs5H/0.13.9/sbt-launch.jar
java.io.IOException: No such file or directory
    at java.io.UnixFileSystem.createFileExclusively(Native Method)

It probably has to do with the fact that sbt expects to be able to write its configuration under $HOME/.sbt, and packages under $HOME/.ivy. I have tried a few settings, but I was not able to make this work.

Is it possible to bind the home directory in some writable dir?

[tailhook]

Yes. I usually "fix" the passwd file:

    - !Sh |
        sed -i '/^root/{s@/root@/work/tmp@;}' /etc/passwd

But alternative is too bind a volume:

     volumes:
       /root: !BindRW /work/tmp

Both settings will put home of the (pseudo-) root user into the "tmp" subdir of your project directory.

[lilianmoraru]

mkdir: cannot create directory '/.sbt': Read-only file system
It looks like it tries to create the folder .sbt into the root /.

sbt expects to be able to write its configuration under $HOME/.sbt - vagga doesn't set HOME by default. So, $HOME expands to nothing and you get /.sbt

Is it possible to bind the home directory in some writable dir? - yes.
What you need is environ

commands:
  scala: !Command
    container: sbt
    environ: { HOME: /root }
    run: ./sbt -sbt-create console

Or if you want to be able to write in root /(which I guess wasn't what you tried to do):

commands:
  scala: !Command
    container: sbt
    write-mode: transient-hard-link-copy
    run: ./sbt -sbt-create console

[tailhook]

@lilianmoraru the problem is that java uses /etc/passwd directly (or maybe via libc) instead of looking in the environment. So changing HOME doesn't help indeed.

[andreaferretti]

Thank you for your help! I tried settings $HOME, but in fact it did not help. Writing /etc/passwd seems to work! Instead, the volume trick seemed to have no effect.

I am now almost done, I have a little issue with SSL, but I will post here a working Scala config as soon as it's ready

[andreaferretti]

A working vagga.yaml for scala is:

containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, curl]
    - !Sh "update-ca-certificates -f"
  sbt:
    setup:
    - !Container jvm
    - !Sh "curl -s https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt > sbt"
    - !Sh "chmod +x sbt"
    - !Sh "sed -i '/^root/{s@/root@/work/tmp@;}' /etc/passwd"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
commands:
  scala: !Command
    container: sbt
    run: ./sbt -sbt-create console
  sh: !Command
    container: sbt
    run: bash

[tailhook]

@andreaferretti nice. Could I steal it to examples? Or maybe you'd like to make a pull request?

[lilianmoraru]

@andreaferretti Another way I solve my issues with the certificates is to install the package ca-certificates.
Example:

- !Install [ca-certificates]
# Or only at container build
- !BuildDeps [ca-certificates]

[andreaferretti]

@tailhook Sure, use it however you prefer! If you want me to make a PR, just ask

@lilianmoraru Yeah, I confirm that installing ca-certificates works as well

[andreaferretti]

Closing this, as everything is working fine. The latest version I have looks like this:

containers:
  jvm:
    setup:
    - !UbuntuRelease { version: 15.04 }
    - !UbuntuUniverse
    - !Install [openjdk-8-jre-headless, ca-certificates]
  sbt:
    setup:
    - !Container jvm
    - !Install [curl]
    - !EnsureDir /work/.home/sbt
    - !Download
      url: https://raw.githubusercontent.com/paulp/sbt-extras/master/sbt
      path: /work/.home/sbt/sbt
      mode: 0o744
    - !Sh "sed -i '/^root/{s@/root@/work/.home@;}' /etc/passwd"
    volumes:
      /tmp: !Tmpfs
      size: 100Mi
      mode: 0o1777
    environ:
      PATH: "/work/.home/sbt:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
      HOME: /work/.home
commands:
  scala: !Command
    container: sbt
    run: sbt -sbt-create console
    description: Run a scala console
  sh: !Command
    container: sbt
    run: bash
    description: Enter a shell with sbt available

Thanks everyone for the help!

[andreaferretti]

By the way, a few more examples