Failing tests in isolated GNU Guix build

[pjotrp]

After successfully including Erlang in GNU Guix (the GNU package manager) I started work on including Elixir.

11 tests in Elixir and 33 tests in Mix fail when building in an isolated environment (i.e. with no access to $HOME, /usr/bin) and without network. We enforce this environment for GNU Guix reproducible installs. No outside inputs can interfere with the build process - i.e., all dependencies need to be explicit.

To have the tests pass I removed two test files:

             (delete-file "./lib/mix/test/mix/tasks/deps.git_test.exs")
             (delete-file "./lib/mix/test/mix/shell_test.exs")

and skipped a few others with these patches: https://github.com/genenetwork/guix-bioinformatics/blob/elixir/elixir-disable-failing-tests.patch and https://github.com/genenetwork/guix-bioinformatics/blob/elixir/elixir-disable-mix-tests.patch. In the Mix patches I switched the reference to git into a full path. Note that the debugger works too.

I wrote it all up in https://github.com/pjotrp/guix-notes/blob/master/ELIXIR.org and the package with these tests disabled has been queued for inclusion in GNU Guix.

Can we work towards making these tests independent of the underlying distribution? This will greatly help reliable reproducible optimized deployment of Elixir and Mix and should make it easier to package Elixir for other environments.

[josevalim]

Awesome work @pjotrp!

I would love to include patches that generally make Mix or Elixir work better but those seem like they would be putting a maintenance burden on us, specially because we don't really know what is available on GNU Guix and what is not. We would be guessing, specially when some of those outcomes is exactly what we are testing (like handling of the HOME environment variable).

Can't we rather try figure out if we can set a reliable PATH for running those tests suites so we can use dialyzer and git as usual?

[pjotrp]

@josevalim thanks for responding!

We can set a $HOME in the package and patch in paths to calls to git, echo etc. I'll address those in the coming month. There may be a number of tests I need help with. Three questions:

Q1: Would it be possible to have all calls out to git in one single function? I think it is useful to make such dependencies explicit.

Q2: Is there any git networking involved? I did not see it in the tests, but just making sure.

Q3: Do you have any idea why the build failed when I patched the test files? It is very strange. See https://github.com/pjotrp/guix-notes/blob/master/ELIXIR.org#fighting-with-elixir-build

[josevalim]

Q1: It should be two: one in the code and another one for the tests.

Q2: No, it is not.

Q3: No, I took a look and I couldn't see why they would be related if you are not changing the source files (and only test ones).

[pjotrp]

We are pleased to announce a recent reproducible build of Elixir+Erlang and debugger is now part of GNU Guix: https://www.gnu.org/software/guix/packages/

The source of the package can be found at http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/elixir.scm#n31

And the current build and dependencies can be viewed at http://hydra.gnu.org/build/1375771#tabs-runtime-deps

Thanks to Ricardo Wurmus we found the answer to Q3 above. I.e., that the Elixir compiler checks whether or not to compile files by inspecting their timestamps. When the timestamp is equal to the epoch no compilation will be performed. Some tests fail when files are older than Jan 1, 2000.

Some of the tests are still skipped. This is a small minority, but I hope we can get them fixed over time.

@josevalim Are you interested in adding GNU Guix to the Elixir installation instructions http://elixir-lang.org/install.html?

I think the coolest language goes rather well with the coolest package system by the GNU project. Both projects give me great peace of mind in developing production systems.

[josevalim]

@josevalim Are you interested in adding GNU Guix to the Elixir installation instructions http://elixir-lang.org/install.html?

Yes, please. Just send a PR here: github.com/elixir-lang/elixir-lang.github.com

And thank you for your work! Shall we close this issue now or is there anything else I can help with?