Fail to build scikit-learn. tests fail.

[timsears]

Issue description

Using scikitlearn in a nix shell. A test fails but it looks like a bug in the test code. The end of the build messages appears below. It actuall looks ok with small formatting differences. Maybe it's possible to disable some or all of the tests, Not sure how to do that.

File "/nix/store/iwqpvl1v6shi8wcdh6kcq46mp2qqcm7l-python3.6-scikit-learn-0.19.1/lib/python3.6/site-packages/sklearn/utils/__init__.py", line 327, in sklearn.utils.shuffle
Failed example:
    X_sparse.toarray()
Expected:
    array([[ 0.,  0.],
           [ 2.,  1.],
           [ 1.,  0.]])
Got:
    array([[0., 0.],
           [2., 1.],
           [1., 0.]])

>>  raise self.failureException(self.format_failure(<_io.StringIO object at 0x7fffd0465708>.getvalue()))


----------------------------------------------------------------------
Ran 8229 tests in 329.068s

FAILED (SKIP=62, failures=64)
builder for '/nix/store/vqvaypqx7vxv0qvi17dn8fm3yywd7gd6-python3.6-scikit-learn-0.19.1.drv' failed with exit code 1
error: build of '/nix/store/ml9p0q3gqa6g41af486b3ba81zl0lbhz-python3.6-tensorflow-1.5.0.drv', '/nix/store/vqvaypqx7vxv0qvi17dn8fm3yywd7gd6-python3.6-scikit-learn-0.19.1.drv' failed

Steps to reproduce

1. Enter the following in shell.nix and run nix-shell

{ pkgs ?
(import <nixpkgs>) {} }: 
with pkgs;

stdenv.mkDerivation {
  name = "MachineLearningEnv";
  buildInputs = [

    ]
    ++ (with python36Packages; [
    scikitlearn
    ]);
}

Technical details

• system: "x86_64-linux"
• host os: Linux 4.9.77, NixOS, 18.03.git.5402412 (Impala)
• multi-user?: yes
• sandbox: no
• version: nix-env (Nix) 1.12pre5849_74f75c85
• channels(root): ""
• channels(timsears): ""
• nixpkgs: /home/timsears/nixpkgs

commit is e9fdcc9
(a recent commit on the master branch)

[rvl]

As a workaround, try using scikitlearn.overridePythonAttrs (oldAttrs: { checkPhase = ""; }) (manual link).

To fix it, possibly we could try updating the upstream version or deleting the test. I've not tested it yet (still building gfortran and numpy for some reason).

[timsears]

@rvl Thanks for the tip. That produces a different error so I can't supply an empty checkPhase. However I noticed that the darwin version skips the tests with this so I will hack that string to make the skipping non-optional.

# Disable doctests on OSX: https://github.com/scikit-learn/scikit-learn/issues/10213
  checkPhase = ''
    HOME=$TMPDIR OMP_NUM_THREADS=1 nosetests ${stdenv.lib.optionalString stdenv.isDarwin "--doctest-options=+SKIP"} $out/${python.sitePackages}/sklearn/
  '';

[timsears]

That worked. Just an ugly workaround however.

Specifically, changing shell.nix to this...

{ pkgs ?
(import <nixpkgs>) {} }:
with pkgs;

stdenv.mkDerivation {
  name = "MachineLearningEnv";
  buildInputs = [

    ]
    ++ (with python36Packages; [
    (scikitlearn.overridePythonAttrs (oldAttrs: {
    checkPhase ="HOME=$TMPDIR OMP_NUM_THREADS=1 nosetests --doc\
test-options=+SKIP $out/${python.sitePackages}/sklearn/" ;}
      )
    )
    ]);

}

[rvl]

Hi, glad to hear you're able to use the sklearn package now.

My mistake with an empty checkPhase. Overriding checkPhase = "true"; would work however.

I had a closer look at the text of the test failure message. Firstly, it is a doctest not a normal test. Secondly, the only difference is whitespace. Apparently, numpy have changed their array __repr__ formatting in 1.14. And we have recently updated numpy in commit 9bb18f6.

I think the best course of action would be to make a nixpkgs pull request disabling scikitlearn doctests unconditionally (i.e. not just darwin), like you have done. Then raise an issue with scikit-learn. Ideally we should run doctests. Maybe there is a better way of invoking the tests.

[ixxie]

Perhaps @FRidh could chime in on this? I could make the fix if they agree.

[FRidh]

I think the best course of action would be to make a nixpkgs pull request disabling scikitlearn doctests unconditionally (i.e. not just darwin), like you have done. Then raise an issue with scikit-learn. Ideally we should run doctests. Maybe there is a better way of invoking the tests.

Seems a good solution. Ideally it would be the other way around (first raising an issue with scikit-learn and then fixing it here). Note that some things were reverted in numpy 1.14.1 which should end up in staging (currently on python-unstable branch) as soon as Hydra is up and running again.

[ashgillman]

FYI https://hydra.nixos.org/build/70681936

[dotlambda]

Did someone raise an issue over at scikit-learn yet? I included a commit that disables doctests in #37250.

[CMCDragonkai]

I can't yet change my commit hash, is there a way to do this #35436 (comment) but with a package that depends on scikitlearn? That is, I'm using imbalanced-learn which relies on scikitlearn and this error prevents imbalanced-learn from building. What is the way to override the checkPhase in this situation?

---

let
  scikitlearn = python35.pkgs.scikitlearn.overridePythonAttrs (oldAttrs: {
    checkPhase = ''
      HOME=$TMPDIR OMP_NUM_THREADS=1 nosetests --doctest-options=+SKIP $out/${python35.sitePackages}/sklearn/
    '';
  });
in
  (imbalanced-learn.override { scikitlearn = scikitlearn; })

I tried the above, but it didn't work.

[rvl]

Maybe try something like this:

with import <nixpkgs> {};

let
  packageOverrides = self: super: {
    scikitlearn = super.scikitlearn.overridePythonAttrs (oldAttrs: { checkPhase = "true"; });
  };
  pythonPackages = (pkgs.python36.override {inherit packageOverrides;}).withPackages (ps: [ps.imbalanced-learn]);

in
  pythonPackages.env  # env is for a nix-shell

Ref: https://nixos.org/nixpkgs/manual/#how-to-override-a-python-package