nose testing suite as an optional spkg

[jasongrout]

Several projects we depend on use nose to do self-tests. In order to test these packages, we'd have to have nose installed.

This is fairly simple without an spkg:

wget http://pypi.python.org/packages/source/n/nose/nose-1.1.2.tar.gz # or the latest version at http://pypi.python.org/pypi/nose/
tar xzvf nose-1.1.2.tar.gz
cd nose-1.1.2
sage -python setup.py install  

However, it might make sense to have nose be an optional spkg.

Matplotlib relies on nose to do tests (http://matplotlib.sourceforge.net/devel/coding_guide.html#testing), as well as numpy/scipy (http://projects.scipy.org/numpy/wiki/TestingGuidelines).

---

New spkg: version 1.1.2.

Apply attachment: trac_9921-nose-scripts.patch to the scripts repo.

CC: @sagetrac-drkirkby @jhpalmieri

Component: packages: optional

Keywords: sd41

Author: John Palmieri

Reviewer: Karl-Dieter Crisman, Martin Raum

Merged: sage-5.7.beta2

Issue created by migration from https://trac.sagemath.org/ticket/9921

[jasongrout]

comment:1

The current website for nose is: http://code.google.com/p/python-nose/

[sagetrac-drkirkby]

comment:3

IMHO it would be sensible to have nose as a standard package.

I suggested that once, and it got a luke warm reception. Having to install an optional item to test a standard package seems a bad idea to me.

Perhaps if a list of packages which make use of 'nose' could be produced, it might be possible to argue for it to be standard.

Dave

[jasongrout]

comment:4

Yes, I'm thinking (according to guidelines) optional for now, standard after a probationary period.

[sagetrac-drkirkby]

comment:5

Replying to @jasongrout:

Yes, I'm thinking (according to guidelines) optional for now, standard after a probationary period.

Yes, according to guidelines it should be optional for a while. But there have been exceptions to that. Since it does not actually link to anything in Sage, and would only be called when running spkg-check, I think one could argue the risk is minimal. In contrast it would allow a number of packages to be checked.

I think any such argument would have to be based on how many packages could benefit from it.

It must be an incredibly low risk package to add.

Dave

[kcrisman]

comment:6

Whether or not it should be standard, I agree that we need to at least have it optional so that it becomes more 'Sage-ic' (totally not the right adjective, but I'm groping here) to test several fundamental python packages from within Sage. Also, it is needed to test some potential other packages, such as Brian (#9675), which is where I originally found out about this.

[sagetrac-uri]

comment:7

Replying to @kcrisman:

Whether or not it should be standard, I agree that we need to at least have it optional so that it becomes more 'Sage-ic' (totally not the right adjective, but I'm groping here) to test several fundamental python packages from within Sage. Also, it is needed to test some potential other packages, such as Brian (#9675), which is where I originally found out about this.

I installed the package and tried it to test Brian, and it worked perfectly. However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. (see the page Producing New Sage Packages to see how it should be).

[nexttime]

comment:8

Replying to @sagetrac-uri:

[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]

? Of course we first have to produce an spkg, be it optional or (later) standard.

Or did I miss something?

[kcrisman]

comment:9

[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]

? Of course we first have to produce an spkg, be it optional or (later) standard.

Or did I miss something?

No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package. No harm done in reminding us of the official rules, though :)

[nexttime]

comment:10

Replying to @kcrisman:

No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package.

Yeah, an spkg just containing spkg-install which is

wget http://code.google.com/p/python-nose/downloads/detail?name=nose-0.11.3.tar.gz
tar xzvf nose-0.11.3.tar.gz
cd nose-0.11.3
sage -python setup.py install  

:D

[nexttime]

comment:11

Ok, we should concatenate the lines with && to make it more robust...

[sagetrac-uri]

comment:12

Replying to @nexttime:

Replying to @sagetrac-uri:

[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]

? Of course we first have to produce an spkg, be it optional or (later) standard.

Or did I miss something?

Oh, right, I missunderstood... sorry :)

[jasongrout]

comment:13

Replying to @kcrisman:

[...] However, I think some changes should be made in order to fit the structure of Sage packages: all the code should be in a directory named src/, it lacks the .hg folder, the .hgignore and SPKG.txt files, etc. [...]

? Of course we first have to produce an spkg, be it optional or (later) standard.

Or did I miss something?

No, I think that uri was perhaps misunderstanding jason's directions to use nose outside of Sage to be a suggestion for how it would become a Sage package. No harm done in reminding us of the official rules, though :)

Yep, the point of the instructions was to show that even without an spkg, using nose "is fairly simple without an spkg". Of course, you can't download something in the spkg, so those instructions would not work for an spkg. However, it would be a very generic spkg-install file that would basically do the normal standard checks and then run "python setup.py install".

[kcrisman]

comment:14

I tried using nose, and it was very interesting. Tested Brian on two different boxes, and then tested both numpy and scipy on OS X 10.6 - discovered no errors on the first, and a fair number on the second! So I think that there should be no doubt this could go straight to optional if an spkg was made.

I couldn't do

import matplotlib
matplotlib.test()

because for some reason our matplotlib doesn't have this method.

Much less importantly, I should also point out that on OS X wget is not a builtin, though I have an alias curl -O that seems to accomplish the same purpose. I just downloaded and double-clicked, actually :)

[kcrisman]

comment:15

Nose is now on Github: https://github.com/nose-devs/nose.

[jhpalmieri]

comment:17

I've posted an spkg. It builds well enough for me, but I haven't actually used it to test anything except for itself (in spkg-check).

[jhpalmieri]

Author: John Palmieri

[jhpalmieri]

scripts repo

[jhpalmieri]

comment:18

Attachment: trac_9921-nose-scripts.patch.gz

[jhpalmieri]

comment:19

By the way, version 1.1.2 was the most recent version for which I could find a tar ball. Or we could use the version from github; would that be better?

[jhpalmieri]

comment:21

I've now also prepared a version from git. This one fails its own self tests, though ;)

[jhpalmieri]

patch for spkg, for review only

[jhpalmieri]

Attachment: trac_9921-nose-spkg.patch.gz

Attachment: trac_9921-nose-spkg.git.patch.gz

patch for spkg, for review only (version 1.1.3)

[kcrisman]

comment:22

I've now also prepared a version from git. This one fails its own self tests, though ;)

Interesting!

You can try using it with the brian optional spkg, and I believe numpy and mpl also use this?

[jasongrout]

comment:23

ipython also uses nose.

[jhpalmieri]

comment:24

I tried running self-tests with numpy, using python -c 'import numpy; numpy.test()'. The good news: tests ran. The bad news: some tests failed. More bad news: this exited with a return code of zero, so it's hard to tell (within spkg-check, for example), that tests failed. I have the same issue with IPython.

[kcrisman]

comment:25

So is that a problem with nose or a problem with the numpy/ipython test suites? Sounds like nose is ready from this point of view :)

[jhpalmieri]

Changed keywords from none to sd41

[kcrisman]

Reviewer: Karl-Dieter Crisman

[kcrisman]

comment:27

This is ridiculous. Nose works fine at testing things on sage.math. It certainly finds various errors and warnings - apparently scipy generates a number as well, I just tried it.

The issue with it not passing its own tests is not so good, and I can confirm this in both cases. On the plus side, it only breaks the Sage installation process in the 1.1.3, so I would say let's go with the 1.1.2 for now.

---

I do get something weird, hopefully unrelated to nose itself, but instead related to our defaults for matplotlib.

kcrisman@sage:~/sage-5.1.beta1-boxen-x86_64-Linux$ ./sage -c 'import matplotlib; matplotlib.test()'
======================================================================
ERROR: Failure: OSError (No such file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/import matplotlib; matplotlib.test())
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/failure.py", line 39, in runTest
    raise self.exc_class(self.exc_val)
OSError: No such file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/import matplotlib; matplotlib.test()

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

sage: import matplotlib  
sage: matplotlib.test()
======================================================================
ERROR: Failure: ValueError (Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/loader.py", line 213, in loadTestsFromFile
    % filename)
ValueError: Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
False

On a computer without nose:

sage: matplotlib.test()
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)

/Users/karl-dietercrisman/Downloads/sage-5.1.beta6/<ipython console> in <module>()

/Users/.../sage-5.1.beta6/local/lib/python2.7/site-packages/matplotlib/__init__.pyc in test(verbosity)
    986 def test(verbosity=0):
    987     """run the matplotlib test suite"""
--> 988     import nose
    989     import nose.plugins.builtin
    990     from testing.noseclasses import KnownFailure

ImportError: No module named nose

I think that in matplotlib's lib/init.py

    success = nose.run( defaultTest=default_test_modules,
                        config=config,
                        )

we aren't using the right default modules, they aren't imported or something. So it goes back to just looking at . for the default test module - I get the same thing.

sage: nose.run(defaultTest='.')                 
E
======================================================================
ERROR: Failure: ValueError (Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/local/lib/python2.7/site-packages/nose-1.1.2-py2.7.egg/nose/loader.py", line 213, in loadTestsFromFile
    % filename)
ValueError: Unable to load tests from file /home/kcrisman/sage-5.1.beta1-boxen-x86_64-Linux/sage

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)
False

Can you see a reason why this shouldn't have positive review? I just think this must be a problem in how we're dealing with mpl. I'm surprised it does this; we only removed the baseline images, not the testing scripts!

[sagetrac-mraum]

comment:28

I have also reviewed this ticket, since I think it is important to have it. The matplotlib error is really a bug (or at least laxness) in matplotlib. If we waited for nosetest to be included until all spkg's pass, we would wait until the cows come home.

If nobody objects, I will set this to positive review by next week.

[sagetrac-mraum]

Changed reviewer from Karl-Dieter Crisman to Karl-Dieter Crisman, Martin Raum

[haraldschilly]

comment:31

spkg is put on the servers.

[jdemeyer]

Merged: sage-5.7.beta2