Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: fix a set of test errors in master. #7639

Merged
merged 4 commits into from
May 18, 2016

Conversation

rgommers
Copy link
Member

These are the ones I ran into on 32-bit Linux with current master and Pathlib installed on Python 2.7:

======================================================================
ERROR: test_format.test_large_archive
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/.local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_format.py", line 841, in test_large_archive
    a = np.empty((2 ** 30, 2), dtype=np.uint8)
ValueError: array is too big; `arr.size * arr.dtype.itemsize` is larger than the maximum possible size.

======================================================================
ERROR: test_mafromtxt (test_io.TestPathUsage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/Code/numpy/numpy/testing/decorators.py", line 147, in skipper_func
    return f(*args, **kwargs)
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_io.py", line 1899, in test_mafromtxt
    f.write('1,2,3.0\n4,5,6.0\n')
TypeError: must be unicode, not str

======================================================================
ERROR: test_ndfromtxt (test_io.TestPathUsage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/Code/numpy/numpy/testing/decorators.py", line 147, in skipper_func
    return f(*args, **kwargs)
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_io.py", line 1888, in test_ndfromtxt
    f.write('1 2\n3 4')
TypeError: must be unicode, not str

======================================================================
ERROR: test_recfromcsv (test_io.TestPathUsage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/Code/numpy/numpy/testing/decorators.py", line 147, in skipper_func
    return f(*args, **kwargs)
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_io.py", line 1923, in test_recfromcsv
    f.write('A,B\n0,1\n2,3')
TypeError: must be unicode, not str

======================================================================
ERROR: test_recfromtxt (test_io.TestPathUsage)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/Code/numpy/numpy/testing/decorators.py", line 147, in skipper_func
    return f(*args, **kwargs)
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_io.py", line 1910, in test_recfromtxt
    f.write('A,B\n0,1\n2,3')
TypeError: must be unicode, not str

======================================================================
ERROR: test_all_outliers (test_twodim_base.TestHistogram2d)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/rgommers/Code/numpy/numpy/lib/tests/test_twodim_base.py", line 257, in test_all_outliers
    r = rand(100) + 1. + 1e6  # histogramdd rounds by decimal=6
  File "/home/rgommers/Code/numpy/numpy/lib/utils.py", line 99, in newfunc
    warnings.warn(depdoc, DeprecationWarning)
DeprecationWarning: `rand` is deprecated!
numpy.testing.rand is deprecated in numpy 1.11. Use numpy.random.rand instead.

----------------------------------------------------------------------

Pathlib was only added to the stdlib in Python 3.4, and there it
can handle string type input for a file with UTF8 encoding.
But the Patlib version on PyPi that can be installed for Python 2.7
doesn't accept strings, gives errors like:

    ======================================================================
    ERROR: test_ndfromtxt (test_io.TestPathUsage)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/rgommers/Code/numpy/numpy/testing/decorators.py", line 147, in skipper_func
        return f(*args, **kwargs)
      File "/home/rgommers/Code/numpy/numpy/lib/tests/test_io.py", line 1888, in test_ndfromtxt
        f.write('1 2\n3 4')
    TypeError: must be unicode, not str
@@ -838,7 +838,11 @@ def test_large_file_support():

@dec.slow
def test_large_archive():
a = np.empty((2 ** 30, 2), dtype=np.uint8)
try:
a = np.empty((2**26, 2), dtype=np.uint8)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is this big enough? The test is for

    BUG: Cast size to int64 when loading from archive

    Prevents overflow errors for large arrays on systems
    where the default int type is int32.

IIRC, the fix was for the product of dimensions that overflowed, so the tested array should probably be multidimensional with total size >= 2**31

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking a bit more, this really only needs to be run on 64 bit windows systems.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, a comment on the test intent would have been helpful.....

Indeed, needs to be skipped on 32-bit systems then. For the original size I get:

In [1]: a = np.empty((2**30, 2), dtype=np.uint8)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-d8d46384be90> in <module>()
----> 1 a = np.empty((2**30, 2), dtype=np.uint8)

ValueError: array is too big.

Will add the skip, but keep the except MemoryError - that's always good when trying to allocate such large arrays.

@rgommers
Copy link
Member Author

Updated. Also marked two tests as slow, for me np.lib.test() now runs in 8s instead of 45s.

@charris charris merged commit cd96464 into numpy:master May 18, 2016
@charris
Copy link
Member

charris commented May 18, 2016

Thanks Ralf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants