Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failures on debian sid #4918

Closed
avalentino opened this issue Sep 21, 2013 · 37 comments
Closed

Test failures on debian sid #4918

avalentino opened this issue Sep 21, 2013 · 37 comments
Labels
Testing pandas testing functions or related to the test suite
Milestone

Comments

@avalentino
Copy link
Contributor

Hi, I'm testing the current pandas master on debian sid with updated numexpr and PyTables from experimental and I'm experimenting some test failures:

$ env PYTHONPATH=. nosetests pandas
.................................EES.FSSS......................................................S.............F..........................SSSS...SS.......................S........................................................................................................................................................................................................................................................................................................................................................S.......................................................................................................................SSSS..........................S.............................................................SSSSSSSSSS....................SS...................................................................................................................................................S...............................S.......................SSSSSSS.........................................F..............................................S....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................can't invoke "event" command:  application has been destroyed
    while executing
"event generate $w <<ThemeChanged>>"
    (procedure "ttk::ThemeChanged" line 6)
    invoked from within
"ttk::ThemeChanged"
E...................................................................................................................................................................................................................................................S........................................................................................................................................................................S..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.................................................................................S.S...............S.S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.............................................................................................S..............................................................................................................................................................................S...S............SSS.........S............SS............SS.....SSSS.................................................................................................................................................................................................................S.......................................................................................................................................................................................................................
======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 55, in test_get_goog_volume
    self.assertEqual(df.Volume.ix['OCT-08-2010'], 2863473)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 631, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/antonio/pandas/pandas/core/indexing.py", line 1273, in _get_label
    return self.obj[key]
  File "/home/antonio/pandas/pandas/core/series.py", line 920, in __getitem__
    return self.index.get_value(self, key)
  File "/home/antonio/pandas/pandas/tseries/index.py", line 1184, in get_value
    raise KeyError(key)
KeyError: 'OCT-08-2010'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('OCT-08-2010',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 68, in test_get_multi1
    testit()
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 64, in testit
    self.assertEquals(ts[0].dayofyear, 96)
  File "/home/antonio/pandas/pandas/tseries/index.py", line 1298, in __getitem__
    val = arr_idx[key]
IndexError: index 0 is out of bounds for axis 0 with size 0
-------------------- >> begin captured stdout << ---------------------
Failed: IndexError('index 0 is out of bounds for axis 0 with size 0',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_axis_limits (pandas.tseries.tests.test_plotting.TestTSPlot)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/tseries/tests/test_plotting.py", line 344, in test_axis_limits
    _test(ax)
  File "/home/antonio/pandas/pandas/tseries/tests/test_plotting.py", line 308, in _test
    ax.get_figure().canvas.draw()
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 349, in draw
    tkagg.blit(self._tkphoto, self.renderer._renderer, colormode=2)
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/tkagg.py", line 20, in blit
    tk.call("PyAggImagePhoto", photoimage, id(aggimage), colormode, id(bbox_array))
TclError: this isn't a Tk application

======================================================================
FAIL: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_data.py", line 42, in test_google
    13.68)
AssertionError: 17.66 != 13.68
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('17.66 != 13.68',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_invalid_url (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/util/testing.py", line 857, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas/pandas/io/tests/test_html.py", line 299, in test_invalid_url
    'http://www.a23950sdfa908sd.com')
AssertionError: No tables found
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('No tables found',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: testWLS (pandas.stats.tests.test_ols.TestOLS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 111, in testWLS
    self._check_wls(X, Y, weights)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 134, in _check_wls
    self.checkMovingOLS('rolling', x, y, weights=weights)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 208, in checkMovingOLS
    result_index=n)
  File "/home/antonio/pandas/pandas/stats/tests/test_ols.py", line 247, in compare
    assert_almost_equal(ref, res)
  File "/home/antonio/pandas/pandas/util/testing.py", line 198, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/home/antonio/pandas/pandas/util/testing.py", line 227, in assert_almost_equal
    1, a / b, decimal=decimal, err_msg=err_msg(a, b), verbose=False)
  File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 5 decimals expected 6.61194 but got 7.66003
-------------------- >> begin captured stdout << ---------------------
Make sure you're using statsmodels 0.5.0.dev-cec4f26 or later.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3933 tests in 407.389s

FAILED (SKIP=64, errors=3, failures=3)

Some cfg info:

PyTables version:  3.0.0
HDF5 version:      1.8.11
NumPy version:     1.7.1
Numexpr version:   2.2 (not using Intel's VML/MKL)
Python version:    2.7.5+ (default, Sep 17 2013, 15:31:50)  [GCC 4.8.1]
Platform:          linux2-x86_64
@jreback
Copy link
Contributor

jreback commented Sep 21, 2013

@avalentino try with PyTables 2.4...!

@cpcloud
Copy link
Member

cpcloud commented Sep 21, 2013

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

@cpcloud
Copy link
Member

cpcloud commented Sep 21, 2013

@avalentino Can you show the output of

ci/print_versions.py

@avalentino
Copy link
Contributor Author

@avalentino try with PyTables 2.4...!

@jreback e are working to push PyTables 3.0 (and numexpr 2.2) in sid and my purpose is to check if the new version of pytables breaks some of the dependent packages.

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

@cpcloud no, mainly a question of habit

@avalentino
Copy link
Contributor Author

@avalentino Any reason why you're setting PYTHONPATH to the current directory?

@cpcloud no, mainly a question of habit

well, I'm running tests from a checkout built with

$  python setup.py build_ext --inplace

@avalentino
Copy link
Contributor Author

@avalentino Can you show the output of
ci/print_versions.py

$ ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: Not installed
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

@cpcloud
Copy link
Member

cpcloud commented Sep 21, 2013

Okay now can you show the output of git describe? I'm guess the fact that pandas isn't "installed" might have something do with it, although I don't see why since you should be able to test it if you've built things....

If you do python setup.py develop nothing except a link to the pandas directory will be added to your site-packages, and pandas will show up as installed....is that a problem?

@avalentino
Copy link
Contributor Author

Okay now can you show the output of git describe? I'm guess the fact that pandas isn't "installed" might have something do with it, although I don't see why since you should be able to test it if you've built things....

$ git describe 
v0.12.0-501-g1675c2a

If you do python setup.py develop nothing except a link to the pandas directory will be added to your site-packages, and pandas will show up as installed....is that a problem?

$ python setup.py clean --all
$ python setup.py install --user

[...]

$ ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: 0.12.0-501-g1675c2a
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

Unfortunately now I'm no more able to run the test suite:

$ nosetests pandas
E
======================================================================
ERROR: Failure: ImportError (cannot import name hashtable)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 413, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/antonio/pandas/pandas/__init__.py", line 6, in <module>
    from . import hashtable, tslib, lib
ImportError: cannot import name hashtable

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

$ cd ~/tmp
$ nosetests pandas

----------------------------------------------------------------------
Ran 0 tests in 0.001s

OK

@cpcloud
Copy link
Member

cpcloud commented Sep 21, 2013

@avalentino Would you be willing to set up a virtualenv? Things will progress much quicker if you can isolate your Python version, packages, etc....I can help you set one up if necessary

@avalentino
Copy link
Contributor Author

@cpcloud OK, I managed to setup a virtual environment with:

$ virtualenv --system-site-packages ssp

and then I installed pandas from the check-out directory (updated this mourning).
Now I have a larger umber of tests that fail.

(ssp)antonio@mac4:~/ssp$ ../pandas/ci/print_versions.py 

INSTALLED VERSIONS
------------------
Python: 2.7.5.final.0
OS: Linux 3.10-3-amd64 #1 SMP Debian 3.10.11-1 (2013-09-10) x86_64
byteorder: little
LC_ALL: None
LANG: it_IT.UTF-8

pandas: 0.12.0-513-g5b0db94
Cython: 0.19.1
Numpy: 1.7.1
Scipy: 0.12.0
statsmodels: 0.4.2
    patsy: Not installed
scikits.timeseries: Not installed
dateutil: 1.5
pytz: 2012c
bottleneck: Not installed
PyTables: 3.0.0
    numexpr: 2.2
matplotlib: 1.3.0
openpyxl: 1.5.8
xlrd: 0.9.2
xlwt: 0.7.4
sqlalchemy: Not installed
lxml: Not installed
bs4: 4.2.1
html5lib: 0.95-dev

$ nosetests lib/python2.7/site-packages/pandas
................................................................................S................S........................................................................................................................................................EES.FSSS......................................................S.............F..........................SSSS...SS.........E........E....S.........E................................E.............................................................................................................................................................................................................................................................................................................S.......................................................................................................................SSSS..........................S.............................................................SSSSSSSSSS....................SS...................................................................................................................................................S...............................S.......................SSSSSSS.........................................F..............................................S........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S........................................................................................................................................................................S..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.................................................................................S.S...............S.S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.............................................................................................S..............................................................................................................................................................................S...S............SSS.........S............SS............SS.....SSSS.................................................................................................................................................................................................................S.......................................................................................................................................................................................................................
======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 55, in test_get_goog_volume
    self.assertEqual(df.Volume.ix['OCT-08-2010'], 2863473)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 631, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/indexing.py", line 1273, in _get_label
    return self.obj[key]
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/core/series.py", line 920, in __getitem__
    return self.index.get_value(self, key)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/tseries/index.py", line 1184, in get_value
    raise KeyError(key)
KeyError: 'OCT-08-2010'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('OCT-08-2010',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 68, in test_get_multi1
    testit()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 64, in testit
    self.assertEquals(ts[0].dayofyear, 96)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/tseries/index.py", line 1298, in __getitem__
    val = arr_idx[key]
IndexError: index 0 is out of bounds for axis 0 with size 0
-------------------- >> begin captured stdout << ---------------------
Failed: IndexError('index 0 is out of bounds for axis 0 with size 0',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_frame_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_pandas.py", line 391, in test_frame_from_json_precise_float
    result = read_json(df.to_json(), precise_float=True)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 180, in read_json
    date_unit).parse()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 466, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: test_series_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_pandas.py", line 386, in test_series_from_json_precise_float
    result = read_json(s.to_json(), typ='series', precise_float=True)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 187, in read_json
    date_unit).parse()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/json.py", line 401, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: testFloatMax (pandas.io.tests.test_json.test_ujson.NumpyJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_ujson.py", line 926, in testFloatMax
    assert_approx_equal(np.float(ujson.decode(ujson.encode(num, double_precision=15))), num, 15)
ValueError: Trailing data

======================================================================
ERROR: test_decimalDecodeTestPrecise (pandas.io.tests.test_json.test_ujson.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_json/test_ujson.py", line 95, in test_decimalDecodeTestPrecise
    decoded = ujson.decode(encoded, precise_float=True)
ValueError: Unexpected character in found when decoding object value

======================================================================
FAIL: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_data.py", line 42, in test_google
    13.68)
AssertionError: 17.390000000000001 != 13.68
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('17.390000000000001 != 13.68',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: test_invalid_url (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/io/tests/test_html.py", line 299, in test_invalid_url
    'http://www.a23950sdfa908sd.com')
AssertionError: No tables found
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('No tables found',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: testWLS (pandas.stats.tests.test_ols.TestOLS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 111, in testWLS
    self._check_wls(X, Y, weights)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 134, in _check_wls
    self.checkMovingOLS('rolling', x, y, weights=weights)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 208, in checkMovingOLS
    result_index=n)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/stats/tests/test_ols.py", line 247, in compare
    assert_almost_equal(ref, res)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 225, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/home/antonio/ssp/lib/python2.7/site-packages/pandas/util/testing.py", line 254, in assert_almost_equal
    1, a / b, decimal=decimal, err_msg=err_msg(a, b), verbose=False)
  File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 5 decimals expected 1.76645 but got 1.72056
-------------------- >> begin captured stdout << ---------------------
Make sure you're using statsmodels 0.5.0.dev-cec4f26 or later.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 4150 tests in 764.420s

FAILED (SKIP=66, errors=6, failures=3)

@jreback
Copy link
Contributor

jreback commented Sep 22, 2013

@avalentino can you show the latest entry in the git log

@jreback
Copy link
Contributor

jreback commented Sep 22, 2013

cc @Komnomnomnom, this is on mac, so maybe something new for the json issues?

@cpcloud
Copy link
Member

cpcloud commented Sep 22, 2013

are u sure it's on a mac? mac paths usually don't look like that

@cpcloud
Copy link
Member

cpcloud commented Sep 22, 2013

@avalentino can you run the tests from the cloned directory (instead of nosetests lib/python2.7/site-packages/pandas do nosetests /path/to/pandas)?

@cpcloud
Copy link
Member

cpcloud commented Sep 22, 2013

@avalentino so to recap:

# assuming you're in the virtualenv
git clone git://github.com/pydata/pandas.git
python setup.py develop
nosetests -w pandas

@jreback
Copy link
Contributor

jreback commented Sep 22, 2013

nvm not a mac...was reading the @mac4 prompt duh!

@avalentino
Copy link
Contributor Author

Hi @jreback, hi @cpcloud,
sorry for generating confusion.
The test has been run in VirualBox on mac HW, the OS is debian sid (amd64) with numexpr 2.2 and pytables 3.0 from the experimental archive.
Also in order to run the test I have set up a virtualenv using the system site packages:

$ virtualenv --system-site-packages pandas-venv

With this setup if I run the commands suggested by @cpcloud

$ git clone git://github.com/pydata/pandas.git
$ python setup.py develop
$ nosetests -w pandas

the result is the one reported in the comment above.

@Komnomnomnom
Copy link
Contributor

I suspect (at least some) of the JSON issues are due to the locale setting (it_IT), will try to test and fix

@cpcloud
Copy link
Member

cpcloud commented Sep 23, 2013

@avalentino can you run in a virtualenv without system site packages? would make it easier to debug...

@Komnomnomnom
Copy link
Contributor

@cpcloud I can reproduce most of these on my Arch64 setup by adding

import locale
locale.setlocale(locale.LC_ALL, 'it_IT.UTF-8')    

to pandas/__init__.py

======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_data.py", line 55, in test_get_goog_volume
    self.assertEqual(df.Volume.ix['OCT-08-2010'], 2863473)
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 635, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 1277, in _get_label
    return self.obj[key]
  File "/home/kieran/work/git/pandas/pandas/core/series.py", line 920, in __getitem__
    return self.index.get_value(self, key)
  File "/home/kieran/work/git/pandas/pandas/tseries/index.py", line 1185, in get_value
    raise KeyError(key)
KeyError: 'OCT-08-2010'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('OCT-08-2010',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_data.py", line 68, in test_get_multi1
    testit()
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_data.py", line 64, in testit
    self.assertEquals(ts[0].dayofyear, 96)
  File "/home/kieran/work/git/pandas/pandas/tseries/index.py", line 1299, in __getitem__
    val = arr_idx[key]
IndexError: index 0 is out of bounds for axis 0 with size 0
-------------------- >> begin captured stdout << ---------------------
Failed: IndexError('index 0 is out of bounds for axis 0 with size 0',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi2 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_data.py", line 77, in test_get_multi2
    result = pan.Close.ix['01-18-12']
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 54, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 635, in _getitem_axis
    return self._get_label(key, axis=axis)
  File "/home/kieran/work/git/pandas/pandas/core/indexing.py", line 66, in _get_label
    return self.obj._xs(label, axis=axis, copy=True)
  File "/home/kieran/work/git/pandas/pandas/core/frame.py", line 2331, in xs
    loc = self.index.get_loc(key)
  File "/home/kieran/work/git/pandas/pandas/tseries/index.py", line 1224, in get_loc
    raise KeyError(key)
KeyError: '01-18-12'
-------------------- >> begin captured stdout << ---------------------
Failed: KeyError('01-18-12',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_frame_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_json/test_pandas.py", line 391, in test_frame_from_json_precise_float
    result = read_json(df.to_json(), precise_float=True)
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 180, in read_json
    date_unit).parse()
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 466, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: test_series_from_json_precise_float (pandas.io.tests.test_json.test_pandas.TestPandasContainer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_json/test_pandas.py", line 386, in test_series_from_json_precise_float
    result = read_json(s.to_json(), typ='series', precise_float=True)
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 187, in read_json
    date_unit).parse()
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 248, in parse
    self._parse_no_numpy()
  File "/home/kieran/work/git/pandas/pandas/io/json.py", line 401, in _parse_no_numpy
    loads(json, precise_float=self.precise_float), dtype=None)
ValueError: Unexpected character in found when decoding object value

======================================================================
ERROR: testFloatMax (pandas.io.tests.test_json.test_ujson.NumpyJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_json/test_ujson.py", line 926, in testFloatMax
    assert_approx_equal(np.float(ujson.decode(ujson.encode(num, double_precision=15))), num, 15)
ValueError: Trailing data

======================================================================
ERROR: test_decimalDecodeTestPrecise (pandas.io.tests.test_json.test_ujson.UltraJSONTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_json/test_ujson.py", line 95, in test_decimalDecodeTestPrecise
    decoded = ujson.decode(encoded, precise_float=True)
ValueError: Unexpected character in found when decoding object value

======================================================================
FAIL: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kieran/work/git/pandas/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/kieran/work/git/pandas/pandas/io/tests/test_data.py", line 42, in test_google
    13.68)
AssertionError: 17.390000000000001 != 13.68
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('17.390000000000001 != 13.68',)

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3958 tests in 780.565s

FAILED (SKIP=82, errors=7, failures=1)

@cpcloud
Copy link
Member

cpcloud commented Sep 23, 2013

i believe the google stuff is time zone sensitive so that should be fixed if possible. not sure about the json stuff. i would bet u have some ideas though 😄

@jreback
Copy link
Contributor

jreback commented Sep 23, 2013

@cpcloud we have 1 build with an encoding
maybe add a locale to it? 2.7 one

@jreback
Copy link
Contributor

jreback commented Sep 23, 2013

or maybe add a separate new build
as not taking that much time nowadays?

@cpcloud
Copy link
Member

cpcloud commented Sep 23, 2013

problem is that only 5 builds can run at a time i believe so going to be slowest build + 6th build time == 😪

@cpcloud
Copy link
Member

cpcloud commented Sep 23, 2013

i'll take a look and assign myself to keep track of things

@ghost ghost assigned cpcloud Sep 23, 2013
@Komnomnomnom
Copy link
Contributor

JSON fix would be to force locale to en_GB (or en_US) during serialisation or deserialisation. Decimal separator is always '.' according to JSON spec. Will send through a PR.

It's weird that @avalentino is seeing these failures though because I didn't think Python used the system's locale unless it was forcibly set using locale.setlocale, like I'm doing above.

@jreback
Copy link
Contributor

jreback commented Sep 23, 2013

@cpcloud why don't we just set it the main build then - or is that weird? (or 2.6 build) just to try to catch stuff like this

@cpcloud
Copy link
Member

cpcloud commented Sep 23, 2013

@jreback i think 2.6 build would be fine, i'll try and get something up this week

@Komnomnomnom thanks! very much appreciated

@avalentino
Copy link
Contributor Author

I can confirm that using the C locale for numbers and dates things go better:

$ env LC_NUMERIC=C LC_TIME=C nosetests -w pandas

[CUT]

======================================================================
FAIL: test_invalid_url (pandas.io.tests.test_html.TestReadHtmlBase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 884, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas-venv/pandas/pandas/io/tests/test_html.py", line 299, in test_invalid_url
    'http://www.a23950sdfa908sd.com')
AssertionError: No tables found
-------------------- >> begin captured stdout << ---------------------
Failed: AssertionError('No tables found',)

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: testWLS (pandas.stats.tests.test_ols.TestOLS)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/stats/tests/test_ols.py", line 111, in testWLS
    self._check_wls(X, Y, weights)
  File "/home/antonio/pandas-venv/pandas/pandas/stats/tests/test_ols.py", line 134, in _check_wls
    self.checkMovingOLS('rolling', x, y, weights=weights)
  File "/home/antonio/pandas-venv/pandas/pandas/stats/tests/test_ols.py", line 208, in checkMovingOLS
    result_index=n)
  File "/home/antonio/pandas-venv/pandas/pandas/stats/tests/test_ols.py", line 247, in compare
    assert_almost_equal(ref, res)
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 225, in assert_almost_equal
    assert_almost_equal(a[i], b[i], check_less_precise)
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 254, in assert_almost_equal
    1, a / b, decimal=decimal, err_msg=err_msg(a, b), verbose=False)
  File "/usr/lib/pymodules/python2.7/numpy/testing/utils.py", line 468, in assert_almost_equal
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 5 decimals expected 1.61882 but got 1.73890
-------------------- >> begin captured stdout << ---------------------
Make sure you're using statsmodels 0.5.0.dev-cec4f26 or later.

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 3935 tests in 391.832s

FAILED (SKIP=64, failures=2)

@jreback
Copy link
Contributor

jreback commented Sep 26, 2013

@cpcloud is the tests in #4242 failing? (e.g. you see them here), because of the locale issue?

@cpcloud
Copy link
Member

cpcloud commented Oct 7, 2013

@avalentino Can you pull down my PR #5073 and see if anything is still failing? Much appreciated!

@avalentino
Copy link
Contributor Author

$ git branch -v
  master        16e2f63 [dietro 244] Merge pull request #4922 from cpcloud/eval-syntax-docs
* travis-locale 051ea88 TST: enable test of locales with a diff locale
$ git describe 
v0.12.0-760-g051ea88
$ nosetests -w pandas

[...]

======================================================================
ERROR: test_get_goog_volume (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 1061, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas-venv/pandas/pandas/io/tests/test_data.py", line 62, in test_get_goog_volume
    with tm.set_locale('en_US.UTF-8'):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 208, in set_locale
    locale.setlocale(lc_var, new_locale)
  File "/home/antonio/pandas-venv/lib/python2.7/locale.py", line 547, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting
-------------------- >> begin captured stdout << ---------------------
Failed: Error('unsupported locale setting',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi1 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 1061, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas-venv/pandas/pandas/io/tests/test_data.py", line 69, in test_get_multi1
    with tm.set_locale('en_US.UTF-8'):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 208, in set_locale
    locale.setlocale(lc_var, new_locale)
  File "/home/antonio/pandas-venv/lib/python2.7/locale.py", line 547, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting
-------------------- >> begin captured stdout << ---------------------
Failed: Error('unsupported locale setting',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_multi2 (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 1061, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas-venv/pandas/pandas/io/tests/test_data.py", line 85, in test_get_multi2
    with tm.set_locale('en_US.UTF-8'):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 208, in set_locale
    locale.setlocale(lc_var, new_locale)
  File "/home/antonio/pandas-venv/lib/python2.7/locale.py", line 547, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting
-------------------- >> begin captured stdout << ---------------------
Failed: Error('unsupported locale setting',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_google (pandas.io.tests.test_data.TestGoogle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 1061, in network_wrapper
    return t(*args, **kwargs)
  File "/home/antonio/pandas-venv/pandas/pandas/io/tests/test_data.py", line 48, in test_google
    with tm.set_locale('en_US.UTF-8'):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 208, in set_locale
    locale.setlocale(lc_var, new_locale)
  File "/home/antonio/pandas-venv/lib/python2.7/locale.py", line 547, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting
-------------------- >> begin captured stdout << ---------------------
Failed: Error('unsupported locale setting',)

--------------------- >> end captured stdout << ----------------------

======================================================================
ERROR: test_get_locales_prefix (pandas.tools.tests.test_util.TestLocaleUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/tools/tests/test_util.py", line 34, in test_get_locales_prefix
    with set_locale('en_US.UTF-8'):
  File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/home/antonio/pandas-venv/pandas/pandas/util/testing.py", line 208, in set_locale
    locale.setlocale(lc_var, new_locale)
  File "/home/antonio/pandas-venv/lib/python2.7/locale.py", line 547, in setlocale
    return _setlocale(category, locale)
Error: unsupported locale setting

======================================================================
FAIL: test_get_locales (pandas.tools.tests.test_util.TestLocaleUtils)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antonio/pandas-venv/pandas/pandas/tools/tests/test_util.py", line 31, in test_get_locales
    assert len(get_locales(prefix='en')) > 0
AssertionError

----------------------------------------------------------------------
Ran 4067 tests in 187.622s

FAILED (SKIP=308, errors=5, failures=1)

@cpcloud
Copy link
Member

cpcloud commented Oct 7, 2013

those tests betray my english centric mind :)

good that the others are not failing anymore these are easy "fix" ... just skip when the locale is not en_US.UTF-8 ill allow the ISO encoding as well

@cpcloud
Copy link
Member

cpcloud commented Oct 9, 2013

@avalentino pls test with #5073 and lmk if anything comes up. thx!

@cpcloud
Copy link
Member

cpcloud commented Oct 9, 2013

if nothing comes up i'll close

@avalentino
Copy link
Contributor Author

@cpcloud all tests pass now, thanks.

@cpcloud
Copy link
Member

cpcloud commented Oct 9, 2013

closed by #5073

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing pandas testing functions or related to the test suite
Projects
None yet
Development

No branches or pull requests

4 participants