nosetest failed for pandas 0.16.0

[leo4183]

Hi there,

I recently upgraded pandas from 0.15.1 to 0.16.0. after running the nosetest i got the following error msgs. wondering if anything series (i tested HDF5 writing/reading. everything looks normal).

ERROR: test_conv_read_write (pandas.io.tests.test_pytables.TestHDFStore)

Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/pandas/io/tests/test_pytables.py", line 205, in test_conv_read_write
assert_series_equal(o, roundtrip('series',o))
File "/usr/local/lib/python3.4/site-packages/pandas/io/tests/test_pytables.py", line 201, in roundtrip
obj.to_hdf(self.path, key,*_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/core/generic.py", line 913, in to_hdf
return pytables.to_hdf(path_or_buf, key, self, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 266, in to_hdf
complib=complib) as store:
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 396, in init
self.open(mode=mode, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 536, in open
self._handle = tables.open_file(self._path, 'r', *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 791, in init
self._g_new(filename, mode, *_params)
File "hdf5extension.pyx", line 359, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3756)
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 157, in check_file_access
raise IOError("%s does not exist" % (filename,))
nose.proxy.OSError: tmp.__5JMnSD5gVu__.h5 does not exist
-------------------- >> begin captured stdout << ---------------------
Opening tmp.5JMnSD5gVu.h5 in read-only mode

--------------------- >> end captured stdout << ----------------------

ERROR: test_copy (pandas.io.tests.test_pytables.TestHDFStore)

Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/pandas/io/tests/test_pytables.py", line 4331, in test_copy
st = HDFStore(self.path)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 396, in init
self.open(mode=mode, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 536, in open
self._handle = tables.open_file(self._path, 'r', *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 791, in init
self._g_new(filename, mode, *_params)
File "hdf5extension.pyx", line 359, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3756)
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 157, in check_file_access
raise IOError("%s does not exist" % (filename,))
nose.proxy.OSError: tmp.__NZTYQk5IW2__.h5 does not exist
-------------------- >> begin captured stdout << ---------------------
Opening tmp.NZTYQk5IW2.h5 in read-only mode

--------------------- >> end captured stdout << ----------------------

ERROR: test_factory_fun (pandas.io.tests.test_pytables.TestHDFStore)

Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/pandas/io/tests/test_pytables.py", line 160, in test_factory_fun
with get_store(self.path) as tbl:
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 1273, in get_store
return HDFStore(path, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 396, in init
self.open(mode=mode, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 536, in open
self._handle = tables.open_file(self._path, 'r', *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 791, in init
self._g_new(filename, mode, **params)
File "hdf5extension.pyx", line 359, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3756)
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 157, in check_file_access
raise IOError("%s does not exist" % (filename,))
nose.proxy.OSError: tmp.__lL9hBPXiYa__.h5 does not exist
-------------------- >> begin captured stdout << ---------------------
Opening tmp.lL9hBPXiYa.h5 in read-only mode

--------------------- >> end captured stdout << ----------------------

my python/pandas environment (Redhat server v6.5):

certifi (14.05.14)
Cython (0.21.1)
ipython (2.3.1)
jieba3k (0.35.1)
Jinja2 (2.7.3)
lxml (3.4.0)
MarkupSafe (0.23)
matplotlib (1.4.2)
mysql-connector-python (2.0.2)
nltk (3.0.1)
nose (1.3.4)
numexpr (2.4)
numpy (1.8.2)
openpyxl (1.6.1)
oursql (0.9.4)
pandas (0.16.0)
patsy (0.3.0)
pip (1.5.6)
pyparsing (2.0.3)
pyreadline (2.0)
python-dateutil (2.2)
pytz (2014.9)
pyzmq (14.4.1)
scipy (0.14.0)
setuptools (2.1)
six (1.8.0)
SQLAlchemy (0.9.8)
statsmodels (0.6.0)
tables (3.1.1)
tornado (4.0.2)
xlrd (0.9.3)

[jreback]

looks like something intermittent with the temporary file. if you rerun just pandas/io/tests/test_pytables.py does this persist?

[leo4183]

yes, the issue remains:

test_api (main.TestHDFStore) ... ok
test_api_default_format (main.TestHDFStore) ... ok
test_append (main.TestHDFStore) ... ok
test_append_all_nans (main.TestHDFStore) ... ok
test_append_diff_item_order (main.TestHDFStore) ... ok
test_append_frame_column_oriented (main.TestHDFStore) ... ok
test_append_hierarchical (main.TestHDFStore) ... ok
test_append_misc (main.TestHDFStore) ... ok
test_append_raise (main.TestHDFStore) ... ok
test_append_series (main.TestHDFStore) ... ok
test_append_some_nans (main.TestHDFStore) ... ok
test_append_to_multiple (main.TestHDFStore) ... ok
test_append_to_multiple_dropna (main.TestHDFStore) ... ok
test_append_with_data_columns (main.TestHDFStore) ... ok
test_append_with_diff_col_name_types_raises_value_error (main.TestHDFStore) ... ok
test_append_with_different_block_ordering (main.TestHDFStore) ... ok
test_append_with_strings (main.TestHDFStore) ... ok
test_append_with_timedelta (main.TestHDFStore) ... ok
test_append_with_timezones_dateutil (main.TestHDFStore) ... ok
test_append_with_timezones_pytz (main.TestHDFStore) ... ok
test_backwards_compat_without_term_object (main.TestHDFStore) ... ok
test_calendar_roundtrip_issue (main.TestHDFStore) ... ok
test_can_serialize_dates (main.TestHDFStore) ... ok
test_categorical (main.TestHDFStore) ... ok
test_column_multiindex (main.TestHDFStore) ... ok
test_contains (main.TestHDFStore) ... ok
test_context (main.TestHDFStore) ... Opening tmp.ZhpolDfsqD.h5 in read-only mode

/usr/local/lib/python3.4/site-packages/tables/utils.py(157)check_file_access()
-> raise IOError("%s does not exist" % (filename,))
(Pdb) exit
ERROR

ERROR: test_context (main.TestHDFStore)

Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 532, in open
self._handle = tables.open_file(self._path, self._mode, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 791, in init
self._g_new(filename, mode, **params)
File "hdf5extension.pyx", line 359, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3756)
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 185, in check_file_access
check_file_access(filename, 'w')
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 180, in check_file_access
"written" % (parentname,))
OSError: directory . exists but it can not be written

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/pandas/io/tests/test_pytables.py", line 179, in test_context
with HDFStore(self.path) as tbl:
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 396, in init
self.open(mode=mode, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/pandas/io/pytables.py", line 536, in open
self._handle = tables.open_file(self._path, 'r', *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 318, in open_file
return File(filename, mode, title, root_uep, filters, *_kwargs)
File "/usr/local/lib/python3.4/site-packages/tables/file.py", line 791, in init
self._g_new(filename, mode, *_params)
File "hdf5extension.pyx", line 359, in tables.hdf5extension.File._g_new (tables/hdf5extension.c:3756)
File "/usr/local/lib/python3.4/site-packages/tables/utils.py", line 157, in check_file_access
raise IOError("%s does not exist" % (filename,))
OSError: tmp.__ZhpolDfsqD__.h5 does not exist

---

[jreback]

so this particular test writes in the current directory rather in a created temporary. It appears that you don't have write permission where its being run (not sure why its using the source dir though).

So I think there are 3 tests (test_context/test_factory_fun/test_conv_read_write) that directly use self.path rather than and ensure_clean* routines (as these are testing the actual creation / destruction). These should use a path = create_tempfile(self.path)

want to do a pull-request?

[vikramls]

I submitted a PR for this ticket: #10080. If there are issues, please let me know how I can improve.

[jreback]

closed via #10080