Windows test failures

[Komnomnomnom]

Windows 7 - 64-bit
Enthought Python 2.7 - 32 bit
VS2008 compiled cython+pandas

======================================================================
FAIL: test_round_trip_frame (pandas.io.tests.test_clipboard.TestClipboard)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\io\tests\test_clipboard.py", line 50, in test
_round_trip_frame
    self.check_round_trip_frame(dt)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\io\tests\test_clipboard.py", line 46, in chec
k_round_trip_frame
    tm.assert_frame_equal(data, result)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 295, in assert_frame_e
qual
    check_less_precise=check_less_precise)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 244, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 161, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 146, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int32') != dtype('int64')

======================================================================
FAIL: test_constructor_generator (pandas.tests.test_frame.TestDataFrame)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_frame.py", line 2727, in test_cons
tructor_generator
    assert_frame_equal(result, expected)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 295, in assert_frame_e
qual
    check_less_precise=check_less_precise)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 244, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 161, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 146, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int64') != dtype('int32')

======================================================================
FAIL: test_plot_xy (pandas.tests.test_graphics.TestDataFramePlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 454, in test_pl
ot_xy
    == np.array((16., 8.))).all())
AssertionError: False is not true

======================================================================
FAIL: test_unsorted_index (pandas.tests.test_graphics.TestDataFramePlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 500, in test_un
sorted_index
    tm.assert_series_equal(rs, df.y)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 244, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 161, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 146, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int64') != dtype('int32')

======================================================================
FAIL: test_plot (pandas.tests.test_graphics.TestSeriesPlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 80, in test_plo
t
    == np.array((16., 8.))).all())
AssertionError: False is not true

======================================================================
FAIL: test_partial_setting (pandas.tests.test_indexing.TestIndexing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_indexing.py", line 1380, in test_p
artial_setting
    assert_frame_equal(df,expected)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 295, in assert_frame_e
qual
    check_less_precise=check_less_precise)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 244, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 161, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 146, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int32') != dtype('int64')

======================================================================
FAIL: test_constructor_iterator (pandas.tests.test_series.TestSeries)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_series.py", line 360, in test_cons
tructor_iterator
    assert_series_equal(result, expected)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 244, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 161, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 146, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int32') != dtype('int64')

----------------------------------------------------------------------
Ran 3705 tests in 1360.689s

FAILED (SKIP=168, failures=7)

[Komnomnomnom]

Only four test failures on latest master

======================================================================
FAIL: test_round_trip_frame (pandas.io.tests.test_clipboard.TestClipboard)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\io\tests\test_clipboard.py", line 50, in test
_round_trip_frame
    self.check_round_trip_frame(dt)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\io\tests\test_clipboard.py", line 46, in chec
k_round_trip_frame
    tm.assert_frame_equal(data, result)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 306, in assert_frame_e
qual
    check_less_precise=check_less_precise)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 255, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 172, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 157, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int32') != dtype('int64')

======================================================================
FAIL: test_plot_xy (pandas.tests.test_graphics.TestDataFramePlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 454, in test_pl
ot_xy
    == np.array((16., 8.))).all())
AssertionError: False is not true

======================================================================
FAIL: test_unsorted_index (pandas.tests.test_graphics.TestDataFramePlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 500, in test_un
sorted_index
    tm.assert_series_equal(rs, df.y)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 255, in assert_series_
equal
    assert_attr_equal('dtype', left, right)
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 172, in assert_attr_eq
ual
    assert_equal(left_attr,right_attr,"attr is not equal [{0}]" .format(attr))
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\util\testing.py", line 157, in assert_equal
    assert a == b, "%s: %r != %r" % (msg.format(a,b), a, b)
AssertionError: attr is not equal [dtype]: dtype('int64') != dtype('int32')

======================================================================
FAIL: test_plot (pandas.tests.test_graphics.TestSeriesPlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 80, in test_plo
t
    == np.array((16., 8.))).all())
AssertionError: False is not true

----------------------------------------------------------------------
Ran 3901 tests in 1334.203s

FAILED (SKIP=232, failures=4)

However test_plot (pandas.tests.test_graphics.TestSeriesPlots) and test_plot_xy (pandas.tests.test_graphics.TestDataFramePlots) pass when TestSeriesPlots and TestDataFramePlots are run in isolation.

[Komnomnomnom]

The dtype failures are easy fixes. However the two other plotting failures only happen when running the full test suite, looks like some weirdness with matplotlib.

C:\Users\kieran\Documents\GitHub\pandas [master +3 ~2 -0 !]> nosetests --ipdb-failure pandas
..SS..SS..SS..SSC:\Users\kieran\AppData\Local\Enthought\Canopy32\User\lib\site-packages\numpy\core\f
romnumeric.py:595: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  a.sort(axis, kind, order)
C:\Users\kieran\Documents\GitHub\pandas\pandas\core\index.py:719: RuntimeWarning: tp_compare didn't
return -1 or -2 for exception
  result.sort()
..SS..SS..SS..SS..SSSS...S............S...S......S.....SSS.........................................S
S..SS..SSSS.S.S...........SS..SS.....................................SSS.....SSSSSSSSSSSS.SSSSSSSSSS
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS.SSS.......................S
....................................................................................................
....................................................................................................
....................................................................................................
............................................S.......................................................
..............................................SSSSSSSSSSS....................SS.....................
....................................................................................................
..........................S...............................S..................S..S.SSSSSSS...........
...........................S.....SSSSSSSSSSSSSSSSSSSSSSSSSS.........................................
....................................................................................................
....................................................................................................
....................................................................................................
....................................................................................................
......................................................................C:\Users\kieran\AppData\Local\
Enthought\Canopy32\User\lib\site-packages\matplotlib\pyplot.py:412: RuntimeWarning: More than 20 fig
ures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are
 retained until explicitly closed and may consume too much memory. (To control this warning, see the
 rcParam `figure.max_num_figures`).
  max_open_warning, RuntimeWarning)
....................................................................................................
....................................................................................................
.................S..................................................................................
................................................................................S...................
...................................................S................................................
....................................................................................................
....................................................................................................
....................................................................................................
....................................................................................................
...............................................................................................S....
............................................................SS....SS.........S.S.......S.S.S.S.S.S.S
.S.S.S.S............................................
- TRACEBACK --------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\App\appdata\canopy-1.1.0.1371.win-x86\lib\u
nittest\case.py", line 327, in run
    testMethod()
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 454, in test_pl
ot_xy
    == np.array((16., 8.))).all())
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\App\appdata\canopy-1.1.0.1371.win-x86\lib\u
nittest\case.py", line 420, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true
--------------------------------------------------------------------------------
> c:\users\kieran\appdata\local\enthought\canopy32\app\appdata\canopy-1.1.0.1371.win-x86\lib\unittes
t\case.py(420)assertTrue()
    419             msg = self._formatMessage(msg, "%s is not true" % safe_repr(expr))
--> 420             raise self.failureException(msg)
    421

ipdb> u
> c:\users\kieran\documents\github\pandas\pandas\tests\test_graphics.py(454)test_plot_xy()
    453         self.assert_((np.round(ax.figure.get_size_inches())
--> 454                       == np.array((16., 8.))).all())
    455

ipdb> ax.figure.get_size_inches()
array([ 12.85,   8.  ])
ipdb> l
    449         plt.close('all')
    450         ax = df.plot(x=1, y=2, title='Test', figsize=(16, 8))
    451
    452         self.assert_(ax.title.get_text() == 'Test')
    453         self.assert_((np.round(ax.figure.get_size_inches())
--> 454                       == np.array((16., 8.))).all())
    455
    456         # columns.inferred_type == 'mixed'
    457         # TODO add MultiIndex test
    458
    459     @slow

ipdb> ax.title.get_text()
'Test'
ipdb> df.plot(x=1, y=2, title='Test', figsize=(16, 8))
<matplotlib.axes.AxesSubplot object at 0x2D526950>
ipdb> df.plot(x=1, y=2, title='Test', figsize=(16, 8)).figure
<matplotlib.figure.Figure object at 0x2D526F70>
ipdb> df.plot(x=1, y=2, title='Test', figsize=(16, 8)).figure.get_size_inches()
array([ 16.,   8.])
ipdb> df.plot(x=1, y=2, title='Test', figsize=(16, 8)).figure.get_size_inches()
array([ 16.,   8.])
ipdb> ax
<matplotlib.axes.AxesSubplot object at 0x2D526950>
ipdb> ax.figure
<matplotlib.figure.Figure object at 0x2D526F70>
ipdb> ax.figure.get_size_inches()
array([ 16.,   8.])
ipdb>

[jreback]

@Komnomnomnom ok...all non-plotting tests look ok

@cpcloud shouldn't these figures be closed after each test?

[jreback]

@Komnomnomnom can you update with I am guessing only the plot failures remaining? (after you update from master)

[cpcloud]

@jreback @Komnomnomnom There's a bit of code rot in the plotting tests....I'm finding things like:

self.assert_(l1.get_color(), l2.get_color())

(this will always test True as long as l1.get_color() isn't False-y; should be self.assertEqual(...) here)

There's a couple of other things I'll clean up as well.

PR to arrive later today

[cpcloud]

And yes...the tearDown method should close all open plots. I'm changing _check_plot_works to close figs as well.

[dragoljub]

I was recently trying to build Pandas from source on Windows and actually ran into compile errors with objToJSON.c. Win 7 X64, VS 2008 Compiler. Not sure if these were resolved already.

If you guys have lots of experience making the builds work on windows a walk-through would be great. 😄 Or having the nightly builds produce binaries for windows more frequently would be great too. ✌️ I used http://blog.victorjabur.com/2011/06/05/compiling-python-2-7-modules-on-windows-32-and-64-using-msvc-2008-express/ tutorial to help me get started building on windows. Any other advice would be great.

pandas/src/ujson/python/objToJSON.c(249) : warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data
pandas/src/ujson/python/objToJSON.c(274) : error C2143: syntax error : missing ';' before 'type'
pandas/src/ujson/python/objToJSON.c(275) : error C2275: 'PyDatetimeScalarObject' : illegal use of this type as an expression
        C:\Python27\lib\site-packages\numpy\core\include\numpy/arrayscalars.h(123) : see declaration of 'PyDatetimeScalarObject'
pandas/src/ujson/python/objToJSON.c(275) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2223: left of '->obval' must point to struct/union
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'obj' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : error C2223: left of '->obmeta' must point to struct/union
pandas/src/ujson/python/objToJSON.c(277) : error C2065: 'dts' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(277) : warning C4047: 'function' : 'npy_datetime' differs in levels of indirection from 'int *'
pandas/src/ujson/python/objToJSON.c(277) : warning C4024: 'pandas_datetime_to_datetimestruct' : different types for formal and actual parameter 1
pandas/src/ujson/python/objToJSON.c(277) : error C2198: 'pandas_datetime_to_datetimestruct' : too few arguments for call
pandas/src/ujson/python/objToJSON.c(278) : error C2065: 'dts' : undeclared identifier
pandas/src/ujson/python/objToJSON.c(278) : warning C4133: 'function' : incompatible types - from 'int *' to 'pandas_datetimestruct *'
pandas/src/ujson/python/objToJSON.c(356) : warning C4244: '=' : conversion from 'npy_intp' to 'int', possible loss of data
error: command '"C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe"' failed with exit status 2

[jreback]

@dragoljub you should be able to update to master and these errors are fix, IOW it should build (as of yesterday master)...so need failry new

[Komnomnomnom]

@dragoljub the only advice I can give is to make sure to use the same compiler as that used to build your Python binary (and numpy, cython etc), otherwise even if your build succeeds you can run into unexpected runtime issues (crashes and segmentation faults). Usually this means working with VS2008 but if working with a different Python distribution (e.g. pythonxy) this can mean using MinGW to compile, or if using Python 3 then VS2010. I'm not at all seasoned in building on windows, but this has been my experience.

As jreback mentioned the build error you're encountering has been fixed by #4778. There's some more detail there of some woes I encountered building on windows.

[Komnomnomnom]

@jreback, @cpcloud I can confirm that the non-dtype plotting test failures are still occurring:

======================================================================
FAIL: test_plot_xy (pandas.tests.test_graphics.TestDataFramePlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 461, in test_pl
ot_xy
    np.array((16., 8.)))
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\User\lib\site-packages\numpy\testing\utils.
py", line 719, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\User\lib\site-packages\numpy\testing\utils.
py", line 645, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 50.0%)
 x: array([ 13.,   8.])
 y: array([ 16.,   8.])

======================================================================
FAIL: test_plot (pandas.tests.test_graphics.TestSeriesPlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\kieran\Documents\GitHub\pandas\pandas\tests\test_graphics.py", line 79, in test_plo
t
    np.array((16., 8.)))
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\User\lib\site-packages\numpy\testing\utils.
py", line 719, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "C:\Users\kieran\AppData\Local\Enthought\Canopy32\User\lib\site-packages\numpy\testing\utils.
py", line 645, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Arrays are not equal

(mismatch 50.0%)
 x: array([ 13.,   8.])
 y: array([ 16.,   8.])

----------------------------------------------------------------------
Ran 3905 tests in 3172.156s

FAILED (SKIP=233, failures=2)

[cpcloud]

ok well now at least we can see what's different...

@Komnomnomnom What is the output of the following code on your Windows box

ioff()
fig, ax = subplots(1, 1, figsize=(16, 8))
t = linspace(0, 2 * pi, 1000)
x = sin(t)
ax.plot(t, x)
print(fig.get_size_inches())

[Komnomnomnom]

@cpcloud I don't think the tests fail unless the whole testsuite is run. I'm gonna try increasing the ram on the VM, in case that is the issue.

[Komnomnomnom]

Which I will do once it shuts down .... "Installing update 17 of 70" sigh