TST: splitlines in rec2txt test #6423

tacaswell · 2016-05-14T21:07:05Z

On windows the output has '\r\n' instead of '\n' for new
lines.

If this passes I plan to self-merge to un-break appveyor

On windows the output has '\r\n' instead of '\n' for new lines.

tacaswell · 2016-05-14T21:07:28Z

attn @dopplershift I think this is an equivalent test .

tacaswell · 2016-05-15T00:11:50Z

@JanSchulz Can you help out with the windows issue here? I do not have a system to test on and debugging via appveyor seems maddeningly painful!

dopplershift · 2016-05-15T17:19:13Z

👍 from me.

tacaswell · 2016-05-16T14:01:16Z

In windows this seems to output one ~~less~~ space is some cases:

windows = ['       x   y   s    s2', '   1.000   2   foo  bing ']
truth =   ['       x   y   s   s2',  '   1.000   2   foo bing ']

tacaswell · 2016-05-16T14:06:38Z

Sorry, had order flipped, windows adds an extra space.

The division was always returning 1, use length of first element instead.

tacaswell · 2016-05-16T14:43:10Z

@dopplershift I think there was a bug in computing the width of the string columns which for some reason was behaving differently on windows/linux.

jankatins · 2016-05-16T16:29:07Z

I have a look... Looks like there are at least two problems:

The one where the link step can't find png.lib and the one where there is a space difference.

Re the png one: it seems that conda-forge recently added a libpng (and that one is installed), could be that there is a difference to the official one? https://github.com/conda-forge/libpng-feedstock/commits/master (cc: @ocefpaf)

tacaswell · 2016-05-16T16:30:36Z

I think I have the space issue fixed

dopplershift · 2016-05-16T17:07:50Z

lib/matplotlib/mlab.py

-            # The division below handles unicode stored in array, which could
-            # have 4 bytes per char
-            length = max(len(colname), column.itemsize // column[0].itemsize)
+            length = max(len(colname), len(column[0]))


I'm not convinced len() will do what we want if the data for a column is ['foo', 'bars']. Probably should change the test to have some differing string sizes and see what happens.

is that allowed in recarrays? The types look like fixed-width string, not variable width.

In [18]: a = np.array([(1, 'foo'), (2, 'bars')], dtype=np.dtype([('i', np.int32), ('s', np.str, 4)])) In [19]: a Out[19]: array([(1, 'foo'), (2, 'bars')], dtype=[('i', '<i4'), ('s', '<U4')]) In [20]: len(a['s'][0]) Out[20]: 3

I couldn't find a numpy function to get that 4 from the <U4, which is what we really want.

But it turns out my original code was broken as well:

In [29]: a['s'][0].itemsize Out[29]: 3

WTF?

Maybe you want a.dtype['s'].itemsize == 16 ( / 4 for Unicode == 4)?

len() is not the right thing. The best I can come up with is int(a.dtype['s'].str[2:])

tacaswell · 2016-05-17T01:11:09Z

And tests are passing, but it is dying as while trying to build the conda package

Fixed width byte/unicode dtypes seem to have the pattern "[|<>][US][1-9][0-9]*" thus if we drop the first two charters we will have the fixed width.

tacaswell · 2016-05-17T03:22:27Z

ok, this is now passing (except for a down-load error on one of the appveyor tests).

I had to turn off the conda package building which is less than great, but makes the test useful again for code review.

cgohlke · 2016-05-23T00:12:35Z

Is this going to be backported? matplotlib 1.5.2rc1 on Windows fails one test:

======================================================================
FAIL: test_csv2txt_basic (matplotlib.tests.test_mlab.rec2txt_testcase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python35\lib\site-packages\matplotlib\tests\test_mlab.py", line 409, in test_csv2txt_basic
    assert_equal(mlab.rec2txt(a), truth)
AssertionError: '       x   y   s    s2\r\n   1.000   2   foo  bing \r\n   2.[16 chars]lah ' != '       x   y   s   s2\n   1.000   2   foo bing \n   2.000   3   bar blah '
-        x   y   s    s2
?                 -     -
+        x   y   s   s2
-    1.000   2   foo  bing
?                   -      -
+    1.000   2   foo bing
-    2.000   3   bar  blah ?                   -
+    2.000   3   bar blah

----------------------------------------------------------------------
Ran 5209 tests in 1399.371s

tacaswell · 2016-05-23T00:28:01Z

Yes, sorry this got lost in the appveyor related issues.

On Sun, May 22, 2016, 20:12 Christoph Gohlke notifications@github.com
wrote:

Is this going to be backported? matplotlib 1.5.2rc1 on Windows fails one
test:

FAIL: test_csv2txt_basic (matplotlib.tests.test_mlab.rec2txt_testcase)

Traceback (most recent call last):
File "X:\Python35\lib\site-packages\matplotlib\tests\test_mlab.py", line 409, in test_csv2txt_basic
assert_equal(mlab.rec2txt(a), truth)
AssertionError: ' x y s s2\r\n 1.000 2 foo bing \r\n 2.[16 chars]lah ' != ' x y s s2\n 1.000 2 foo bing \n 2.000 3 bar blah '
   x   y   s    s2
? - -
   x   y   s   s2
1.000 2 foo bing
? - -

1.000 2 foo bing

2.000 3 bar blah ? -

2.000 3 bar blah
Ran 5209 tests in 1399.371s

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#6423 (comment)

TST: splitlines in rec2txt test

tacaswell · 2016-05-23T01:28:41Z

backported to v1.5.x as cab8cf7

TST: splitlines in rec2txt test

966f50b

On windows the output has '\r\n' instead of '\n' for new lines.

tacaswell added the status: needs review label May 14, 2016

FIX: compute str column width

1900372

The division was always returning 1, use length of first element instead.

tacaswell added this to the 2.0 (style change major release) milestone May 16, 2016

dopplershift reviewed May 16, 2016
View reviewed changes

tacaswell force-pushed the tst_fix_windows_print_test branch from a48c04b to 1900372 Compare May 17, 2016 00:48

CI: disable conda package building

0cb2c3a

tacaswell force-pushed the tst_fix_windows_print_test branch from 8577491 to 0cb2c3a Compare May 17, 2016 01:12

FIX: try more reliable way to get string length

7765024

Fixed width byte/unicode dtypes seem to have the pattern "[|<>][US][1-9][0-9]*" thus if we drop the first two charters we will have the fixed width.

tacaswell force-pushed the tst_fix_windows_print_test branch from 137f123 to 7765024 Compare May 17, 2016 01:33

jenshnielsen merged commit 326cc05 into matplotlib:master May 17, 2016

mdboom removed the status: needs review label May 17, 2016

jenshnielsen mentioned this pull request May 17, 2016

Showraise gtk gtk3 #6417

Merged

tacaswell deleted the tst_fix_windows_print_test branch May 22, 2016 15:49

tacaswell mentioned this pull request May 22, 2016

re-enable building conda package artifacts on appveyor / fix conda recipe #6460

Closed

tacaswell pushed a commit that referenced this pull request May 23, 2016

Merge pull request #6423 from tacaswell/tst_fix_windows_print_test

cab8cf7

TST: splitlines in rec2txt test

QuLogic added this to the 1.5.2 (Critical bug fix release) milestone May 23, 2016

QuLogic removed this from the 2.0 (style change major release) milestone May 23, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TST: splitlines in rec2txt test #6423

TST: splitlines in rec2txt test #6423

tacaswell commented May 14, 2016

tacaswell commented May 14, 2016

tacaswell commented May 15, 2016

dopplershift commented May 15, 2016

tacaswell commented May 16, 2016 •

edited

tacaswell commented May 16, 2016

tacaswell commented May 16, 2016

jankatins commented May 16, 2016 •

edited

tacaswell commented May 16, 2016

dopplershift May 16, 2016 •

edited

tacaswell May 16, 2016

dopplershift May 16, 2016 •

edited

dopplershift May 16, 2016

QuLogic May 17, 2016

tacaswell May 17, 2016

tacaswell commented May 17, 2016

tacaswell commented May 17, 2016

cgohlke commented May 23, 2016

tacaswell commented May 23, 2016

FAIL: test_csv2txt_basic (matplotlib.tests.test_mlab.rec2txt_testcase)

tacaswell commented May 23, 2016

TST: splitlines in rec2txt test #6423

TST: splitlines in rec2txt test #6423

Conversation

tacaswell commented May 14, 2016

tacaswell commented May 14, 2016

tacaswell commented May 15, 2016

dopplershift commented May 15, 2016

tacaswell commented May 16, 2016 • edited

tacaswell commented May 16, 2016

tacaswell commented May 16, 2016

jankatins commented May 16, 2016 • edited

tacaswell commented May 16, 2016

dopplershift May 16, 2016 • edited

Choose a reason for hiding this comment

tacaswell May 16, 2016

Choose a reason for hiding this comment

dopplershift May 16, 2016 • edited

Choose a reason for hiding this comment

dopplershift May 16, 2016

Choose a reason for hiding this comment

QuLogic May 17, 2016

Choose a reason for hiding this comment

tacaswell May 17, 2016

Choose a reason for hiding this comment

tacaswell commented May 17, 2016

tacaswell commented May 17, 2016

cgohlke commented May 23, 2016

tacaswell commented May 23, 2016

FAIL: test_csv2txt_basic (matplotlib.tests.test_mlab.rec2txt_testcase)

tacaswell commented May 23, 2016

tacaswell commented May 16, 2016 •

edited

jankatins commented May 16, 2016 •

edited

dopplershift May 16, 2016 •

edited

dopplershift May 16, 2016 •

edited