Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

test_unicode fails on Python 3.3 #2750

Closed
cgohlke opened this Issue Nov 17, 2012 · 6 comments

Comments

Projects
None yet
4 participants
Contributor

cgohlke commented Nov 17, 2012

Test_unicode fails with numpy-1.7.0rc1.dev-win*-py3.3 (3a52aa0), built with Visual Studio 2010 and Intel MKL 11. All other tests pass.

======================================================================
FAIL: Check assignment of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 209, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 223, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 215, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 209, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 223, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 215, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 209, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 223, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check assignment of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 215, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 199, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 123, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 134, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 128, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 123, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 134, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 128, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of 0-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 123, in test_values0D
    self.content_check(ua, ua[()], 4*self.ulen)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of multi-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 134, in test_valuesMD
    self.content_check(ua, ua[0,0,0], 4*self.ulen*2*3*4)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true

======================================================================
FAIL: Check creation of single-dimensional objects with values
----------------------------------------------------------------------
Traceback (most recent call last):
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 128, in test_valuesSD
    self.content_check(ua, ua[0], 4*self.ulen*2)
  File "X:\Python33\lib\site-packages\numpy\core\tests\test_unicode.py", line 114, in content_check
    self.assertTrue(buffer_length(ua_scalar) == 2*2*self.ulen)
AssertionError: False is not true
Owner

rgommers commented Dec 16, 2012

@cgohlke do you have time to have a look at this? Is it critical to fix for the 1.7 release?

Contributor

cgohlke commented Dec 16, 2012

Can this be reproduced on other platforms?
It seems that numpy/core/tests/test_unicode.py has not been updated for the new flexible string representation. On win-py3.3, the buffer_length function returns the wrong value for strings of ucs4_value because it assumes ucs2 storage, as determined by array.array('u').itemsize, which is deprecated btw.
This would be easy to fix if there was a way (independent of numpy) to determine the character storage size used by a Python 3.3 string. The C API macros PyUnicode_KIND and PyUnicode_CHARACTER_SIZE are not exported in the Python DLL.
There might be other workarounds, or the tests could be marked knownfail for py3.3 for the release.

Owner

rgommers commented Dec 16, 2012

I can't reproduce this on Linux.

Seems indeed OK to mark as knownfail for 1.7

y-p commented Dec 16, 2012

python3.3:

>>> def get_basesize(s):
...     import sys
...     return(sys.getsizeof(s+"a")-sys.getsizeof(s))
... 
>>> 
>>> print(get_basesize("a"))
1
>>> print(get_basesize(u"\u1f00"))
2
>>> print(get_basesize(u"\U0010FFFF"))
4
Contributor

cgohlke commented Dec 17, 2012

@y-p Thank you, that worked.

Contributor

certik commented Dec 21, 2012

This should be fixed now.

@certik certik closed this Dec 21, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment