Unit tests fail on i686

[bochecha]

I'm trying to update the Fedora Rawhide package ot pygit2 0.22 (we already have libgit2 0.22 there).

On x86_64, unit tests run just fine.

However, on i686, these 3 fail:

======================================================================
ERROR: test_push_fast_forward_commits_to_remote_succeeds (test.test_remote.PushTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygit2-0.22.0/test/test_remote.py", line 277, in test_push_fast_forward_commits_to_remote_succeeds
    self.remote.push(['refs/heads/master'])
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/remote.py", line 337, in push
    check_error(err)
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/errors.py", line 41, in check_error
    message = ffi.string(giterr.message).decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in position 17: ordinal not in range(128)

======================================================================
ERROR: test_push_non_fast_forward_commits_to_remote_fails (test.test_remote.PushTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygit2-0.22.0/test/test_remote.py", line 298, in test_push_non_fast_forward_commits_to_remote_fails
    self.assertRaises(pygit2.GitError, self.remote.push, ['refs/heads/master'])
  File "/usr/lib/python2.7/unittest/case.py", line 511, in assertRaises
    callableObj(*args, **kwargs)
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/remote.py", line 337, in push
    check_error(err)
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/errors.py", line 41, in check_error
    message = ffi.string(giterr.message).decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8b in position 16: ordinal not in range(128)

======================================================================
ERROR: test_push_when_up_to_date_succeeds (test.test_remote.PushTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygit2-0.22.0/test/test_remote.py", line 281, in test_push_when_up_to_date_succeeds
    self.remote.push(['refs/heads/master'])
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/remote.py", line 337, in push
    check_error(err)
  File "/builddir/build/BUILD/pygit2-0.22.0/build/lib.linux-i686-2.7/pygit2/errors.py", line 41, in check_error
    message = ffi.string(giterr.message).decode()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in position 17: ordinal not in range(128)

----------------------------------------------------------------------
Ran 225 tests in 15.428s

FAILED (errors=3)

I'm still trying to investigate the root cause, so I'm not sure this is a problem in cffi, libgit2 or pygit2, so I'm reporting it here for now, in case someone has a brilliant idea. 😃

[bochecha]

So, looking into this a bit more, it seems that in the 3 failing cases, these are the error messages returned by ffi.string(giterr.message):

invalid refspec E��pE������
invalid refspec �� Օ������
invalid refspec v�@�j�

So I'd say it's expected that .decode() doesn't work on them.

Now, where do these bytes come from? 😕

[carlosmn]

That's uninitialized memory. Something is releasing the string before the C code gets a look at it.

[bochecha]

So, running the tests in a loop, they don't always fail.

Running them in valgrind, they consistently succeed. 😕

[carlosmn]

That's what memory errors look like. Using valgrind on your "normal" python isn't going to work very well. You'd need a build of the python interpreter which is built with debug symbols and with valgrind support; that is, that it allocates and releases memory directly to the OS (or valgrind in this case). The python interpreter has its own allocator for speed which means valgrind is mostly useless against the default versions since it can't see what it's actually doing to the memory.