double free or corruption (out)

[cwaeland]

Our setup:

• OS: Ubuntu 12.04
• Python: 2.7
• pylibmc: 1.3.0
• libmemcached-devel-1.4.13

GDB trace (only a few symbols unfortunately):

cat README
got _pylibmc.so from a frontend:
scp production-frontend-721ad59d.managed.compass.com:/home/urbancompass/.pex/install/pylibmc-1.3.0-cp27-none-linux_x86_64.whl.d8b8feefa7058b4e65dafa31ab1d091bfd04117a/pylibmc-1.3.0-cp27-none-linux_x86_64.whl/_pylibmc.so ./
Also copied the stacktrace from chat.
Downloaded gdb binaries:
sudo apt-get install gdb
$ gdb
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>.
(gdb) file ./_pylibmc.so
Reading symbols from /home/ugo/tmp/pylibmc_debug/_pylibmc.so...done.
(gdb) info symbol 0x4304
_PylibMC_Inflate + 180 in section .text
(gdb) info symbol 0x7072
_PylibMC_parse_memcached_value + 434 in section .text
(gdb) info symbol 0x760c
PylibMC_Client_get + 268 in section .text
(gdb) quit

Seems to be in _PylibMC_Inflate. So possibly related to: #205

Note we are pushing some payloads that are over 500K to memached.

I've looked into a possible version upgrade to 1.5.0, but after diffing the code there doesn't seem to be any change in that function.

It seems like it probably gets to the error: label which frees out and somehow out is already freed. Perhaps it's an issue with realloc or in zlib?

Any help would be appreciated.

[lericson]

I'm looking into this in the aforementioned issue, very probably a double free due to realloc. Will exercise this code path better in tests. Thanks for reporting!

• Ludvig

On 13 feb. 2016, at 00:24, Cameron Waeland notifications@github.com wrote:

Our setup:

OS: Ubuntu 12.04
Python: 2.7
pylibmc: 1.3.0
libmemcached-devel-1.4.13
GDB trace (only a few symbols unfortunately):

cat README
got _pylibmc.so from a frontend:
scp production-frontend-721ad59d.managed.compass.com:/home/urbancompass/.pex/install/pylibmc-1.3.0-cp27-none-linux_x86_64.whl.d8b8feefa7058b4e65dafa31ab1d091bfd04117a/pylibmc-1.3.0-cp27-none-linux_x86_64.whl/_pylibmc.so ./
Also copied the stacktrace from chat.
Downloaded gdb binaries:
sudo apt-get install gdb
$ gdb
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
http://bugs.launchpad.net/gdb-linaro/.
(gdb) file ./_pylibmc.so
Reading symbols from /home/ugo/tmp/pylibmc_debug/_pylibmc.so...done.
(gdb) info symbol 0x4304
_PylibMC_Inflate + 180 in section .text
(gdb) info symbol 0x7072
_PylibMC_parse_memcached_value + 434 in section .text
(gdb) info symbol 0x760c
PylibMC_Client_get + 268 in section .text
(gdb) quit
Seems to be in _PylibMC_Inflate. So possibly related to: #205

Note we are pushing some payloads that are over 500K to memached.

I've looked into a possible version upgrade to 1.5.0, but after diffing the code there doesn't seem to be any change in that function.

It seems like it probably gets to the error: label which frees out and somehow out is already freed. Perhaps it's an issue with realloc or in zlib?

Any help would be appreciated.

—
Reply to this email directly or view it on GitHub.

[cwaeland]

Hey Ludvig,

Thanks for the quick response.

In the meantime I was wondering if there was anything we can do on our side in the meantime. Right now, when we see that error it's basically killing the Python process. Is there a way to handle that error more gracefully?

Thanks.

[ltdowney]

Here's a pull request that should address this:

#208

Wish there was better test coverage here. I'll add a note to Pivotal, Cam.

[lericson]

fixed in #205