random heap corruption using threads

[tjac]

I am using python-magic in an application where I spawn off a working thread that is given a python-magic object that is already loaded. Within the thread I use the from_buffer method to get the magic info from a previously loaded file. At first i was having minor response corruption issues where the first few bytes of the response were corrupted every so often (not consistently). I fixed this by reloading the magic object ever 20 uses. This project requires an extremely fast response time so reloading magic every single time would not work. As you can see below (after my code snippet) it looks like somewhere between magic and the libmagic something is getting freed twice which is making glibc very unhappy.

As an added data point, I typically run four of these programs in parallel on the same machine. When one of them dies via this error, the remaining instances also die. This is odd since they arent forked but run separately from the commandline. I'm using Ubuntu 10.04 LTS

<--- code snippet --->

def RefreshMagicObject(self):
    self.m_dwMagicUseCount = 0
    if self.m_oMagic != None:
        try:
            del self.m_oMagic
        except:
            pass

    self.m_oMagic = magic.Magic(magic_file="/db/magic")
    if self.m_oMagic == None:
        return False
    return True

def GetFileTypeString(self, carrFilestream):
    if self.m_dwMagicUseCount > 20:
        if self.RefreshMagicObject() == False:
            return "$$MAGICERROR$$"
    # increment the use count
    self.m_dwMagicUseCount += 1

    if self.m_oMagic == None: return None
    strMagic = self.m_oMagic.from_buffer(carrFilestream)

<---- error on console ---->

*** glibc detected *** python2.7: double free or corruption (fasttop): 0x00007f121198e9b0 ***
======= Backtrace: =========
/lib/libc.so.6[0x7f1224e27928]
/lib/libc.so.6(cfree+0x76)[0x7f1224e29a36]
/usr/lib/libmagic.so.1(file_vprintf+0x51)[0x7f1220ebdab1]
/usr/lib/libmagic.so.1(file_printf+0x88)[0x7f1220ebdb88]
/usr/lib/libmagic.so.1[0x7f1220eb632c]
/usr/lib/libmagic.so.1(file_softmagic+0x22b)[0x7f1220eb69fb]
/usr/lib/libmagic.so.1(file_buffer+0x18b)[0x7f1220ebdd1b]
/usr/lib/libmagic.so.1(magic_buffer+0x55)[0x7f1220eb0385]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(ffi_call_unix64+0x4c)[0x7f12216ee758]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(ffi_call+0x19a)[0x7f12216edeba]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so(_CallProc+0x322)[0x7f12216e8ba2]
/usr/local/lib/python2.7/site-packages/ctypes-1.0.2-py2.7-linux-x86_64.egg/_ctypes.so[0x7f12216e221f]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_EvalFrameEx+0x3477)[0x4a9077]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalCodeEx+0x809)[0x4ad5c9]
python2.7[0x5104cb]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_EvalFrameEx+0x1846)[0x4a7446]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalFrameEx+0x6d5b)[0x4ac95b]
python2.7(PyEval_EvalCodeEx+0x809)[0x4ad5c9]
python2.7[0x5103ce]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7[0x423738]
python2.7(PyObject_Call+0x5d)[0x41bc4d]
python2.7(PyEval_CallObjectWithKeywords+0x56)[0x4a4a96]
python2.7[0x4e4fa2]
/lib/libpthread.so.0[0x7f1225797fc7]
/lib/libc.so.6(clone+0x6d)[0x7f1224e8359d]
======= Memory map: ========
00400000-0056f000 r-xp 00000000 08:05 56555 /usr/local/bin/python2.7
0076f000-007ab000 rw-p 0016f000 08:05 56555 /usr/local/bin/python2.7
007ab000-007ba000 rw-p 007ab000 00:00 0
01ef2000-357d9000 rw-p 01ef2000 00:00 0 [heap]
402c8000-402c9000 ---p 402c8000 00:00 0
402c9000-40ac9000 rwxp 402c9000 00:00 0
40ac9000-40aca000 ---p 40ac9000 00:00 0
40aca000-412ca000 rwxp 40aca000 00:00 0
41a94000-41a95000 ---p 41a94000 00:00 0
41a95000-42295000 rwxp 41a95000 00:00 0
42295000-42296000 ---p 42295000 00:00 0
42296000-42a96000 rwxp 42296000 00:00 0
42a96000-42a97000 ---p 42a96000 00:00 0
42a97000-43297000 rwxp 42a97000 00:00 0
43297000-43298000 ---p 43297000 00:00 0
43298000-43a98000 rwxp 43298000 00:00 0
43a98000-43a99000 ---p 43a98000 00:00 0
43a99000-44299000 rwxp 43a99000 00:00 0
44299000-4429a000 ---p 44299000 00:00 0
4429a000-44a9a000 rwxp 4429a000 00:00 0
44a9a000-44a9b000 ---p 44a9a000 00:00 0
44a9b000-4529b000 rwxp 44a9b000 00:00 0
4529b000-4529c000 ---p 4529b000 00:00 0
4529c000-45a9c000 rwxp 4529c000 00:00 0
45a9c000-45a9d000 ---p 45a9c000 00:00 0
45a9d000-4629d000 rwxp 45a9d000 00:00 0
4629d000-4629e000 ---p 4629d000 00:00 0
4629e000-46a9e000 rwxp 4629e000 00:00 0
46a9e000-46a9f000 ---p 46a9e000 00:00 0
46a9f000-4729f000 rwxp 46a9f000 00:00 0
4729f000-472a0000 ---p 4729f000 00:00 0
472a0000-47aa0000 rwxp 472a0000 00:00 0
47aa0000-47aa1000 ---p 47aa0000 00:00 0
47aa1000-482a1000 rwxp 47aa1000 00:00 0
482a1000-482a2000 ---p 482a1000 00:00 0
482a2000-48aa2000 rwxp 482a2000 00:00 0
48aa2000-48aa3000 ---p 48aa2000 00:00 0
48aa3000-492a3000 rwxp 48aa3000 00:00 0
492a3000-492a4000 ---p 492a3000 00:00 0
492a4000-49aa4000 rwxp 492a4000 00:00 0
49aa4000-49aa5000 ---p 49aa4000 00:00 0
49aa5000-4a2a5000 rwxp 49aa5000 00:00 0
4a2a5000-4a2a6000 ---p 4a2a5000 00:00 0
4a2a6000-4aaa6000 rwxp 4a2a6000 00:00 0
4aaa6000-4aaa7000 ---p 4aaa6000 00:00 0
4aaa7000-4b2a7000 rwxp 4aaa7000 00:00 0
4b2a7000-4b2a8000 ---p 4b2a7000 00:00 0
4b2a8000-4baa8000 rwxp 4b2a8000 00:00 0
4baa8000-4baa9000 ---p 4baa8000 00:00 0
4baa9000-4c2a9000 rwxp 4baa9000 00:00 0
4c2a9000-4c2aa000 ---p 4c2a9000 00:00 0
4c2aa000-4caaa000 rwxp 4c2aa000 00:00 0
4caaa000-4caab000 ---p 4caaa000 00:00 0
4caab000-4d2ab000 rwxp 4caab000 00:00 0
4d2ab000-4d2ac000 ---p 4d2ab000 00:00 0
4d2ac000-4daac000 rwxp 4d2ac000 00:00 0
4daac000-4daad000 ---p 4daac000 00:00 0
4daad000-4e2ad000 rwxp 4daad000 00:00 0
4e2ad000-4e2ae000 ---p 4e2ad000 00:00 0
4e2ae000-4eaae000 rwxp 4e2ae000 00:00 0
4eaae000-4eaaf000 ---p 4eaae000 00:00 0
4eaaf000-4f2af000 rwxp 4eaaf000 00:00 0
4f2af000-4f2b0000 ---p 4f2af000 00:00 0
4f2b0000-4fab0000 rwxp 4f2b0000 00:00 0
4fab0000-4fab1000 ---p 4fab0000 00:00 0
4fab1000-502b1000 rwxp 4fab1000 00:00 0
502b1000-502b2000 ---p 502b1000 00:00 0
502b2000-50ab2000 rwxp 502b2000 00:00 0
50ab2000-50ab3000 ---p 50ab2000 00:00 0
50ab3000-512b3000 rwxp 50ab3000 00:00 0
512b3000-512b4000 ---p 512b3000 00:00 0
512b4000-51ab4000 rwxp 512b4000 00:00 0
51ab4000-51ab5000 ---p 51ab4000 00:00 0
51ab5000-522b5000 rwxp 51ab5000 00:00 0
522b5000-522b6000 ---p 522b5000 00:00 0
522b6000-52ab6000 rwxp 522b6000 00:00 0
52ab6000-52ab7000 ---p 52ab6000 00:00 0
52ab7000-532b7000 rwxp 52ab7000 00:00 0
532b7000-532b8000 ---p 532b7000 00:00 0
532b8000-53ab8000 rwxp 532b8000 00:00 0
53ab8000-53ab9000 ---p 53ab8000 00:00 0
53ab9000-542b9000 rwxp 53ab9000 00:00 0
542b9000-542ba000 ---p 542b9000 00:00 0
542ba000-54aba000 rwxp 542ba000 00:00 0
54aba000-54abb000 ---p 54aba000 00:00 0
54abb000-552bb000 rwxp 54abb000 00:00 0
552bb000-552bc000 ---p 552bb000 00:00 0
552bc000-55abc000 rwxp 552bc000 00:00 0
55abc000-55abd000 ---p 55abc000 00:00 0
55abd000-562bd000 rwxp 55abd000 00:00 0
562bd000-562be000 ---p 562bd000 00:00 0
562be000-56abe000 rwxp 562be000 00:00 0
56abe000-56abf000 ---p 56abe000 00:00 0
56abf000-572bf000 rwxp 56abf000 00:00 0
572bf000-572c0000 ---p 572bf000 00:00 0
572c0000-57ac0000 rwxp 572c0000 00:00 0
57ac0000-57ac1000 ---p 57ac0000 00:00 0
57ac1000-582c1000 rwxp 57ac1000 00:00 0
582c1000-582c2000 ---p 582c1000 00:00 0
582c2000-58ac2000 rwxp 582c2000 00:00 0
58ac2000-58ac3000 ---p 58ac2000 00:00 0
58ac3000-592c3000 rwxp 58ac3000 00:00 0
592c3000-592c4000 ---p 592c3000 00:00 0
592c4000-59ac4000 rwxp 592c4000 00:00 0
59ac4000-59ac5000 ---p 59ac4000 00:00 0
59ac5000-5a2c5000 rwxp 59ac5000 00:00 0
5a2c5000-5a2c6000 ---p 5a2c5000 00:00 0
5a2c6000-5aac6000 rwxp 5a2c6000 00:00 0
5aac6000-5aac7000 ---p 5aac6000 00:00 0
5aac7000-5b2c7000 rwxp 5aac7000 00:00 0
5b2c7000-5b2c8000 ---p 5b2c7000 00:00 0
5b2c8000-5bac8000 rwxp 5b2c8000 00:00 0
5bac8000-5bac9000 ---p 5bac8000 00:00 0
5bac9000-5c2c9000 rwxp 5bac9000 00:00 0
5c2c9000-5c2ca000 ---p 5c2c9000 00:00 0
5c2ca000-5caca000 rwxp 5c2ca000 00:00 0
5caca000-5cacb000 ---p 5caca000 00:00 0
5cacb000-5d2cb000 rwxp 5cacb000 00:00 0
5d2cb000-5d2cc000 ---p 5d2cb000 00:00 0
5d2cc000-5dacc000 rwxp 5d2cc000 00:00 0
5dacc000-5dacd000 ---p 5dacc000 00:00 0
5dacd000-5e2cd000 rwxp 5dacd000 00:00 0
5e2cd000-5e2ce000 ---p 5e2cd000 00:00 0
5e2ce000-5eace000 rwxp 5e2ce000 00:00 0
5eace000-5eacf000 ---p 5eace000 00:00 0
5eacf000-5f2cf000 rwxp 5eacf000 00:00 0
5f2cf000-5f2d0000 ---p 5f2cf000 00:00 0
5f2d0000-5fad0000 rwxp 5f2d0000 00:00 0
5fad0000-5fad1000 ---p 5fad0000 00:00 0
5fad1000-602d1000 rwxp 5fad1000 00:00 0
602d1000-602d2000 ---p 602d1000 00:00 0
602d2000-60ad2000 rwxp 602d2000 00:00 0
60ad2000-60ad3000 ---p 60ad2000 00:00 0
60ad3000-612d3000 rwxp 60ad3000 00:00 0
612d3000-612d4000 ---p 612d3000 00:00 0
612d4000-61ad4000 rwxp 612d4000 00:00 0
61ad4000-61ad5000 ---p 61ad4000 00:00 0
61ad5000-622d5000 rwxp 61ad5000 00:00 0
622d5000-622d6000 ---p 622d5000 00:00 0
622d6000-62ad6000 rwxp 622d6000 00:00 0
62ad6000-62ad7000 ---p 62ad6000 00:00 0
62ad7000-632d7000 rwxp 62ad7000 00:00 0
632d7000-632d8000 ---p 632d7000 00:00 0
632d8000-63ad8000 rwxp 632d8000 00:00 0
63ad8000-63ad9000 ---p 63ad8000 00:00 0
63ad9000-642d9000 rwxp 63ad9000 00:00 0
642d9000-642da000 ---p 642d9000 00:00 0
642da000-64ada000 rwxp 642da000 00:00 0
64ada000-64adb000 ---p 64ada000 00:00 0
64adb000-652db000 rwxp 64adb000 00:00 0
652db000-652dc000 ---p 652db000 00:00 0
652dc000-65adc000 rwxp 652dc000 00:00 0
65adc000-65add000 ---p 65adc000 00:00 0
65add000-662dd000 rwxp 65add000 00:00 0
662dd000-662de000 ---p 662dd000 00:00 0
662de000-66ade000 rwxp 662de000 00:00 0
66ade000-66adf000 ---p 66ade000 00:00 0
66adf000-672df000 rwxp 66adf000 00:00 0
672df000-672e0000 ---p 672df000 00:00 0
672e0000-67ae0000 rwxp 672e0000 00:00 0
67ae0000-67ae1000 ---p 67ae0000 00:00 0
67ae1000-682e1000 rwxp 67ae1000 00:00 0
682e1000-682e2000 ---p 682e1000 00:00 0
682e2000-68ae2000 rwxp 682e2000 00:00 0
68ae2000-68ae3000 ---p 68ae2000 00:00 0
68ae3000-692e3000 rwxp 68ae3000 00:00 0
692e3000-692e4000 ---p 692e3000 00:00 0
692e4000-69ae4000 rwxp 692e4000 00:00 0
69ae4000-69ae5000 ---p 69ae4000 00:00 0
69ae5000-6a2e5000 rwxp 69ae5000 00:00 0
6a2e5000-6a2e6000 ---p 6a2e5000 00:00 0
6a2e6000-6aae6000 rwxp 6a2e6000 00:00 0
6aae6000-6aae7000 ---p 6aae6000 00:00 0
6aae7000-6b2e7000 rwxp 6aae7000 00:00 0
6b2e7000-6b2e8000 ---p 6b2e7000 00:00 0
6b2e8000-6bae8000 rwxp 6b2e8000 00:00 0
6bae8000-6bae9000 ---p 6bae8000 00:00 0
6bae9000-6c2e9000 rwxp 6bae9000 00:00 0
6c2e9000-6c2ea000 ---p 6c2e9000 00:00 0
6c2ea000-6caea000 rwxp 6c2ea000 00:00 0
6caea000-6caeb000 ---p 6caea000 00:00 0
6caeb000-6d2eb000 rwxp 6caeb000 00:00 0
6d2eb000-6d2ec000 ---p 6d2eb000 00:00 0
6d2ec000-6daec000 rwxp 6d2ec000 00:00 0
6daec000-6daed000 ---p 6daec000 00:00 0
6daed000-6e2ed000 rwxp 6daed000 00:00 0
6e2ed000-6e2ee000 ---p 6e2ed000 00:00 0
6e2ee000-6eaee000 rwxp 6e2ee000 00:00 0
6eaee000-6eaef000 ---p 6eaee000 00:00 0
6eaef000-6f2ef000 rwxp 6eaef000 00:00 0
6f2ef000-6f2f0000 ---p 6f2ef000 00:00 0
6f2f0000-6faf0000 rwxp 6f2f0000 00:00 0
6faf0000-6faf1000 ---p 6faf0000 00:00 0
6faf1000-702f1000 rwxp 6faf1000 00:00 0
702f1000-702f2000 ---p 702f1000 00:00 0
702f2000-70af2000 rwxp 702f2000 00:00 0
70af2000-70af3000 ---p 70af2000 00:00 0
70af3000-712f3000 rwxp 70af3000 00:00 0
712f3000-712f4000 ---p 712f3000 00:00 0
712f4000-71af4000 rwxp 712f4000 00:00 0
71af4000-71af5000 ---p 71af4000 00:00 0
71af5000-722f5000 rwxp 71af5000 00:00 0
722f5000-722f6000 ---p 722f5000 00:00 0
722f6000-72af6000 rwxp 722f6000 00:00 0
72af6000-72af7000 ---p 72af6000 00:00 0
72af7000-732f7000 rwxp 72af7000 00:00 0
732f7000-732f8000 ---p 732f7000 00:00 0
732f8000-73af8000 rwxp 732f8000 00:00 0
73af8000-73af9000 ---p 73af8000 00:00 0
73af9000-742f9000 rwxp 73af9000 00:00 0
742f9000-742fa000 ---p 742f9000 00:00 0
742fa000-74afa000 rwxp 742fa000 00:00 0
74afa000-74afb000 ---p 74afa000 00:00 0
74afb000-752fb000 rwxp 74afb000 00:00 0
752fb000-752fc000 ---p 752fb000 00:00 0
752fc000-75afc000 rwxp 752fc000 00:00 0
75afc000-75afd000 ---p 75afc000 00:00 0
75afd000-762fd000 rwxp 75afd000 00:00 0
762fd000-762fe000 ---p 762f

[ahupp]

Hi, sorry for the delayed response. I'm not entirely sure what's going on. It's extremely odd that you see all instances die at once, I can't think of a way that would happen.

One possibility is that somehow magic_close() is being called on self.cookie and then that cookie is re-used. I don't see how that could happen. I did add some defensive checks here:

4142c79

If you use a current version of the wrapper does the problem still occur?

Another option (unlikely) is some bug in file_vprintf. I've traced through the logic there and am somewhat confident it won't double free ms->o.buf, but it's hard to be sure.

Now, I believe the GIL is held during all these calls. If it is not held then that's a whole different problem.

[ahupp]

I'm going to optimistically assume that this is resolved, if not please re-open.

[devendralaulkar]

I am facing a similar issue. We are using python-jira library, which depends upon python-magic==0.4.2
Running under Django 1.4.2 Ubuntu 12.04 64 bit - After making a call from jira python, the Django server segfaults.

*** glibc detected *** python: free(): invalid pointer: 0xb612be98 ***
======= Backtrace: =========
/lib/i386-linux-gnu/tls/i686/nosegneg/libc.so.6(+0x75ee2)[0xb73d3ee2]
/usr/lib/libmagic.so.1(magic_close+0x4a)[0xb3409f8a]
/usr/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_SYSV+0x1a)[0xb707e70a]
/usr/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x6f)[0xb707e43f]
/usr/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x7d0)[0xb706e830]
/usr/lib/python2.7/lib-dynload/_ctypes.so(+0x8ca6)[0xb706eca6]
python(PyObject_Call+0x3f)[0x81287ef]
python(PyEval_EvalFrameEx+0xab6)[0x81951e6]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python[0x819c82e]
python(PyObject_Call+0x3f)[0x81287ef]
python[0x80d8e37]
python(PyObject_Call+0x3f)[0x81287ef]
python(PyEval_CallObjectWithKeywords+0x39)[0x8128c59]
python[0x80d9268]
python(PyDict_Clear+0x1e2)[0x8175f52]
python[0x817601f]
python[0x8157b11]
python(_PyObject_GC_Malloc+0xe5)[0x8158075]
python(PyType_GenericAlloc+0x3e)[0x812a75e]
python[0x812c629]
python[0x80ef5ba]
python(PyObject_Call+0x3f)[0x81287ef]
python(PyEval_CallObjectWithKeywords+0x39)[0x8128c59]
python(PyErr_NormalizeException+0xb1)[0x80d8521]
python(PyEval_EvalFrameEx+0x390b)[0x819803b]
python(PyEval_EvalFrameEx+0x7bc)[0x8194eec]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalFrameEx+0x7bc)[0x8194eec]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalFrameEx+0x7bc)[0x8194eec]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python[0x819c972]
python(PyObject_Call+0x3f)[0x81287ef]
python(PyEval_EvalFrameEx+0x1d8b)[0x81964bb]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python[0x819c82e]
python(PyObject_Call+0x3f)[0x81287ef]
python(PyObject_CallFunctionObjArgs+0x6b)[0x815825b]
python[0x8159359]
python(_PyObject_GenericGetAttrWithDict+0x157)[0x80f15c7]
python(PyObject_GetAttr+0x47)[0x80f1797]
python(PyEval_EvalFrameEx+0x1b2)[0x81948e2]
python(PyEval_EvalFrameEx+0x7bc)[0x8194eec]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
python(PyEval_EvalFrameEx+0x6cc)[0x8194dfc]
python(PyEval_EvalCodeEx+0x150)[0x819af70]
======= Memory map: ========
08048000-0829e000 r-xp 00000000 ca:01 739 /usr/bin/python2.7
0829e000-0829f000 r--p 00256000 ca:01 739 /usr/bin/python2.7
0829f000-082f4000 rw-p 00257000 ca:01 739 /usr/bin/python2.7
082f4000-08300000 rw-p 00000000 00:00 0
0970e000-0a8a3000 rw-p 00000000 00:00 0 [heap]
b26ff000-b2700000 rw-p 00000000 00:00 0
b2700000-b2821000 rw-p 00000000 00:00 0
b2821000-b2900000 ---p 00000000 00:00 0
b2900000-b29f4000 rw-p 00000000 00:00 0
b29f4000-b2a00000 ---p 00000000 00:00 0
b2af2000-b2af7000 r-xp 00000000 ca:01 393658 /lib/i386-linux-gnu/tls/i686/nosegneg/libnss_dns-2.15.so
b2af7000-b2af8000 r--p 00004000 ca:01 393658 /lib/i386-linux-gnu/tls/i686/nosegneg/libnss_dns-2.15.so
b2af8000-b2af9000 rw-p 00005000 ca:01 393658 /lib/i386-linux-gnu/tls/i686/nosegneg/libnss_dns-2.15.so
b2b00000-b2c00000 rw-p 00000000 00:00 0
b2cf3000-b2cfe000 r-xp 00000000 ca:01 7619 /usr/lib/python2.7/lib-dynload/_elementtree.so
b2cfe000-b2cff000 r--p 0000a000 ca:01 7619 /usr/lib/python2.7/lib-dynload/_elementtree.so
b2cff000-b2d00000 rw-p 0000b000 ca:01 7619 /usr/lib/python2.7/lib-dynload/_elementtree.so
b2d00000-b2dc5000 rw-p 00000000 00:00 0
b2dc5000-b2e00000 ---p 00000000 00:00 0
b2f00000-b30f1000 rw-p 00000000 00:00 0
b30f1000-b3100000 ---p 00000000 00:00 0
b3100000-b31fe000 rw-p 00000000 00:00 0
b31fe000-b3200000 ---p 00000000 00:00 0
b32f8000-b32fd000 r-xp 00000000 ca:01 7614 /usr/lib/python2.7/lib-dynload/_csv.so
b32fd000-b32fe000 r--p 00004000 ca:01 7614 /usr/lib/python2.7/lib-dynload/_csv.so
b32fe000-b3300000 rw-p 00005000 ca:01 7614 /usr/lib/python2.7/lib-dynload/_csv.so
b3300000-b33d1000 rw-p 00000000 00:00 0
b33d1000-b3400000 ---p 00000000 00:00 0
b3405000-b3420000 r-xp 00000000 ca:01 1144 /usr/lib/libmagic.so.1.0.0
b3420000-b3421000 r--p 0001a000 ca:01 1144 /usr/lib/libmagic.so.1.0.0
b3421000-b3422000 rw-p 0001b000 ca:01 1144 /usr/lib/libmagic.so.1.0.0
b3429000-b342e000 r-xp 00000000 ca:01 1548 /usr/lib/i386-linux-gnu/libgdbm.so.3.0.0
b342e000-b342f000 r--p 00004000 ca:01 1548 /usr/lib/i386-linux-gnu/libgdbm.so.3.0.0
b342f000-b3430000 rw-p 00005000 ca:01 1548 /usr/lib/i386-linux-gnu/libgdbm.so.3.0.0
b3433000-b3435000 r-xp 00000000 ca:01 7633 /usr/lib/python2.7/lib-dynload/dbm.so
b3435000-b3436000 r--p 00002000 ca:01 7633 /usr/lib/python2.7/lib-dynload/dbm.so
b3436000-b3437000 rw-p 00003000 ca:01 7633 /usr/lib/python2.7/lib-dynload/dbm.so
b3437000-b35b9000 r-xp 00000000 ca:01 1535 /usr/lib/i386-linux-gnu/libdb-5.1.so
b35b9000-b35ba000 ---p 00182000 ca:01 1535 /usr/lib/i386-linux-gnu/libdb-5.1.so
b35ba000-b35be000 r--p 00182000 ca:01 1535 /usr/lib/i386-linux-gnu/libdb-5.1.so
b35be000-b35bf000 rw-p 00186000 ca:01 1535 /usr/lib/i386-linux-gnu/libdb-5.1.so
b35bf000-b35ec000 r-xp 00000000 ca:01 7607 /usr/lib/python2.7/lib-dynload/_bsddb.so
b35ec000-b35ed000 r--p 0002c000 ca:01 7607 /usr/lib/python2.7/lib-dynload/_bsddb.so
b35ed000-b35ef000 rw-p 0002d000 ca:01 7607 /usr/lib/python2.7/lib-dynload/_bsddb.so
b35ef000-b35f0000 rw-p 00000000 00:00 0
b35f0000-b35fe000 r-xp 00000000 ca:01 3856 /usr/lib/python2.7/dist-packages/Crypto/Cipher/DES3.so
b35fe000-b35ff000 r--p 0000d000 ca:01 3856 /usr/lib/python2.7/dist-packages/Crypto/Cipher/DES3.so
b35ff000-b3600000 rw-p 0000e000 ca:01 3856 /usr/lib/python2.7/dist-packages/Crypto/Cipher/DES3.so
b3600000-b37d0000 rw-p 00000000 00:00 0
b37d0000-b3800000 ---p 00000000 00:00 0
b3800000-b3900000 rw-p 00000000 00:00 0
b3906000-b397d000 r-xp 00000000 ca:01 1552 /usr/lib/i386-linux-gnu/libgmp.so.10.0.2
b397d000-b397e000 r--p 00076000 ca:01 1552 /usr/lib/i386-linux-gnu/libgmp.so.10.0.2
b397e000-b3985000 rw-p 00077000 ca:01 1552 /usr/lib/i386-linux-gnu/libgmp.so.10.0.2
b3985000-b3995000 r-xp 00000000 ca:01 3916 /usr/lib/python2.7/dist-packages/Crypto/PublicKey/_fastmath.so
b3995000-b3996000 r--p 0000f000 ca:01 3916 /usr/lib/python2.7/dist-packages/Crypto/PublicKey/_fastmath.so
b3996000-b3997000 rw-p 00010000 ca:01 3916 /usr/lib/python2.7/dist-packages/Crypto/PublicKey/_fastmath.so
b3997000-b39f6000 r-xp 00000000 ca:01 4217 /usr/lib/python2.7/dist-packages/M2Crypto/__m2crypto.so
b39f6000-b39f7000 r--p 0005e000 ca:01 4217 /usr/lib/python2.7/dist-packages/M2Crypto/__m2crypto.so
b39f7000-b39fb000 rw-p 0005f000 ca:01 4217 /usr/lib/python2.7/dist-packages/M2Crypto/__m2crypto.so
b39fb000-b39fc000 rw-p 00000000 00:00 0
b39fc000-b39fd000 ---p 00000000 00:00 0
b39fd000-b41fd000 rw-p 00000000 00:00 0
b41fd000-b41fe000 ---p 00000000 00:00 0
b41fe000-b49fe000 rw-p 00000000 00:00 0
b49fe000-b49ff000 ---p 00000000 00:00 0
b49ff000-b51ff000 rw-p 00000000 00:00 0
b51ff000-b5200000 ---p 00000000 00:00 0
b5200000-b5a00000 rw-p 00000000 00:00 0
b5a00000-b5aea000 rw-p 00000000 00:00 0
b5aea000-b5b00000 ---p 00000000 00:00 0
b5b02000-b5b05000 r-xp 00000000 ca:01 7637 /usr/lib/python2.7/lib-dynload/gdbm.so
b5b05000-b5b06000 r--p 00002000 ca:01 7637 /usr/lib/python2.7/lib-dynload/gdbm.so
b5b06000-b5b07000 rw-p 00003000 ca:01 7637 /usr/lib/python2.7/lib-dynload/gdbm.so
b5b07000-b5b09000 r-xp 00000000 ca:01 3852 /usr/lib/python2.7/dist-packages/Crypto/Cipher/ARC4.so
b5b09000-b5b0a000 r--p 00001000 ca:01 3852 /usr/lib/python2.7/dist-packages/Crypto/Cipher/ARC4.so
b5b0a000-b5b0b000 rw-p 00002000 ca:01 3852 /usr/lib/python2.7/dist-packages/Crypto/Cipher/ARC4.so
b5b0b000-b5b0e000 r-xp 00000000 ca:01 4067 /usr/lib/python2.7/dist-packages/Crypto/Util/_counter.so
b5b0e000-b5b0f000 r--p 00002000 ca:01 4067 /usr/lib/python2.7/dist-packages/Crypto/Util/_counter.so
b5b0f000-b5b10000 rw-p 00003000 ca:01 4067 /usr/lib/python2.7/dist-packages/Crypto/Util/_counter.so
b5b10000-b5b18000 r-xp 00000000 ca:01 3825 /usr/lib/python2.7/dist-packages/Crypto/Cipher/AES.so
b5b18000-b5b19000 r--p 00007000 ca:01 3825 /usr/lib/python2.7/dist-packages/Crypto/Cipher/AES.so
b5b19000-b5b1a000 rw-p 00008000 ca:01 3825 /usr/lib/python2.7/dist-packages/Crypto/Cipher/AES.so
b5b1a000-b5b40000 r-xp 00000000 ca:01 394376 /lib/i386-linux-gnu/libexpat.so.1.5.2
b5b40000-b5b41000 ---p 00026000 ca:01 394376 /lib/i386-linux-gnu/libexpat.so.1.5.2
b5b41000-b5b43000 r--p 00026000 ca:01 394376 /lib/i386-linux-gnu/libexpat.so.1.5.2
b5b43000-b5b44000 rw-p 00028000 ca:01 394376 /lib/i386-linux-gnu/libexpat.so.1.5.2
b5b44000-b5b4f000 r-xp 00000000 ca:01 7644 /usr/lib/python2.7/lib-dynload/pyexpat.so
b5b4f000-b5b50000 r--p 0000b000 ca:01 7644 /usr/lib/python2.7/lib-dynload/pyexpat.so
b5b50000-b5b51000 rw-p 0000c000 ca:01 7644 /usr/lib/python2.7/lib-dynload/pyexpat.so
b5b51000-b5b52000 rw-p 00000000 00:00 0
b5b52000-b5b97000 r-xp 00000000 ca:01 16944 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
b5b97000-b5b98000 r--p 00044000 ca:01 16944 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
b5b98000-b5b99000 rw-p 00045000 ca:01 16944 /usr/lib/i386-linux-gnu/libjpeg.so.8.0.2
b5b99000-b5ba9000 rw-p 00000000 00:00 0
b5ba9000-b5be5000 r-xp 00000000 ca:01 17246 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
b5be5000-b5be7000 r--p 0003b000 ca:01 17246 /usr/lib/python2.7/dist-packages/PIL/_imaging.so
b5be7000-b5be9000 rw-p 0003d000 ca:01 17246 /usr/lib/python2.7/dist-packages/PIL/_imaging.so./debug.sh: line 5: 13233 Aborted (core dumped) pytho

[thesteed]

I too am facing this double free corruption error, I am also using this in a multi-threaded environment. If there is any more information I can provide to help, let me know!

    mime_type = magic.from_file(file_path, mime=True)
    if not mime_type:
        mime_type = 'unknown'

*** glibc detected *** python: double free or corruption (fasttop): 0xb220a738 ***
======= Backtrace: =========
/lib/i386-linux-gnu/libc.so.6(+0x75ee2)[0xb758dee2]
/usr/lib/i386-linux-gnu/libmagic.so.1(file_reset+0x45)[0xb6753275]
/usr/lib/i386-linux-gnu/libmagic.so.1(+0x3091)[0xb6746091]
/usr/lib/python2.7/lib-dynload/_ctypes.so(ffi_call_SYSV+0x1a)[0xb6788a5e]
/usr/lib/python2.7/lib-dynload/_ctypes.so(ffi_call+0x6f)[0xb67887bf]
/usr/lib/python2.7/lib-dynload/_ctypes.so(_ctypes_callproc+0x7a0)[0xb677d320]
/usr/lib/python2.7/lib-dynload/_ctypes.so(+0xc7be)[0xb677d7be]
python(PyEval_EvalFrameEx+0xd25)[0x80a0c65]
python(PyEval_EvalFrameEx+0x9b8)[0x80a08f8]
python(PyEval_EvalCodeEx+0x149)[0x80a6c99]
python(PyEval_EvalFrameEx+0xa5d)[0x80a099d]
python(PyEval_EvalCodeEx+0x149)[0x80a6c99]
python(PyEval_EvalFrameEx+0xa5d)[0x80a099d]
python(PyEval_EvalCodeEx+0x149)[0x80a6c99]
python(PyEval_EvalFrameEx+0xa5d)[0x80a099d]
python(PyEval_EvalFrameEx+0x9b8)[0x80a08f8]
python(PyEval_EvalFrameEx+0x9b8)[0x80a08f8]
python(PyEval_EvalFrameEx+0x9b8)[0x80a08f8]
python(PyEval_EvalFrameEx+0x9b8)[0x80a08f8]
python(PyEval_EvalCodeEx+0x149)[0x80a6c99]
python[0x80e23ae]
python[0x80fe8d9]
python(PyEval_CallObjectWithKeywords+0x75)[0x80dd625]
python[0x81104a3]
/lib/i386-linux-gnu/libpthread.so.0(+0x6d4c)[0xb76ebd4c]
/lib/i386-linux-gnu/libc.so.6(clone+0x5e)[0xb7606d3e]
======= Memory map: ========
08048000-0825a000 r-xp 00000000 08:01 81789905 /usr/bin/python2.7
0825a000-0825b000 r--p 00211000 08:01 81789905 /usr/bin/python2.7
0825b000-082b0000 rw-p 00212000 08:01 81789905 /usr/bin/python2.7
082b0000-082bd000 rw-p 00000000 00:00 0
0934b000-0a0f5000 rw-p 00000000 00:00 0 [heap]
9fdbe000-9fdbf000 ---p 00000000 00:00 0
9fdbf000-a05bf000 rw-p 00000000 00:00 0
a1dc2000-a1dc3000 ---p 00000000 00:00 0
a1dc3000-a25c3000 rw-p 00000000 00:00 0
a2dc4000-a2dc5000 ---p 00000000 00:00 0
a2dc5000-a35c5000 rw-p 00000000 00:00 0 [stack:21658]
a35c5000-a35c6000 ---p 00000000 00:00 0
a35c6000-a3dc6000 rw-p 00000000 00:00 0
a3dc6000-a3dc7000 ---p 00000000 00:00 0
a3dc7000-a45c7000 rw-p 00000000 00:00 0 [stack:21602]
a55c9000-a55ca000 ---p 00000000 00:00 0
a55ca000-a5dca000 rw-p 00000000 00:00 0 [stack:21501]
a5dca000-a5dcb000 ---p 00000000 00:00 0
a5dcb000-a65cb000 rw-p 00000000 00:00 0
a65cb000-a65cc000 ---p 00000000 00:00 0
a65cc000-a6dcc000 rw-p 00000000 00:00 0 [stack:21361]
a6dcc000-a6dcd000 ---p 00000000 00:00 0
a6dcd000-a75cd000 rw-p 00000000 00:00 0 [stack:21296]
a75cd000-a75ce000 ---p 00000000 00:00 0
a75ce000-a7dce000 rw-p 00000000 00:00 0 [stack:21342]
a7dce000-a7dcf000 ---p 00000000 00:00 0
a7dcf000-a85cf000 rw-p 00000000 00:00 0 [stack:21168]
a85cf000-a85d0000 ---p 00000000 00:00 0
a85d0000-a8dd0000 rw-p 00000000 00:00 0 [stack:21395]
a8dd0000-a8dd1000 ---p 00000000 00:00 0
a8dd1000-a95d1000 rw-p 00000000 00:00 0 [stack:21093]
a9900000-a99bc000 rw-p 00000000 00:00 0
a99bc000-a9a00000 ---p 00000000 00:00 0
a9c00000-a9cfc000 rw-p 00000000 00:00 0
a9cfc000-a9d00000 ---p 00000000 00:00 0
a9dd2000-a9fff000 rw-p 00000000 08:01 82321969 /usr/share/file/magic.mgc
a9fff000-aa000000 ---p 00000000 00:00 0
aa000000-aa800000 rw-p 00000000 00:00 0 [stack:21089]
aa800000-aa8f2000 rw-p 00000000 00:00 0
aa8f2000-aa900000 ---p 00000000 00:00 0
aa9f1000-aa9f2000 ---p 00000000 00:00 0
aa9f2000-ab1f2000 rw-p 00000000 00:00 0 [stack:21088]
ab1f2000-ab1f3000 ---p 00000000 00:00 0
ab1f3000-ab9f3000 rw-p 00000000 00:00 0 [stack:21087]
ab9f3000-ab9f4000 ---p 00000000 00:00 0
ab9f4000-ac1f4000 rw-p 00000000 00:00 0 [stack:21086]
ac1f4000-ac1f5000 ---p 00000000 00:00 0
ac1f5000-ac9f5000 rw-p 00000000 00:00 0 [stack:21085]
ac9f5000-ac9f6000 ---p 00000000 00:00 0
ac9f6000-ad1f6000 rw-p 00000000 00:00 0 [stack:21084]
ad1f6000-ad1f7000 ---p 00000000 00:00 0
ad1f7000-ad9f7000 rw-p 00000000 00:00 0 [stack:21083]
ad9f7000-ad9f8000 ---p 00000000 00:00 0
ad9f8000-ae1f8000 rw-p 00000000 00:00 0 [stack:21082]
ae1f8000-ae1f9000 ---p 00000000 00:00 0
ae1f9000-ae9f9000 rw-p 00000000 00:00 0 [stack:21081]
ae9f9000-ae9fa000 ---p 00000000 00:00 0
ae9fa000-af1fa000 rw-p 00000000 00:00 0 [stack:21080]
af1fa000-af1fb000 ---p 00000000 00:00 0
af1fb000-af9fb000 rw-p 00000000 00:00 0 [stack:21079]
af9fb000-af9fc000 ---p 00000000 00:00 0
af9fc000-b01fc000 rw-p 00000000 00:00 0 [stack:21078]
b01fc000-b01fd000 ---p 00000000 00:00 0
b01fd000-b09fd000 rw-p 00000000 00:00 0 [stack:21076]
b0e00000-b0ee4000 rw-p 00000000 00:00 0
b0ee4000-b0f00000 ---p 00000000 00:00 0
b11fe000-b11ff000 ---p 00000000 00:00 0
b11ff000-b19ff000 rw-p 00000000 00:00 0 [stack:21077]
b19ff000-b1a00000 ---p 00000000 00:00 0
b1a00000-b2200000 rw-p 00000000 00:00 0 [stack:21071]
b2200000-b22fd000 rw-p 00000000 00:00 0
b22fd000-b2300000 ---p 00000000 00:00 0
b23ff000-b2400000 ---p 00000000 00:00 0
b2400000-b2c00000 rw-p 00000000 00:00 0 [stack:21070]
b2c00000-b2cbf000 rw-p 00000000 00:00 0
b2cbf000-b2d00000 ---p 00000000 00:00 0
b2dff000-b2e00000 ---p 00000000 00:00 0
b2e00000-b3600000 rw-p 00000000 00:00 0 [stack:21069]
b3600000-b3700000 rw-p 00000000 00:00 0
b37ff000-b3800000 ---p 00000000 00:00 0
b3800000-b4000000 rw-p 00000000 00:00 0 [stack:21068]
b4000000-b40d1000 rw-p 00000000 00:00 0
b40d1000-b4100000 ---p 00000000 00:00 0
b41ff000-b4200000 ---p 00000000 00:00 0
b4200000-b4a00000 rw-p 00000000 00:00 0 [stack:21067]
b4a00000-b4a21000 rw-p 00000000 00:00 0
b4a21000-b4b00000 ---p 00000000 00:00 0
b4bff000-b4c00000 ---p 00000000 00:00 0
b4c00000-b5400000 rw-p 00000000 00:00 0 [stack:21066]
b5400000-b54bd000 rw-p 00000000 00:00 0
b54bd000-b5500000 ---p 00000000 00:00 0
b55c9000-b55e6000 r-xp 00000000 08:01 82055385 /usr/lib/python2.7/lib-dynload/_codecs_cn.so
b55e6000-b55e7000 ---p 0001d000 08:01 82055385 /usr/lib/python2.7/lib-dynload/_codecs_cn.so
b55e7000-b55ea000 r--p 0001d000 08:01 82055385 /usr/lib/python2.7/lib-dynload/_codecs_cn.so
b55ea000-b55eb000 rw-p 00020000 08:01 82055385 /usr/lib/python2.7/lib-dynload/_codecs_cn.so
b55eb000-b55fd000 r-xp 00000000 08:01 82055421 /usr/lib/python2.7/lib-dynload/parser.so
b55fd000-b55fe000 r--p 00012000 08:01 82055421 /usr/lib/python2.7/lib-dynload/parser.so
b55fe000-b55ff000 rw-p 00013000 08:01 82055421 /usr/lib/python2.7/lib-dynload/parser.so
b55ff000-b5600000 ---p 00000000 00:00 0
b5600000-b5e00000 rw-p 00000000 00:00 0 [stack:21065]
b5e00000-b5eda000 rw-p 00000000 00:00 0
b5eda000-b5f00000 ---p 00000000 00:00 0
b5f08000-b5f0d000 r-xp 00000000 08:01 67896053 /lib/i386-linux-gnu/libnss_dns-2.15.so
b5f0d000-b5f0e000 r--p 00004000 08:01 67896053 /lib/i386-linux-gnu/libnss_dns-2.15.so
b5f0e000-b5f0f000 rw-p 00005000 08:01 67896053 /lib/i386-linux-gnu/libnss_dns-2.15.so
b5f0f000-b5f10000 ---p 00000000 00:00 0
b5f10000-b6710000 rw-p 00000000 00:00 0 [stack:21063]
b6710000-b671b000 r-xp 00000000 08:01 67896055 /lib/i386-linux-gnu/libnss_files-2.15.so
b671b000-b671c000 r--p 0000a000 08:01 67896055 /lib/i386-linux-gnu/libnss_files-2.15.so
b671c000-b671d000 rw-p 0000b000 08:01 67896055 /lib/i386-linux-gnu/libnss_files-2.15.so
b671d000-b6727000 r-xp 00000000 08:01 67896059 /lib/i386-linux-gnu/libnss_nis-2.15.so
b6727000-b6728000 r--p 00009000 08:01 67896059 /lib/i386-linux-gnu/libnss_nis-2.15.so
b6728000-b6729000 rw-p 0000a000 08:01 67896059 /lib/i386-linux-gnu/libnss_nis-2.15.so
b6729000-b673f000 r-xp 00000000 08:01 67896049 /lib/i386-linux-gnu/libnsl-2.15.so
b673f000-b6740000 r--p 00015000 08:01 67896049 /lib/i386-linux-gnu/libnsl-2.15.so
b6740000-b6741000 rw-p 00016000 08:01 67896049 /lib/i386-linux-gnu/libnsl-2.15.so
b6741000-b6743000 rw-p 00000000 00:00 0
b6743000-b675d000 r-xp 00000000 08:01 81794140 /usr/lib/i386-linux-gnu/libmagic.so.1.0.0
b675d000-b675e000 r--p 0001a000 08:01 81794140 /usr/lib/i386-linux-gnu/libmagic.so.1.0.0
b675e000-b675f000 rw-p 0001b000 08:01 81794140 /usr/lib/i386-linux-gnu/libmagic.so.1.0.0
b6769000-b676f000 r-xp 00000000 08:01 82055402 /usr/lib/python2.7/lib-dynload/_multibytecodec.so
b676f000-b6770000 r--p 00005000 08:01 82055402 /usr/lib/python2.7/lib-dynload/_multibytecodec.so
b6770000-b6771000 rw-p 00006000 08:01 82055402 /usr/lib/python2.7/lib-dynload/_multibytecodec.so
b6771000-b678c000 r-xp 00000000 08:01 82055392 /usr/lib/python2.7/lib-dynload/_ctypes.so
b678c000-b6791000 r--p 0001a000 08:01 82055392 /usr/lib/python2.7/lib-dynload/_ctypes.so
b6791000-b6794000 rw-p 0001f000 08:01 82055392 /usr/lib/python2.7/lib-dynload/_ctypes.so
b6794000-b679c000 r-xp 00000000 08:01 67895998 /lib/i386-linux-gnu/libcrypt-2.15.so
b679c000-b679d000 r--p 00007000 08:01 67895998 /lib/i386-linux-gnu/libcrypt-2.15.so
b679d000-b679e000 rw-p 00008000 08:01 67895998 /lib/i386-linux-gnu/libcrypt-2.15.so
b679e000-b67c5000 rw-p 00000000 00:00 0
b67c5000-b6870000 r-xp 00000000 08:01 81794322 /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
b6870000-b6871000 r--p 000ab000 08:01 81794322 /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
b6871000-b6872000 rw-p 000ac000 08:01 81794322 /usr/lib/i386-linux-gnu/libsqlite3.so.0.8.6
b6872000-b6873000 rw-p 00000000 00:00 0
b6873000-b68b5000 r-xp 00000000 08:01 81794064 /usr/lib/i386-linux-gnu/libhx509.so.5.0.0
b68b5000-b68b6000 r--p 00042000 08:01 81794064 /usr/lib/i386-linux-gnu/libhx509.so.5.0.0
b68b6000-b68b7000 rw-p 00043000 08:01 81794064 /usr/lib/i386-linux-gnu/libhx509.so.5.0.0
b68b7000-b68b8000 rw-p 00000000 00:00 0
b68b8000-b68c5000 r-xp 00000000 08:01 81794058 /usr/lib/i386-linux-gnu/libheimbase.so.1.0.0
b68c5000-b68c6000 r--p 0000c000 08:01 81794058 /usr/lib/i386-linux-gnu/libheimbase.so.1.0.0
b68c6000-b68c7000 rw-p 0000d000 08:01 81794058 /usr/lib/i386-linux-gnu/libheimbase.so.1.0.0
b68c7000-b68ee000 r-xp 00000000 08:01 81794406 /usr/lib/i386-linux-gnu/libwind.so.0.0.0
b68ee000-b68ef000 r--p 00027000 08:01 81794406 /usr/lib/i386-linux-gnu/libwind.so.0.0.0
b68ef000-b68f0000 rw-p 00028000 08:01 81794406 /usr/lib/i386-linux-gnu/libwind.so.0.0.0
b68f0000-b68f3000 r-xp 00000000 08:01 67896021 /lib/i386-linux-gnu/libgpg-error.so.0.8.0
b68f3000-b68f4000 r--p 00002000 08:01 67896021 /lib/i386-linux-gnu/libgpg-error.so.0.8.0
b68f4000-b68f5000 rw-p 00003000 08:01 67896021 /lib/i386-linux-gnu/libgpg-error.so.0.8.0
b68f5000-b6907000 r-xp 00000000 08:01 81794197 /usr/lib/i386-linux-gnu/libp11-kit.so.0.0.0
b6907000-b6908000 r--p 00011000 08:01 81794197 /usr/lib/i386-linux-gnu/libp11-kit.so.0.0.0
b6908000-b6909000 rw-p 00012000 08:01 81794197 /usr/lib/i386-linux-gnu/libp11-kit.so.0.0.0
b6909000-b6919000 r-xp 00000000 08:01 81794340 /usr/lib/i386-linux-gnu/libtasn1.so.3.1.16
b6919000-b691a000 r--p 0000f000 08:01 81794340 /usr/lib/i386-linux-gnu/libtasn1.so.3.1.16
b691a000-b691b000 rw-p 00010000 08:01 81794340 /usr/lib/i386-linux-gnu/libtasn1.so.3.1.16
b691b000-b692e000 r-xp 00000000 08:01 81794274 /usr/lib/i386-linux-gnu/libroken.so.18.1.0
b692e000-b692f000 r--p 00013000 08:01 81794274 /usr/lib/i386-linux-gnu/libroken.so.18.1.0
b692f000-b6930000 rw-p 00014000 08:01 81794274 /usr/lib/i386-linux-gnu/libroken.so.18.1.0
b6930000-b6961000 r-xp 00000000 08:01 81794056 /usr/lib/i386-linux-gnu/libhcrypto.so.4.1.0
b6961000-b6962000 r--p 00031000 08:01 81794056 /usr/lib/i386-linux-gnu/libhcrypto.so.4.1.0
b6962000-b6963000 rw-p 00032000 08:01 81794056 /usr/lib/i386-linux-gnu/libhcrypto.so.4.1.0
b6963000-b6964000 rw-p 00000000 00:00 0
b6964000-b6a00000 r-xp 00000000 08:01 81793735 /usr/lib/i386-linux-gnu/libasn1.so.8.0.0
b6a00000-b6a01000 r--p 0009c000 08:01 81793735 /usr/lib/i386-linux-gnu/libasn1.so.8.0.0
b6a01000-b6a03000 rw-p 0009d000 08:01 81793735 /usr/lib/i386-linux-gnu/libasn1.so.8.0.0
b6a03000-b6a80000 r-xp 00000000 08:01 81794113 /usr/lib/i386-linux-gnu/libkrb5.so.26.0.0
b6a80000-b6a82000 r--p 0007c000 08:01 81794113 /usr/lib/i386-linux-gnu/libkrb5.so.26.0.0
b6a82000-b6a84000 rw-p 0007e000 08:01 81794113 /usr/lib/i386-linux-gnu/libkrb5.so.26.0.0
b6a84000-b6a8a000 r-xp 00000000 08:01 81794060 /usr/lib/i386-linux-gnu/libheimntlm.so.0.1.0
b6a8a000-b6a8b000 r--p 00005000 08:01 81794060 /usr/lib/i386-linux-gnu/libheimntlm.so.0.1.0
b6a8b000-b6a8c000 rw-p 00006000 08:01 81794060 /usr/lib/i386-linux-gnu/libheimntlm.so.0.1.0
b6a8c000-b6b0d000 r-xp 00000000 08:01 67896017 /lib/i386-linux-gnu/libgcrypt.so.11.7.0
b6b0d000-b6b0e000 r--p 00080000 08:01 67896017 /lib/i386-linux-gnu/libgcrypt.so.11.7.0
b6b0e000-b6b10000 rw-p 00081000 08:01 67896017 /lib/i386-linux-gnu/libgcrypt.so.11.7.0
b6b10000-b6bcf000 r-xp 00000000 08:01 81795453 /usr/lib/i386-linux-gnu/libgnutls.so.26.21.8
b6bcf000-b6bd3000 r--p 000be000 08:01 81795453 /usr/lib/i386-linux-gnu/libgnutls.so.26.21.8
b6bd3000-b6bd4000 rw-p 000c2000 08:01 81795453 /usr/lib/i386-linux-gnu/libgnutls.so.26.21.8
b6bd4000-b6c0d000 r-xp 00000000 08:01 81793990 /usr/lib/i386-linux-gnu/libgssapi.so.3.0.0
b6c0d000-b6c0e000 r--p 00038000 08:01 81793990 /usr/lib/i386-linux-gnu/libgssapi.so.3.0.0
b6c0e000-b6c0f000 rw-p 00039000 08:01 81793990 /usr/lib/i386-linux-gnu/libgssapi.so.3.0.0
b6c0f000-b6c10000 rw-p 00000000 00:00 0
b6c10000-b6c29000 r-xp 00000000 08:01 81794291 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25
b6c29000-b6c2a000 r--p 00019000 08:01 81794291 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25
b6c2a000-b6c2b000 rw-p 0001a000 08:01 81794291 /usr/lib/i386-linux-gnu/libsasl2.so.2.0.25
b6c2b000-b6c38000 r-xp 00000000 08:01 81794119 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.8.3
b6c38000-b6c39000 r--p 0000c000 08:01 81794119 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.8.3
b6c39000-b6c3a000 rw-p 0000d000 08:01 81794119 /usr/lib/i386-linux-gnu/liblber-2.4.so.2.8.3
b6c3a000-b6c60000 r-xp 00000000 08:01 81794109 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1
b6c60000-b6c61000 r--p 00026000 08:01 81794109 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1
b6c61000-b6c62000 rw-p 00027000 08:01 81794109 /usr/lib/i386-linux-gnu/libk5crypto.so.3.1
b6c62000-b6cb0000 r-xp 00000000 08:01 81794126 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.8.3
b6cb0000-b6cb1000 r--p 0004e000 08:01 81794126 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.8.3
b6cb1000-b6cb2000 rw-p 0004f000 08:01 81794126 /usr/lib/i386-linux-gnu/libldap_r-2.4.so.2.8.3
b6cb2000-b6cb3000 rw-p 00000000 00:00 0
b6cb3000-b6cee000 r-xp 00000000 08:01 81793992 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2
b6cee000-b6cef000 r--p 0003a000 08:01 81793992 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2
b6cef000-b6cf0000 rw-p 0003b000 08:01 81793992 /usr/lib/i386-linux-gnu/libgssapi_krb5.so.2.2
b6cf0000-b6db7000 r-xp 00000000 08:01 81794115 /usr/lib/i386-linux-gnu/libkrb5.so.3.3
b6db7000-b6dbd000 r--p 000c6000 08:01 81794115 /usr/lib/i386-linux-gnu/libkrb5.so.3.3
b6dbd000-b6dbe000 rw-p 000cc000 08:01 81794115 /usr/lib/i386-linux-gnu/libkrb5.so.3.3
b6dbe000-b6de5000 r-xp 00000000 08:01 81794012 /usr/lib/libpq.so.5.4
b6de5000-b6de6000 ---p 00027000 08:01 81794012 /usr/lib/libpq.so.5.4
b6de6000-b6de7000 r--p 00027000 08:01 81794012 /usr/lib/libpq.so.5.4
b6de7000-b6de8000 rw-p 00028000 08:01 81794012 /usr/lib/libpq.so.5.4
b6ded000-b6def000 r-xp 00000000 08:01 67895335 /lib/libnss_mdns4_minimal.so.2
b6def000-b6df0000 r--p 00001000 08:01 67895335 /lib/libnss_mdns4_minimal.so.2
b6df0000-b6df1000 rw-p 00002000 08:01 67895335 /lib/libnss_mdns4_minimal.so.2
b6df1000-b6df8000 r-xp 00000000 08:01 82320416 /usr/local/lib/python2.7/dist-packages/ssdeep.so
b6df8000-b6df9000 r--p 00006000 08:01 82320416 /usr/local/lib/python2.7/dist-packages/ssdeep.so
b6df9000-b6dfa000 rw-p 00007000 08:01 82320416 /usr/local/lib/python2.7/dist-packages/ssdeep.so
b6dfa000-b6e1e000 r-xp 00000000 08:01 86379651 /usr/local/lib/python2.7/dist-packages/psycopg2/_psycopg.so
b6e1e000-b6e1f000 r--p 00023000 08:01 86379651 /usr/local/lib/python2.7/dist-packages/psycopg2/_psycopg.so
b6e1f000-b6e22000 rw-p 00024000 08:01 86379651 /usr/local/lib/python2.7/dist-packages/psycopg2/_psycopg.so
b6e22000-b6e3f000 r-xp 00000000 08:01 82055399 /usr/lib/python2.7/lib-dynload/_io.so
b6e3f000-b6e40000 r--p 0001c000 08:01 82055399 /usr/lib/python2.7/lib-dynload/_io.so
b6e40000-b6e48000 rw-p 0001d000 08:01 82055399 /usr/lib/python2.7/lib-dynload/_io.so
b6e48000-b6e8a000 rw-p 00000000 00:00 0
b6e8a000-b6e91000 r-xp 00000000 08:01 67896051 /lib/i386-linux-gnu/libnss_compat-2.15.so
b6e91000-b6e92000 r--p 00006000 08:01 67896051 /lib/i386-linux-gnu/libnss_compat-2.15.so
b6e92000-b6e93000 rw-p 00007000 08:01 67896051 /lib/i386-linux-gnu/libnss_compat-2.15.so
b6e93000-b6ea6000 r-xp 00000000 08:01 67896092 /lib/i386-linux-gnu/libresolv-2.15.so
b6ea6000-b6ea7000 r--p 00013000 08:01 67896092 /lib/i386-linux-gnu/libresolv-2.15.so
b6ea7000-b6ea8000 rw-p 00014000 08:01 67896092 /lib/i386-linux-gnu/libresolv-2.15.so
b6ea8000-b6eaa000 rw-p 00000000 00:00 0
b6eaa000-b6eb2000 r-xp 00000000 08:01 82054324 /usr/lib/python2.7/dist-packages/simplejson/_speedups.so
b6eb2000-b6eb3000 r--p 00008000 08:01 82054324 /usr/lib/python2.7/dist-packages/simplejson/_speedups.so
b6eb3000-b6eb4000 rw-p 00009000 08:01 82054324 /usr/lib/python2.7/dist-packages/simplejson/_speedups.so
b6eb4000-b6ec7000 r-xp 00000000 08:01 82055411 /usr/lib/python2.7/lib-dynload/datetime.so
b6ec7000-b6ec8000 r--p 00012000 08:01 82055411 /usr/lib/python2.7/lib-dynload/datetime.so
b6ec8000-b6ecb000 rw-p 00013000 08:01 82055411 /usr/lib/python2.7/lib-dynload/datetime.so
b6ecb000-b6fae000 rw-p 00000000 00:00 0
b6fae000-b6ffd000 r-xp 00000000 08:01 67896101 /lib/i386-linux-gnu/libssl.so.1.0.0
b6ffd000-b6ffe000 ---p 0004f000 08:01 67896101 /lib/i386-linux-gnu/libssl.so.1.0.0
b6ffe000-b7000000 r--p 0004f000 08:01 67896101 /lib/i386-linux-gnu/libssl.so.1.0.0
b7000000-b7004000 rw-p 00051000 08:01 67896101 /lib/i386-linux-gnu/libssl.so.1.0.0
b7004000-b7045000 rw-p 00000000 00:00 0
b7045000-b71d7000 r-xp 00000000 08:01 67896000 /lib/i386-linux-gnu/libcrypto.so.1.0.0
b71d7000-b71d8000 ---p 00192000 08:01 67896000 /lib/i386-linux-gnu/libcrypto.so.1.0.0
b71d8000-b71e7000 r--p 00192000 08:01 67896000 /lib/i386-linux-gnu/libcrypto.so.1.0.0
b71e7000-b71ee000 rw-p 001a1000 08:01 67896000 /lib/i386-linux-gnu/libcrypto.so.1.0.0
b71ee000-b71f3000 rw-p 00000000 00:00 0
b71f3000-b71f7000 r-xp 00000000 08:01 67896119 /lib/i386-linux-gnu/libuuid.so.1.3.0
b71f7000-b71f8000 r--p 00003000 08:01 67896119 /lib/i386-linux-gnu/libuuid.so.1.3.0
b71f8000-b71f9000 rw-p 00004000 08:01 67896119 /lib/i386-linux-gnu/libuuid.so.1.3.0
b71f9000-b71fa000 rwxp 00000000 00:00 0
b71fa000-b7201000 r-xp 00000000 08:01 81794117 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1
b7201000-b7202000 r--p 00006000 08:01 81794117 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1
b7202000-b7203000 rw-p 00007000 08:01 81794117 /usr/lib/i386-linux-gnu/libkrb5support.so.0.1
b7203000-b7206000 r-xp 00000000 08:01 82055397 /usr/lib/python2.7/lib-dynload/_hashlib.so
b7206000-b7207000 r--p 00002000 08:01 82055397 /usr/lib/python2.7/lib-dynload/_hashlib.so
b7207000-b7208000 rw-p 00003000 08:01 82055397 /usr/lib/python2.7/lib-dynload/_hashlib.so
b7208000-b7209000 r--p 005e0000 08:01 81796337 /usr/lib/locale/locale-archive
b7209000-b7409000 r--p 00000000 08:01 81796337 /usr/lib/locale/locale-archive
b7409000-b74ce000 rw-p 00000000 00:00 0
b74ce000-b74f8000 r-xp 00000000 08:01 67896032 /lib/i386-linux-gnu/libm-2.15.so
b74f8000-b74f9000 r--p 00029000 08:01 67896032 /lib/i386-linux-gnu/libm-2.15.so
b74f9000-b74fa000 rw-p 0002a000 08:01 67896032 /lib/i386-linux-gnu/libm-2.15.so
b74fa000-b7516000 r-xp 00000000 08:01 67896015 /lib/i386-linux-gnu/libgcc_s.so.1
b7516000-b7517000 r--p 0001b000 08:01 67896015 /lib/i386-linux-gnu/libgcc_s.so.1
b7517000-b7518000 rw-p 0001c000 08:01 67896015 /lib/i386-linux-gnu/libgcc_s.so.1
b7518000-b76bb000 r-xp 00000000 08:01 67895990 /lib/i386-linux-gnu/libc-2.15.so
b76bb000-b76bc000 ---p 001a3000 08:01 67895990 /lib/i386-linux-gnu/libc-2.15.so
b76bc000-b76be000 r--p 001a3000 08:01 67895990 /lib/i386-linux-gnu/libc-2.15.so
b76be000-b76bf000 rw-p 001a5000 08:01 67895990 /lib/i386-linux-gnu/libc-2.15.so
b76bf000-b76c2000 rw-p 00000000 00:00 0
b76c2000-b76d9000 r-xp 00000000 08:01 67896124 /lib/i386-linux-gnu/libz.so.1.2.7
b76d9000-b76da000 r--p 00016000 08:01 67896124 /lib/i386-linux-gnu/libz.so.1.2.7
b76da000-b76db000 rw-p 00017000 08:01 67896124 /lib/i386-linux-gnu/libz.so.1.2.7
b76db000-b76dd000 r-xp 00000000 08:01 67896116 /lib/i386-linux-gnu/libutil-2.15.so
b76dd000-b76de000 r--p 00001000 08:01 67896116 /lib/i386-linux-gnu/libutil-2.15.so
b76de000-b76df000 rw-p 00002000 08:01 67896116 /lib/i386-linux-gnu/libutil-2.15.so
b76df000-b76e0000 rw-p 00000000 00:00 0
b76e0000-b76e3000 r-xp 00000000 08:01 67896005 /lib/i386-linux-gnu/libdl-2.15.so
b76e3000-b76e4000 r--p 00002000 08:01 67896005 /lib/i386-linux-gnu/libdl-2.15.so
b76e4000-b76e5000 rw-p 00003000 08:01 67896005 /lib/i386-linux-gnu/libdl-2.15.so
b76e5000-b76fc000 r-xp 00000000 08:01 67896086 /lib/i386-linux-gnu/libpthread-2.15.so
b76fc000-b76fd000 r--p 00016000 08:01 67896086 /lib/i386-linux-gnu/libpthread-2.15.so
b76fd000-b76fe000 rw-p 00017000 08:01 67896086 /lib/i386-linux-gnu/libpthread-2.15.so
b76fe000-b7700000 rw-p 00000000 00:00 0
b7700000-b7702000 r-xp 00000000 08:01 67896028 /lib/i386-linux-gnu/libkeyutils.so.1.4
b7702000-b7703000 r--p 00002000 08:01 67896028 /lib/i386-linux-gnu/libkeyutils.so.1.4
b7703000-b7704000 rw-p 00003000 08:01 67896028 /lib/i386-linux-gnu/libkeyutils.so.1.4
b7704000-b7707000 r-xp 00000000 08:01 67895997 /lib/i386-linux-gnu/libcom_err.so.2.1
b7707000-b7708000 r--p 00002000 08:01 67895997 /lib/i386-linux-gnu/libcom_err.so.2.1
b7708000-b7709000 rw-p 00003000 08:01 67895997 /lib/i386-linux-gnu/libcom_err.so.2.1
b7709000-b7710000 r-xp 00000000 08:01 82055405 /usr/lib/python2.7/lib-dynload/_ssl.so
b7710000-b7711000 r--p 00006000 08:01 82055405 /usr/lib/python2.7/lib-dynload/_ssl.so
b7711000-b7712000 rw-p 00007000 08:01 82055405 /usr/lib/python2.7/lib-dynload/_ssl.so
b7712000-b7714000 rw-p 00000000 00:00 0
b7714000-b7715000 r-xp 00000000 00:00 0 [vdso]
b7715000-b7735000 r-xp 00000000 08:01 67895968 /lib/i386-linux-gnu/ld-2.15.so
b7735000-b7736000 r--p 0001f000 08:01 67895968 /lib/i386-linux-gnu/ld-2.15.so
b7736000-b7737000 rw-p 00020000 08:01 67895968 /lib/i386-linux-gnu/ld-2.15.so
bff9a000-bffbb000 rw-p 00000000 00:00 0 [stack]

[ryancassar]

I was experiencing similar issues in a multithreaded environment, I found giving each thread it's own instance of the Magic class fixed it. Something like this:

import collections
import magic
import threading

class magicdict(dict):
    def __missing__(self, flags):
        self[flags] = value = magic.Magic(mime=flags & magic.MAGIC_MIME, mime_encoding=flags & magic.MAGIC_MIME_ENCODING)

        return value

def lets_see_some_magic(flags=magic.MAGIC_NONE, wand_cache=collections.defaultdict(magicdict)):
    return wand_cache[threading.current_thread()][flags]

I'm not convinced this should be handled inside python-magic, but maybe documented as a known work around, or an optional activation, magic.attempt_thread_safety() or so.

[ahupp]

Oh wow. I spent some time digging through libmagic looking for the magic_close issue and couldn't find the issue but @thesteed's stack is really helpful. Fix coming.

[liucougar]

i see a fix is landed. could you consider releasing a new version to pypi?

[ahupp]

For some reason I was under the impression the GIL was held while in extension calls by default, but turns out CDLL does not hold the GIL (PyDLL does). This was a huge oversight on my part, my apologies.

I will push a new version shortly, I'd like to triage all the new issues first.