mmap trap

[SilvanScherrer]

https://github.com/bitwiseworks/libunistring-os2/blob/master/tests/test-memchr.c
generates a nice trap when mmap is enabled.

______________________________________________________________________

 Exception Report - created 2019/11/11 17:18:05
______________________________________________________________________

 OS2/eCS Version:  2.45
 # of Processors:  2
 Physical Memory:  3058 mb
 Virt Addr Limit:  1536 mb
 Exceptq Version:  7.11.3-shl (Jul  5 2016)

______________________________________________________________________

 Exception C0000005 - Access Violation
______________________________________________________________________

 Process:  E:\TREES\LIBUNISTRING\BUILD\TESTS\TEST-MEMCHR.EXE (11/11/2019 16:56:26 8,394)
 PID:      E8EF (59631)
 TID:      01 (1)
 Priority: 200

 Filename: E:\TREES\LIBUNISTRING\BUILD\TESTS\TEST-MEMCHR.EXE (11/11/2019 16:56:26 8,394)
 Address:  005B:00010603 (0001:00000603)
 Cause:    Attempted to write to 21032FFF
           (uncommitted memory allocated by LIBCX0)

______________________________________________________________________

 Failing Instruction
______________________________________________________________________

 000105F9  MOV       EDX, ECX               (89ca)
 000105FB  MOV       EDI, ESI               (89f7)
 000105FD  MOV       [ESP+0x20], ECX        (894c24 20)
 00010601  MOV       AL, 0x58               (b0 58)
 00010603 >REP STOSB                        (f3 aa)
 00010605  MOV       [ESP+0x8], EDX         (895424 08)
 00010609  MOV       DWORD [ESP+0x4], 0x55  (c74424 04 55000000)
 00010611  MOV       [ESP], ESI             (893424)

______________________________________________________________________

 Registers
______________________________________________________________________

 EAX : 00000158   EBX  : 00000020   ECX : 00000001   EDX  : 00000001
 ESI : 21032FFF   EDI  : 21032FFF
 ESP : 0012FF00   EBP  : 0012FF48   EIP : 00010603   EFLG : 00010206
 CS  : 005B       CSLIM: FFFFFFFF   SS  : 0053       SSLIM: FFFFFFFF

 EAX : not a valid address
 EBX : not a valid address
 ECX : not a valid address
 EDX : not a valid address
 ESI : uncommitted memory allocated by LIBCX0
 EDI : uncommitted memory allocated by LIBCX0

______________________________________________________________________

 Stack Info for Thread 01
______________________________________________________________________

   Size       Base        ESP         Max         Top
 00100000   00130000 -> 0012FF00 -> 0012D000 -> 00030000

______________________________________________________________________

 Call Stack
______________________________________________________________________

   EBP     Address    Module     Obj:Offset    Nearest Public Symbol
 --------  ---------  --------  -------------  -----------------------
 Trap  ->  00010603   TEST-MEM  0001:00000603  test-memchr.c#106 main + 277 0001:0000038C (E:\Trees\libunistring\git\tests\test-memchr.c)

 0012FF48  00010047   TEST-MEM  0001:00000047  crt0.s#90 __text + 47 0001:00000000 (D:\Temp\ccu3YHGD.s)

 0012FF84  1F25B421   LIBCX0    0001:0000B421   ___init_app + 11 0001:0000B410 (main.obj)

 0012FFE0  1F45947B   LIBCN0    0001:0003947B 

the whole trap is available of course :)

[SilvanScherrer]

below find the debug output of libcx log:

Opened log at 2019-11-12 15:29:38.42
Process ID: 0xdf36 (57142) Parent PID: 0x3b (59) Type: 2
Exe hmte  : 0x17a8 (E:\TREES\LIBUNISTRING\BUILD\TESTS\TEST-MEMCHR.EXE)
First arg : test-memchr
Second arg: 
Cur dir   : E:\trees\libunistring\build\tests
CRT Module: LIBCN0 hmod=0x8da (C:\USR\LIB\LIBCN0.DLL)
__libc_logInit: addr 0x1f4650ed iObj=0 offObj=0x450ed
   Millsecond Timestamp.
   |     Thread ID.
   |     |  Call Nesting Level.
   |     |  |   Log Group.
   |     |  |   |    Message Type.
   |     |  |   |    |    errno in hex (0xface if not available).
   |     |  |   |    |    |      Function Name.
   |     |  |   |    |    |      |       Millisconds In function (Optional).
   v     v  v   v    v    v      v       v
xxxxxxxx tt nn gggg dddd eeee function [(ms)]: message
LIBCx version : 0.6.6_debug
LIBCx module  : E:\TREES\LIBUNISTRING\BUILD\LIB\.LIBS\LIBCX0.DLL (hmod=16bd)
0695c6c1 01 shared.c:491:_DLL_InitTerm: hModule 16bd, ulFlag 0
0695c6c1 01 shared.c:179:shared_init: DosOpenMutexSem = 187
0695c6c1 01 shared.c:229:shared_init: DosCreateMutexSem = 0
0695c6c2 01 shared.c:249:shared_init: DosAllocSharedMem(OBJ_ANY) = 0
0695c6c2 01 shared.c:261:shared_init: gpData 0x5d170000
0695c6c2 01 shared.c:273:shared_init: gpData->heap = 0x5d17003c
0695c6c4 01 shared.c:283:shared_init: gpData->procs = 0x5d1701c0
0695c6c4 01 shared.c:287:shared_init: gpData->files = 0x5d170220
0695c6c4 01 shared.c:327:shared_init: done
0695c6c6 01 mmap.c:249:mmap: addr 0, len 8192, prot 3=RW-, flags 1002=-P-A, fildes -1, off 0
0695c6c6 01 mmap.c:71:DosMyAllocMem: DosAllocMem(OBJ_ANY) = 0
0695c6c6 01 mmap.c:1029:find_mmap: mapping not found
0695c6c6 01 mmap.c:984:mmap: mmap 0x5d1743a8 (21030000..21032000 (8192))
0695c6c8 01 mmap.c:2154:mprotect: addr 0x21031000, len 4096, prot 0
0695c6c8 01 mmap.c:1039:find_mmap: found m 0x5d1743a8 (21030000..21032000 (8192), flags 1002=-P-A, dos_flags 3, refcnt 0, fmem 0 (0))
0695c6c8 01 mmap.c:2121:protect_map: m 0x5d1743a8, adjusted addr 21031000, len 4096, dos_flags ffffffff
0695c6c8 01 mmap.c:2260:mprotect: _std_mprotect = 0 (Error 0)
0695c6c8 01 mmap.c:2121:protect_map: m 0x5d1743a8, adjusted addr 21031000, len 4096, dos_flags 0
0695c6c8 01 mmap.c:2137:protect_map: changing dos_flags from 3 to 0
0695c6cc 01 mmap.c:249:mmap: addr 0, len 8192, prot 3=RW-, flags 1002=-P-A, fildes -1, off 0
0695c6cc 01 mmap.c:71:DosMyAllocMem: DosAllocMem(OBJ_ANY) = 0
0695c6cc 01 mmap.c:1029:find_mmap: mapping not found
0695c6cc 01 mmap.c:984:mmap: mmap 0x5d1743c4 (21032000..21034000 (8192))
0695c6cc 01 mmap.c:2154:mprotect: addr 0x21033000, len 4096, prot 0
0695c6cc 01 mmap.c:1039:find_mmap: found m 0x5d1743c4 (21032000..21034000 (8192), flags 1002=-P-A, dos_flags 3, refcnt 0, fmem 0 (0))
0695c6ce 01 mmap.c:2121:protect_map: m 0x5d1743c4, adjusted addr 21033000, len 4096, dos_flags ffffffff
0695c6ce 01 mmap.c:2260:mprotect: _std_mprotect = 0 (Error 0)
0695c6ce 01 mmap.c:2121:protect_map: m 0x5d1743c4, adjusted addr 21033000, len 4096, dos_flags 0
0695c6ce 01 mmap.c:2137:protect_map: changing dos_flags from 3 to 0
0695c6ce 01 mmap.c:1651:mmap_exception: XCPT_ACCESS_VIOLATION [flags 0 nested 0 addr 0x10603]: addr 21032fff, info 2
0695c6d0 01 mmap.c:1039:find_mmap: found m 0x5d1743c4 (21032000..21034000 (8192), flags 1002=-P-A, dos_flags 0, refcnt 0, fmem 0 (0))
0695c6d0 01 mmap.c:1855:mmap_exception: not retrying
0695c838 01 shared.c:544:ProcessExit: reason 0
0695c838 01 shared.c:337:shared_term: gMutex 800100f4, gpData 0x5d170000 (heap 0x5d17003c, refcnt 1), gSeenAssertion 0
0695c838 01 fcntl.c:552:fcntl_locking_term: gpData->fcntl_locking->blocked 0
0695c838 01 fcntl.c:575:fcntl_locking_term: DosCloseEventSem = 0
0695c839 01 mmap.c:1222:free_mmap: private mapping 0x5d1743a8 (21030000..21032000)
0695c839 01 mmap.c:1222:free_mmap: private mapping 0x5d1743c4 (21032000..21034000)
0695c839 01 mmap.c:1566:mmap_term: DosCloseEventSem = 0
0695c839 01 shared.c:378:shared_term: proc 0x5d170440
0695c83b 01 shared.c:402:shared_term: proc->files 0
0695c83b 01 shared.c:424:shared_term: gpData->files 0x5d170220
0695c83b 01 shared.c:432:shared_term: gpData->procs 0x5d1701c0
0695c83b 01 shared.c:443:shared_term: 
LIBCx resource usage
--------------------
Reserved memory size:  2097152 bytes
Committed memory size: 65536 bytes
Heap size total:       65088 bytes
Heap size used now:    0 bytes
Heap size used max:    684 bytes
ProcDesc structs used now:       0
ProcDesc structs used max:       1
FileDesc structs used now:       0
FileDesc structs used max:       0
SharedFileDesc structs used now: 0
SharedFileDesc structs used max: 0
0695c83d 01 shared.c:459:shared_term: _udestroy = 0 (0)
0695c83d 01 shared.c:464:shared_term: DosFreeMem = 0
0695c83d 01 shared.c:477:shared_term: DosCloseMutexSem = 0
0695c83d 01 shared.c:491:_DLL_InitTerm: hModule 16bd, ulFlag 1

[dmik]

Ok, I know what's going on there. As a matter of fact, our mprotect implementation only supports cases where protection flags of the whole region previously allocated with mmap (or a number of whole regions) are changed. While partial change is not reported as an error by mprotect it actually causes protection flags of the whole region to be changed (from the mmap point of view).

The above test, however, relies on the fact that partial changing works as expected. They first allocate a mmap region of 2 pages, then set protection flags of the second page to PROT_NONE (so that accessing it should cause a SIGSEGV). Then they work with the first page (which is should be still read-writable). However, given that the partial mprotect causes the whole region to be set to PROT_NONE, a subsequent write to the first page causes an immediate (but unexpected) SIGSEGV initiated by the mmap code.

I need to think on how to work around this. Adding partial region support to mprotect might not be fast (and we don't have enough time to work on that now). As a fast solution, we may change the test case to use munmap instead of mprotect. Our munmap implementation is perfectly capable of spliting and joining mmap regions. In fact, mprotect(PROT_NONE) is very similar to munmap for the purpose of this test. I've already tried it locally and it works like a charm.

But our mprotect will remain lacking this functionality in this case. I wonder how important it is in real apps. We may postpone it until we find this out. And simply make mprotect return EINVAL or such when it detects a split until then.

[dmik]

There is no easy way to do this now. I will create a separate ticket and will simply disable partial mprotect for mmapped regions (making it return EINVAL as said above).

[dmik]

My bad, EINVAL in the above two comments should read EACCES.

[dmik]

Note that the above commit fixes the test case as well — mprotect and related memchr tests are simply not run in such a case.