AV when writting into internal trace log

[hvlad]

Reported by @pavel-zotov privately.

[hvlad]

Stack trace:

# Child-SP          RetAddr               Call Site

00 00000000`022dc2d8 00007ffe`d8b11b40     ntdll!NtWaitForMultipleObjects+0x14
01 00000000`022dc2e0 00007ffe`d8b11a3e     KERNELBASE!WaitForMultipleObjectsEx+0xf0
02 00000000`022dc5d0 00007ffe`d9a00eba     KERNELBASE!WaitForMultipleObjects+0xe
03 00000000`022dc610 00007ffe`d9a008f6     kernel32!WerpLaunchAeDebug+0x2cba
04 00000000`022dc730 00007ffe`d8bedf99     kernel32!WerpLaunchAeDebug+0x26f6
05 00000000`022dc770 00007ffe`db175570     KERNELBASE!UnhandledExceptionFilter+0x3d9
06 00000000`022dc890 00007ffe`db15c8c6     ntdll!memset+0x13b0
07 00000000`022dc8d0 00007ffe`db17247f     ntdll!_C_specific_handler+0x96
08 00000000`022dc940 00007ffe`db1214f4     ntdll!_chkstk+0x11f
09 00000000`022dc970 00007ffe`db170f8e     ntdll!RtlRaiseException+0x484
0a 00000000`022dd080 00007ffe`d26317e9     ntdll!KiUserExceptionDispatcher+0x2e
*** WARNING: Unable to verify checksum for engine13.dll
0b 00000000`022dd638 00007ffe`9d7d6c1b     VCRUNTIME140!memcpy+0x4d9
0c 00000000`022dd640 00007ffe`9d7d72a3     engine13!Jrd::TraceLog::extend(unsigned int size = <Value unavailable error>)+0xeb [C:\firebird-build\src\jrd\trace\TraceLog.cpp @ 244]
0d 00000000`022dd770 00007ffe`9d7ddbb6     engine13!Jrd::TraceLog::write(void * buf = <Value unavailable error>, unsigned int size = <Value unavailable error>)+0xb3 [C:\firebird-build\src\jrd\trace\TraceLog.cpp @ 165]
0e 00000000`022dd7b0 00007ffe`9d7dde17     engine13!Jrd::TraceLogWriterImpl::write(void * buf = <Value unavailable error>, unsigned int size = 0x1ff)+0x36 [C:\firebird-build\src\jrd\trace\TraceObjects.cpp @ 504]
0f 00000000`022dd9a0 00007ffe`9d7dc7a0     engine13!Jrd::TraceLogWriterImpl::write_s(class Firebird::CheckStatusWrapper * status = 0x00000000`022dda00, void * buf = 0x00000000`0dd4b670, unsigned int size = 0x22dd9d8)+0x17 [C:\firebird-build\src\jrd\trace\TraceObjects.cpp @ 534]
*** WARNING: Unable to verify checksum for fbtrace.dll
10 00000000`022dd9e0 00000000`00bc2034     engine13!Firebird::ITraceLogWriterBaseImpl<Jrd::TraceLogWriterImpl,Firebird::CheckStatusWrapper,Firebird::IReferenceCountedImpl<Jrd::TraceLogWriterImpl,Firebird::CheckStatusWrapper,Firebird::Inherit<Firebird::IVersionedImpl<Jrd::TraceLogWriterImpl,Firebird::CheckStatusWrapper,Firebird::Inherit<Firebird::ITraceLogWriter> > > > >::cloopwrite_sDispatcher(class Firebird::ITraceLogWriter * self = 0x00000000`0ca425c8, class Firebird::IStatus * status = 0x00000000`0dd31dc0, void * buf = 0x00000000`0dd4b670, unsigned int size = 0x1ff)+0x40 [C:\firebird-build\src\include\firebird\IdlFbInterfaces.h @ 18696]
11 (Inline Function) --------`--------     fbtrace!Firebird::ITraceLogWriter::write_s(void)+0xaa [C:\firebird-build\src\include\firebird\IdlFbInterfaces.h @ 6082]
12 00000000`022dda40 00000000`00bc2580     fbtrace!TracePluginImpl::logRecord(char * action = <Value unavailable error>)+0x224 [C:\firebird-build\src\utilities\ntrace\TracePluginImpl.cpp @ 282]

Values of some variables (frame 0c):

newSize	0xa00000
oldSize	0x800000
toMoveR	0x5f56c8
toMoveW	0x20a794

Note, toMoveW < toMoveR and toMoveW > newSize - oldSize thus memmove uses memory out of allocated space.

[hvlad]

Possible workaround for affected versions - set value of MaxUserTraceLogSize setting to the power of two, for example 8, 16, etc