trimming broken for some traces

[zmike]

Looking at the trace from #900, if I try to trim a gameplay trace (e.g., gltrim -k -f3470), the trim operation completes but then replay crashes immediately like

#0  0x0000000000584eed in retrace_memcpy (call=...) at /home/zmike/src/apitrace/retrace/retrace_stdc.cpp:64
#1  0x0000000000558242 in retrace::Retracer::retrace (this=0x83e0a0 <retracer>, call=...) at /home/zmike/src/apitrace/retrace/retrace.cpp:159
#2  0x000000000055be4c in retrace::retraceCall (call=0x13642b0) at /home/zmike/src/apitrace/retrace/retrace_main.cpp:406
#3  0x000000000055c0dd in retrace::RelayRunner::runLeg (this=0x127d7d0, call=0x13642b0) at /home/zmike/src/apitrace/retrace/retrace_main.cpp:562
#4  0x000000000055c047 in retrace::RelayRunner::runRace (this=0x127d7d0) at /home/zmike/src/apitrace/retrace/retrace_main.cpp:540
#5  0x000000000055c2be in retrace::RelayRunner::runnerThread (_this=0x127d7d0) at /home/zmike/src/apitrace/retrace/retrace_main.cpp:621
#6  0x0000000000584be4 in std::__invoke_impl<void, void (*)(retrace::RelayRunner*), retrace::RelayRunner*> (
    __f=@0xc67b00: 0x55c2a6 <retrace::RelayRunner::runnerThread(retrace::RelayRunner*)>) at /usr/include/c++/13/bits/invoke.h:61
#7  0x0000000000584a9a in std::__invoke<void (*)(retrace::RelayRunner*), retrace::RelayRunner*> (
    __fn=@0xc67b00: 0x55c2a6 <retrace::RelayRunner::runnerThread(retrace::RelayRunner*)>) at /usr/include/c++/13/bits/invoke.h:96
#8  0x000000000058496d in std::thread::_Invoker<std::tuple<void (*)(retrace::RelayRunner*), retrace::RelayRunner*> >::_M_invoke<0ul, 1ul> (this=0xc67af8)
    at /usr/include/c++/13/bits/std_thread.h:292
#9  0x00000000005848b6 in std::thread::_Invoker<std::tuple<void (*)(retrace::RelayRunner*), retrace::RelayRunner*> >::operator() (this=0xc67af8)
    at /usr/include/c++/13/bits/std_thread.h:299
#10 0x0000000000584844 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(retrace::RelayRunner*), retrace::RelayRunner*> > >::_M_run (
    this=0xc67af0) at /usr/include/c++/13/bits/std_thread.h:244
#11 0x00007ffff7ae31f3 in execute_native_thread_routine () from /lib64/libstdc++.so.6
#12 0x00007ffff78ae947 in start_thread () from /lib64/libc.so.6
#13 0x00007ffff7934870 in clone3 () from /lib64/libc.so.6

I've noticed this behavior is consistent in traces from a number of AAA titles.

[gerddie]

Unfortunately the trace is no longer available.

[zmike]

Oh whoops, forgot I replaced it with a working trace. Link

[gerddie]

I've found the root of the problem: gltrim is not yet capable to deal with multi-threaded traces, i.e. in this case a texture is set up in one thread, and other texture bindings happen in another thread, since the code ignores the thread, the texture setup calls are not attributed to the right objects, and we get an illegal call sequence. In this case calls to glCompressedTextureSubImage2DEXT happen on textures that are no longer correctly set up, because the setup calls to glCompressedTexImage2D that are needed for this setup must use the binding points that are clobbered by the trimmer.

[gerddie]

With this PR #902 trimming results in a trace that can be replayed.

[zmike]

Awesome!

[gerddie]

Well, the output is broken, I'm not yet sure whether it is the usual "one has to use try-and -error" to get it right, or whether it is a "normal" bug ( at first sight it looks like the latter)

[gerddie]

After some testing I was able to get correct rendering with

gltrim -k -s 3362-3372 -f 3470-3500 wolf.trace -o wolf-k+s3362-3372+f3470-3500-check.trace

i.e. one has to add some setup frames to get it right, so it is the "try-an-error" problem. (Frames 3362-3372 seem to setup the scene)