Read-acquire Class Unload Mutex for compilation before checkpoint

[dsouzai]

The following deadlock is possible:

1. Comp thread read-enters class unload mutex
2. Checkpointing thread acquires comp monitor
3. Comp thread blocks trying to acquire comp monitor
4. Checkpointing thread blocks trying to write-enter class unload mutex

The reason the Checkpointing thread acquires the class unload mutex is to prevent class unloading while it collects methods. However, a read-acquire is sufficient for this task.

Therefore, having the Checkpointing thread do a read-acquire will prevent the deadlock since it will not block waiting to enter.

[dsouzai]

@mpirvu could you please review?

[dsouzai]

This will need to be backported to 0.38 since this issue was seen in Liberty testing.

[mpirvu]

LGTM.
In the future we may need to add some comments for these methods regarding the monitors they already hold when entered and the monitors they acquire internally.
Can the same effect of blocking GC be achieved by acquiring VM access?

[mpirvu]

jenkins test sanity xlinux jdk17

[mpirvu]

If the hook static void jitHookPrepareCheckpoint(J9HookInterface * * hookInterface, UDATA eventNum, void * eventData, void * userData) already holds VM access (to be verified) we don't need to acquire the classUnloadMutex later on.

[dsouzai]

If the hook static void jitHookPrepareCheckpoint(J9HookInterface * * hookInterface, UDATA eventNum, void * eventData, void * userData) already holds VM access (to be verified) we don't need to acquire the classUnloadMutex later on.

The checkpointing thread releases VM access because of #15202. However, at this point we're not actually waiting for comp threads so it may be that we can actually acquire and release VM Access at this point. I will have to look into the code paths to double check, but for 0.38 I think the change in this PR is the safest solution.

[pshipton]

This change doesn't compile everywhere. I spotted the following failure so far, there may be more later.

This is gcc 11.2
https://openj9-jenkins.osuosl.org/job/Build_JDK20_s390x_linux_Nightly/34/console

19:27:30  /home/jenkins/workspace/Build_JDK20_s390x_linux_Nightly/openj9/runtime/compiler/control/CompileBeforeCheckpoint.cpp: In member function 'void TR::CompileBeforeCheckpoint::collectAndCompileMethodsBeforeCheckpoint()':
19:27:30  /home/jenkins/workspace/Build_JDK20_s390x_linux_Nightly/openj9/runtime/compiler/control/CompileBeforeCheckpoint.cpp:116:71: error: empty parentheses were disambiguated as a function declaration [-Werror=vexing-parse]
19:27:30    116 |    TR::ClassUnloadMonitorCriticalSection collectMethodsCriticalSection();
19:27:30        |                                                                       ^~
19:27:30  /home/jenkins/workspace/Build_JDK20_s390x_linux_Nightly/openj9/runtime/compiler/control/CompileBeforeCheckpoint.cpp:116:71: note: remove parentheses to default-initialize a variable
19:27:30    116 |    TR::ClassUnloadMonitorCriticalSection collectMethodsCriticalSection();
19:27:30        |                                                                       ^~
19:27:30        |                                                                       --
19:27:30  /home/jenkins/workspace/Build_JDK20_s390x_linux_Nightly/openj9/runtime/compiler/control/CompileBeforeCheckpoint.cpp:116:71: note: or replace parentheses with braces to value-initialize a variable