Main thread locking overagressively

[jonahgraham]

Extracted from discussion in #560

When opening large files, like the example C++ file in #560 I see the main thread spending most of its time locked on an object, here is the stack trace of the two key threads:

"main" #1 prio=6 os_prio=0 cpu=19432.94ms elapsed=82.09s tid=0x00007fd198028e90 nid=0x13a2e waiting for monitor entry  [0x00007fd1a00f4000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.eclipse.tm4e.core.model.AbstractModelLines.getOrNull(AbstractModelLines.java:147)
        - waiting to lock <0x0000000098580240> (a java.util.LinkedList)
        at org.eclipse.tm4e.core.model.TMModel.getLineTokens(TMModel.java:286)
        at org.eclipse.tm4e.ui.text.TMPresentationReconciler.colorize(TMPresentationReconciler.java:540)
        at org.eclipse.tm4e.ui.text.TMPresentationReconciler$InternalListener.colorize(TMPresentationReconciler.java:410)
        at org.eclipse.tm4e.ui.text.TMPresentationReconciler$InternalListener.lambda$0(TMPresentationReconciler.java:386)
        at org.eclipse.tm4e.ui.text.TMPresentationReconciler$InternalListener$$Lambda$1335/0x000000010096d970.run(Unknown Source)
        at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
        at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)

---

"org.eclipse.tm4e.ui.internal.model.TMDocumentModel" #103 daemon prio=1 os_prio=0 cpu=44439.56ms elapsed=47.43s tid=0x00007fd198795200 nid=0x13b48 runnable  [0x00007fd068fae000]
   java.lang.Thread.State: RUNNABLE
        at java.util.LinkedList.node(java.base@17.0.7/LinkedList.java:580)
        at java.util.LinkedList.get(java.base@17.0.7/LinkedList.java:481)
        at org.eclipse.tm4e.core.model.AbstractModelLines.getOrNull(AbstractModelLines.java:149)
        - locked <0x0000000098580240> (a java.util.LinkedList)
        at org.eclipse.tm4e.core.model.TMModel.getLineTokens(TMModel.java:286)
        at org.eclipse.tm4e.ui.internal.utils.MarkerUtils.updateTextMarkers(MarkerUtils.java:119)
        at org.eclipse.tm4e.ui.internal.utils.MarkerUtils.updateTextMarkers(MarkerUtils.java:66)
        at org.eclipse.tm4e.ui.text.TMPresentationReconciler$InternalListener.modelTokensChanged(TMPresentationReconciler.java:390)
        at org.eclipse.tm4e.core.model.TMModel.emit(TMModel.java:279)
        at org.eclipse.tm4e.core.model.TMModel.buildAndEmitEvent(TMModel.java:273)
        at org.eclipse.tm4e.core.model.TMModel$TokenizerThread.revalidateTokens(TMModel.java:121)
        at org.eclipse.tm4e.core.model.TMModel$TokenizerThread.run(TMModel.java:101)

This leaves the main thread somewhat unresponsive, but not totally hung as the lock must be getting released on a regular basis. A quick sample of the JVM with visualvm shows that most of the processing time is in the TMDocumentModel thread:

[sebthom]

Please try the latest TM4E snapshot if you see the same behaviour.

[jonahgraham]

Yes, this issue is resolved by #540. Thank you!