Dialog deadlock in AbstractTextEditor

[basilevs]

My Eclipse-based application occasionally hangs (does not respond to UI interactions) on MacOS in the following trace:

"main" #3 [259] prio=6 os_prio=31 cpu=22806.79ms elapsed=466.19s tid=0x000000012603a000 nid=259 runnable  [0x000000016ef63000]
   java.lang.Thread.State: RUNNABLE
	at org.eclipse.swt.internal.cocoa.OS.objc_msgSend_bool(Native Method)
	at org.eclipse.swt.internal.cocoa.NSRunLoop.runMode(NSRunLoop.java:40)
	at org.eclipse.swt.widgets.Display.sleep(Display.java:5223)
	at org.eclipse.jface.window.Window.runEventLoop(Window.java:825)
	at org.eclipse.jface.window.Window.open(Window.java:804)
	at org.eclipse.ui.texteditor.AbstractTextEditor.handleEditorInputChanged(AbstractTextEditor.java:4995)
	at org.eclipse.ui.texteditor.StatusTextEditor.handleEditorInputChanged(StatusTextEditor.java:290)
	at org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.handleEditorInputChanged(AbstractDecoratedTextEditor.java:1611)
	at org.eclipse.ui.texteditor.AbstractTextEditor.sanityCheckState(AbstractTextEditor.java:5148)
	at org.eclipse.ui.texteditor.StatusTextEditor.sanityCheckState(StatusTextEditor.java:276)
	at org.eclipse.ui.texteditor.AbstractTextEditor.safelySanityCheckState(AbstractTextEditor.java:5125)
	at org.eclipse.ui.texteditor.AbstractTextEditor$ActivationListener.handleActivation(AbstractTextEditor.java:1042)
	at org.eclipse.ui.texteditor.AbstractTextEditor$ActivationListener.partActivated(AbstractTextEditor.java:1003)
	at org.eclipse.ui.internal.PartService$1.run(PartService.java:85)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
	at org.eclipse.ui.internal.PartService.partActivated(PartService.java:82)
	at org.eclipse.ui.internal.WorkbenchWindow$WWinPartService.partActivated(WorkbenchWindow.java:3167)
	at org.eclipse.ui.internal.WorkbenchPage$2.run(WorkbenchPage.java:4889)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
	at org.eclipse.ui.internal.WorkbenchPage.firePartActivated(WorkbenchPage.java:4886)
	at org.eclipse.ui.internal.WorkbenchPage$E4PartListener.partActivated(WorkbenchPage.java:216)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl$2.run(PartServiceImpl.java:250)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl.firePartActivated(PartServiceImpl.java:247)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl.activate(PartServiceImpl.java:789)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl.activate(PartServiceImpl.java:695)
	at org.eclipse.e4.ui.internal.workbench.swt.AbstractPartRenderer.activate(AbstractPartRenderer.java:95)
	at org.eclipse.e4.ui.workbench.renderers.swt.ContributedPartRenderer.lambda$0(ContributedPartRenderer.java:63)
	at org.eclipse.e4.ui.workbench.renderers.swt.ContributedPartRenderer$$Lambda/0x00000ff8015f5680.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4660)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1657)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1680)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1665)
	at org.eclipse.swt.widgets.Shell.setActiveControl(Shell.java:1626)
	at org.eclipse.swt.widgets.Shell.setActiveControl(Shell.java:1589)
	at org.eclipse.swt.widgets.Control.sendFocusEvent(Control.java:3419)
	at org.eclipse.swt.widgets.Canvas.sendFocusEvent(Canvas.java:80)
	at org.eclipse.swt.widgets.Display.checkFocus(Display.java:750)
	at org.eclipse.swt.widgets.Shell.becomeKeyWindow(Shell.java:526)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:6132)
	at org.eclipse.swt.internal.cocoa.OS.objc_msgSend(Native Method)
	at org.eclipse.swt.internal.cocoa.NSWindow.close(NSWindow.java:57)
	at org.eclipse.swt.widgets.Shell.destroyWidget(Shell.java:811)
	at org.eclipse.swt.widgets.Widget.release(Widget.java:1442)
	at org.eclipse.swt.widgets.Control.release(Control.java:2943)
	at org.eclipse.swt.widgets.Widget.dispose(Widget.java:731)
	at org.eclipse.jface.window.Window.close(Window.java:335)
	at org.eclipse.jface.dialogs.Dialog.close(Dialog.java:983)
	at org.eclipse.jface.dialogs.ProgressMonitorDialog.close(ProgressMonitorDialog.java:349)
	at org.eclipse.jface.dialogs.ProgressMonitorDialog.finishedRun(ProgressMonitorDialog.java:558)
	at org.eclipse.jface.dialogs.ProgressMonitorDialog.run(ProgressMonitorDialog.java:475)
	at org.eclipse.rcptt.ui.launching.LaunchUtils.doLaunch(LaunchUtils.java:378)
	at org.eclipse.rcptt.ui.launching.LaunchUtils$3.run(LaunchUtils.java:367)
	at org.eclipse.rcptt.ui.launching.LaunchUtils.runInUI(LaunchUtils.java:417)
	at org.eclipse.rcptt.ui.launching.LaunchUtils.launch(LaunchUtils.java:365)
	at org.eclipse.rcptt.ui.launching.aut.BasicAUTComposite.launch(BasicAUTComposite.java:174)
	at org.eclipse.rcptt.ui.launching.aut.AUTViewComposite$1.run(AUTViewComposite.java:48)
	at org.eclipse.jface.action.Action.runWithEvent(Action.java:474)
	at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:581)
	at org.eclipse.jface.action.ActionContributionItem.lambda$5(ActionContributionItem.java:454)
	at org.eclipse.jface.action.ActionContributionItem$$Lambda/0x00000ff8015be508.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:91)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4660)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1657)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1680)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1665)
	at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1394)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4427)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4003)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1147)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:339)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1038)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:153)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:677)
	at org.eclipse.ui.internal.Workbench$$Lambda/0x00000ff80128c820.run(Unknown Source)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:339)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:583)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:173)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:185)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:219)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:149)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:115)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:467)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:298)
	at java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.base@25/DirectMethodHandle$Holder)
	at java.lang.invoke.LambdaForm$MH/0x00000ff801050000.invoke(java.base@25/LambdaForm$MH)
	at java.lang.invoke.LambdaForm$MH/0x00000ff801050400.invokeExact_MT(java.base@25/LambdaForm$MH)
	at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.base@25/DirectMethodHandleAccessor.java:156)
	at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.base@25/DirectMethodHandleAccessor.java:104)
	at java.lang.reflect.Method.invoke(java.base@25/Method.java:565)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:615)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:563)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1415)

Environment:
MacOS 12.7.6 (probably Windows 10 too)
Eclipse 2025-12 (probably since Kepler)

This seems to happen when an editor for a changed external file is activated when a dialog is being closed.
Can SWT for Cocoa handle nested native calls?

[iloveeclipse]

Which Eclipse version is used? Is is a recent regression?

[basilevs]

Which Eclipse version is used? Is is a recent regression?

This is on 2025-12, but I vaguely remember observing this years ago (probably on Windows)
I think this was reported on Bugzilla but it unresponsive these days, so hard to confirm.

[fedejeanne]

This seems to happen when an editor for a changed external file is activated when a dialog is being closed.
Can SWT for Cocoa handle nested native calls?

So the steps to reproduce are:

• Open 2 editors: A and B
• Select A
• Edit B outside of Eclipse
• Close A

?

[basilevs]

This seems to happen when an editor for a changed external file is activated when a dialog is being closed.
Can SWT for Cocoa handle nested native calls?

So the steps to reproduce are:

* Open 2 editors: `A` and `B`

* Select `A`

* Edit `B` outside of Eclipse

* Close `A`

?

No, there has to be a dialog opened, while file is being modified, so another editor will not do. Also, editor should be activated upon dialog closure.

[tomaswolf]

AbstractTExtEditor.handleEditorInputChanged uses getSite().getShell() as parent shell for a modal dialog. See

eclipse.platform.ui/bundles/org.eclipse.ui.workbench.texteditor/src/org/eclipse/ui/texteditor/AbstractTextEditor.java

Line 4938 in c6e9f0f

Shell shell = getSite().getShell();

If there is already a modal dialog open this may result in two event loops running, which typically produces such a UI deadlock.

For modal dialogs, use PlatformUI.getWorkbench().getModalDialogShellProvider().getShell() as parent shell of the dialog.

[tomaswolf]

@basilevs : as a work-around, close your dialog first, then activate the editor. Might be as simple as running the editor activation in a Display.asyncExec() or in a UIJob, but whether that might be good enough depends on what your application does exactly.

[basilevs]

@tomaswolf thanks, I'll to use the Shell provider.

as a work-around, close your dialog first

The Dialog is already being closed, I can't close it again. The two involved components (editor and dialog) have no knowledge of each other and can't coordinate.
Editor activation is out my control, it happens due to focus switch on dialog closing. The platform would be hard to refactor for asynchronous sanity check, so it can't help much too.

[tomaswolf]

All modal dialogs should use the modal IShellProvider if there is any chance of another modal dialog being open. So I doubt just changing it in your dialog is going to help; AbstractTextEditor should also be changed.

[basilevs]

All modal dialogs should use the modal IShellProvider if there is any chance of another modal dialog being open. So I doubt just changing it in your dialog is going to help; AbstractTextEditor should also be changed.

This expectation is very limiting. Should this be handled in SWT instead? How and why would clients know whether there are other dialogs?