Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in Tree from Bug 575787 #287

Closed
jonahgraham opened this issue Aug 4, 2022 · 23 comments · Fixed by #318
Closed

Regression in Tree from Bug 575787 #287

jonahgraham opened this issue Aug 4, 2022 · 23 comments · Fixed by #318
Assignees
Labels
regression Something that used to work Windows Happens on Windows OS

Comments

@jonahgraham
Copy link
Contributor

Bug 575787 which optimized Tree.setItemCount causes Tree.indexOf(TreeItem) to return negative numbers.

This issue was originally reported as a bug with CDT in Bug 580518

To Reproduce

At the moment the steps to reproduce involve using CDT on Windows:

  1. Install CDT 10.7 in Platform 4.24, gcc+gdb will be needed too
  2. Create two Hello World projects
  3. In project A insert 5 breakpoints
  4. In project B insert 2 breakpoints
  5. Debug project B
  6. In the Breakpoints view toggle "Show Breakpoints Supported by Selected Target"

I'll try to find time to make a standalone test case. The above test case uses virtual trees + platform debug's async view model which makes it fairly complicated.

Expected behavior

When toggle is on, only 2 breakpoints of project B displayed. When toggle is off, all 7 breakpoints displayed.

Actual behaviour

When toggled back off, the breakpoints view is "corrupted" - see screenshot. There is also an exception in the CDT code due to the index not being in the expected range of >= -1. See bug 580518 for full stack trace.

Screenshots

image

Environment:

  1. Select the platform(s) on which the behavior is seen:
    • All OS
    • Windows
    • Linux
    • macOS

Definitely Windows, can't seem to reproduce on Linux.

  1. Additional OS info (e.g. OS version, Linux Desktop, etc)

  2. JRE/JDK version

JustJ Java 17 as shipped with EPP.

Version since

Since commit 17b4024 - reverting 17b4024 resolves the issue.

@jonahgraham
Copy link
Contributor Author

Instead of the steps in my original I can reproduce with Java + some other debug environment. By creating 5ish Java breakpoints and 5ish non-Java breakpoints and then toggling "Show Breakpoints Supported by Selected Target" while debugging the Java project will lead to the same problem, with the following exception:

java.lang.ArrayIndexOutOfBoundsException: arraycopy: source index -1 out of bounds for object array[8]
	at java.base/java.lang.System.arraycopy(Native Method)
	at org.eclipse.debug.internal.ui.model.elements.ElementContentProvider.getElements(ElementContentProvider.java:197)
	at org.eclipse.debug.internal.ui.model.elements.BreakpointManagerContentProvider.getChildren(BreakpointManagerContentProvider.java:994)
	at org.eclipse.debug.internal.ui.model.elements.ElementContentProvider.retrieveChildren(ElementContentProvider.java:99)
	at org.eclipse.debug.internal.ui.model.elements.ElementContentProvider$1.run(ElementContentProvider.java:58)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

and screenshot:

image

@jonahgraham
Copy link
Contributor Author

Instead of the steps in my original [...]

Explicit steps that don't need as much setup to experiment with:

  1. With eclipse-SDK-I20220804-1800-win32-x86_64 install org.eclipse.lsp4e and org.eclipse.lsp4e.debug
  2. Create a Java project
  3. Create a Java class with a main, and enough lines of code to insert 5 breakpoints
  4. Create another file (e.g. hello.js), open in Generic Editor and add enough lines of code to insert 5 breakpoints
  5. Debug main
  6. In Breakpoints view turn on and then off "Show Breakpoints Supported by Selected Target"
  7. The Breakpoints view will now be corrupted:

image

@niraj-modi
Copy link
Member

@SyntevoAlex Please have a look.

@niraj-modi niraj-modi added regression Something that used to work Windows Happens on Windows OS labels Aug 5, 2022
@SyntevoAlex
Copy link
Member

Will do, but it will probably have to wait a few days.

@Phillipus
Copy link
Contributor

Definitely Windows, can't seem to reproduce on Linux.

I was going to test on Mac but couldn't get past:

install org.eclipse.lsp4e and org.eclipse.lsp4e.debug

I tried adding the update site at https://download.eclipse.org/lsp4e/releases/latest/ to Eclipse but get "There are no items available"

@merks
Copy link
Contributor

merks commented Aug 5, 2022

@Phillipus
Copy link
Contributor

It's in every release train repository:

It's certainly there in the repo, but when I try to install from within Eclipse ("Help -> Install New Software") I can only see LSP4J SDK in the list of available software. Unless it's part of another package?

@jonahgraham
Copy link
Contributor Author

@Phillipus Perhaps you typed lsp4j instead of lsp4e? Anyway the mapping from ID to name is not always obvious/known - so here is more step-by-step:

  1. [...] install org.eclipse.lsp4e and org.eclipse.lsp4e.debug

In the install new software it is called "Language Server Protocol client for Eclipse IDE (incubation)" and "Debug Adapter Client for Eclipse IDE (incubation)"*:

image

Though you may prefer to do it explicitly at the command line with ./eclipse/eclipse -data data -consoleLog -nosplash -application org.eclipse.equinox.p2.director -repository https://download.eclipse.org/releases/2022-09 -installIU org.eclipse.lsp4e -installIU org.eclipse.lsp4e.debug

* Perhaps a feature request for "Install New Software" is to know/understand plug-in IDs as typing "org.eclipse.lsp4e" does not give useful results for installing LSP4E

@merks
Copy link
Contributor

merks commented Aug 5, 2022

@jonahgraham
Copy link
Contributor Author

I think this feature [...]

@merks comment refers to lsp4j - you need lsp4e

FWIW The lsp4e bundles are in p2 and can be installed in the gui or command line even though they are not features.

@Phillipus
Copy link
Contributor

@jonahgraham Thanks for the more detailed explanation.

I installed those plugins and am now able to set breakpoints on JS files as well as Java files and these are showing in the Breakpoints view. However, pressing "Show Breakpoints Supported by Selected Target" on and off does nothing. (Testing on Windows before I test on Mac.)

@jonahgraham
Copy link
Contributor Author

I installed those plugins and am now able to set breakpoints on JS files as well as Java files and these are showing in the Breakpoints view. However, pressing "Show Breakpoints Supported by Selected Target" on and off does nothing. (Testing on Windows before I test on Mac.)

Did you debug main? Are you stopped at a Java breakpoint:

  1. Debug main

image

@Phillipus
Copy link
Contributor

Did you debug main? Are you stopped at a Java breakpoint:

OK, thanks for that. Got it working now.

So I can reproduce this on Windows but not Mac. 👍 So, as you say, a Windows issue.

SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 8, 2022
…exOf to return wrong values

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 8, 2022
…exOf to return wrong values

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
@SyntevoAlex
Copy link
Member

Thanks, I was able to reproduce using your steps. I'm working on a patch.

@iloveeclipse
Copy link
Member

Thanks, I was able to reproduce using your steps. I'm working on a patch

@SyntevoAlex : do you think you will make it for 4.25 M3? M3 is planned to be this week.

@SyntevoAlex
Copy link
Member

SyntevoAlex commented Aug 15, 2022

I'm not completely sure. A few other tasks required my attention.

I already fixed one problem in the patch, but there is another problem. Seems to be a regression from some other recent patch. I need to investigate it.

@iloveeclipse
Copy link
Member

@SyntevoAlex : would it be OK to drop optimization then, since this causes a regression?

@jonahgraham : how severe the problem is? Should we revert 17b4024 for 4.25 M3?

@SyntevoAlex
Copy link
Member

I understand that it's been 8 months since the patch was merged and the problem is only noticed now. Sounds like it's not too much of a nuisance.

@jonahgraham
Copy link
Contributor Author

For the problematic use case in CDT it is bad as breakpoints view is busted and requires restarting IDE. However I don't know how many people use that feature vs benefit from optimization. I would lean on leaving it broken in 4.25 over removing the optimization as my understanding is reverting optimization would essentially be a big performance regression?

Is there a mitigation we can do in 4.25? Eg can breakpoints view behave differently?

@iloveeclipse
Copy link
Member

my understanding is reverting optimization would essentially be a big performance regression?

Not really. The Windows port worked "good enough" for years without this optimization.
Broken Breakpoints view and hanging UI is a severe recent regression.

However I don't know how many people use that feature

We also don't know how many people are affected by other things that do not work anymore, simply because they don't know where the problem is coming from.

Waiting 3 months longer for optimization (for performance issue that was there for years) is better as delivering second broken platform release.

@SyntevoAlex : could you please provide a clean revert of the code in question for M3?

@SyntevoAlex
Copy link
Member

I will try to fix tomorrow, and if it doesn't work, I'll compose a revert.

SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 16, 2022
…exOf to return wrong values

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 16, 2022
…exOf to return wrong values

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 16, 2022
…exOf to return wrong values

This patch solves two problems after `Tree.setItemCount()`
* Unexpected values of `Tree.indexOf()` - was caused by non-coherent
  cached values in `Tree.lastIndexOf`, `Tree.hLastIndexOf`. This was an
  oversight in Bug 575787 that optimized `Tree.setItemCount()`.
* `SWT.SetData` events were received with the same value in
  `Event.index` - this was partially fixed years ago in Bug 206806,
  but it overlooked the case of items that are direct descendants of
  Tree root. It didn't matter much until `Tree.setItemCount()` was
  optimized in Bug 575787, which changed how items are inserted.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
@SyntevoAlex
Copy link
Member

Please find patch in #318

niraj-modi pushed a commit that referenced this issue Aug 17, 2022
…rong values

This patch solves two problems after `Tree.setItemCount()`
* Unexpected values of `Tree.indexOf()` - was caused by non-coherent
  cached values in `Tree.lastIndexOf`, `Tree.hLastIndexOf`. This was an
  oversight in Bug 575787 that optimized `Tree.setItemCount()`.
* `SWT.SetData` events were received with the same value in
  `Event.index` - this was partially fixed years ago in Bug 206806,
  but it overlooked the case of items that are direct descendants of
  Tree root. It didn't matter much until `Tree.setItemCount()` was
  optimized in Bug 575787, which changed how items are inserted.

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
@iloveeclipse
Copy link
Member

iloveeclipse commented Aug 24, 2022

Schould be fixed via #335

SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 25, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue eclipse-platform#287
* Fix for Issue eclipse-platform#333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Aug 25, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue eclipse-platform#287
* Fix for Issue eclipse-platform#333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
akurtakov pushed a commit to syntevo/eclipse.platform.swt that referenced this issue Sep 4, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue eclipse-platform#287
* Fix for Issue eclipse-platform#333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
akurtakov pushed a commit to syntevo/eclipse.platform.swt that referenced this issue Sep 7, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue eclipse-platform#287
* Fix for Issue eclipse-platform#333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit to syntevo/eclipse.platform.swt that referenced this issue Sep 14, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue eclipse-platform#287
* Fix for Issue eclipse-platform#333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
SyntevoAlex added a commit that referenced this issue Sep 14, 2022
On Windows, Tree maintains its items with a single linked list. For this
reason, "find previous" operation is O(N) because it requires walking
from start of the list.

When inserting an item, it needs to be linked into the list, which
involves finding the previous item and updating its 'next' field, which
is O(N) for one item and O(N*N) for inserting N items.

The workaround is to bulk insert all items at the same position. This
means that if TreeItem had no children, inserting N items into it is
O(N) instead of O(N*N).

On top of that, this patch preallocates 'Tree.items[]' to hold exactly
the required number of items.

Resulting performance improvement on my machine:
                |      before patch     |      after patch      |
                +-----------+-----------+-----------+-----------+
                |   REGULAR |   VIRTUAL |   REGULAR |   VIRTUAL |
----------------+-----------+-----------+-----------+-----------+
   10_000 items |   0,44sec |   0,43sec |   0,07sec |   0,06sec |
  100_000 items | 113,83sec | 111,95sec |   0,70sec |   0,63sec |
1_000_000 items |  too long |  too long |   6,82sec |   6,28sec |

This patch combines:
* Original patch for Bug 575787 (the parts that were not reverted later)
* Fix for Issue #287
* Fix for Issue #333
* Converting some test snippets into JUnit tests

Signed-off-by: Alexandr Miloslavskiy <alexandr.miloslavskiy@syntevo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
regression Something that used to work Windows Happens on Windows OS
Projects
None yet
6 participants