Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse Indexables under a source root as a group while running EmbeddedIndexers #3516

Merged
merged 3 commits into from
Jan 31, 2022

Conversation

dbalek
Copy link
Contributor

@dbalek dbalek commented Jan 26, 2022

This is an attempt to fix the performance problem found while trying to open and index larger mx based project (GraalVM's Truffle). Because of an EmbeddingIndexer registered for the text/x-java mime type (TestMethodFinderImpl), thousands of the javac compiler instances are created to index the project sources (one instance per source root via JavaCustomIndexer and instance per source/indexable via RepositoryUpdater.indexEmbeddings.
This change tries to parse all sources/indexables under a source root as a group while running EmbeddedIndexers. Is saves thousands of the javac instances (only two instances per source root are created - one via JavaCustomIndexer, the second via RepositoryUpdater.indexEmbeddings) and approximately 20% of indexing time for the Truffle mx project.

@dbalek dbalek added work-in-progress do not merge Don't merge this PR, it is not ready or just demonstration purposes. labels Jan 26, 2022
Copy link

@JaroslavTulach JaroslavTulach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding of the code changes is limited, but:

  • reducing the number of javac instances by thousands is a great thing
  • allowing people to use the same API without writing special case of Java support is great too

You have my emotional support and approval!

@dbalek dbalek removed work-in-progress do not merge Don't merge this PR, it is not ready or just demonstration purposes. labels Jan 28, 2022
Copy link
Contributor

@jlahoda jlahoda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems sensible to me.

@dbalek dbalek merged commit 416e6fb into apache:master Jan 31, 2022
@dbalek dbalek deleted the dbalek/indexing branch January 31, 2022 07:30
jhorvath pushed a commit to jhorvath/netbeans that referenced this pull request Feb 2, 2022
jhorvath pushed a commit to jhorvath/netbeans that referenced this pull request Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants