Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize MimePath#validate by first querying the MimePath cache before attempting to parse #3976

Merged

Conversation

matthiasblaesing
Copy link
Contributor

No description provided.

@matthiasblaesing
Copy link
Contributor Author

I was annoyed by the performance of scanning of JS sources and I had the feeling, that the performance dropped after the changes from #3516. While I'm still not sure whether performance suffered, I noticed a strange thing in the profile (npss in the file):

baseline.zip

image

A significant potion of the total time is spend in MimePath#get. As this was one of the big contributors, I had a look there and MimePath has a cache for resolved String -> MimePath mappings and the idea is: If we know the mapping from string to MimePath, we also know, that the string is a valid mimepath, so I added a fast path for the validation.

A totally unscientific test looks promising:

Baseline

 4.7.2log4j:WARN No appenders could be found for logger (nu.validator.source.LocationRecorder).
     [exec] log4j:WARN Please initialize the log4j system properly.
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/tu/edu-udo-itmc-dev-presencetracking took: 413.883 ms (New or modified files: 41.000, Deleted files: 0) [Adding listeners took: 4 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 1 source roots took: 413.883 ms (New or modified files: 41.000, Deleted files: 0) [Adding listeners took: 4 ms]
     [exec] INFO [org.netbeans.ui.indexing]: Indexing finished, indexing took 414.351 ms.
     [exec] INFO [org.netbeans.ui.indexing]: Indexing started, time from last indexing 0 ms.
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Resolving dependencies took: 15 ms
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 0 binary roots took: 0 ms
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/reststubs.zip took: 161 ms (New or modified files: 14, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/corestubs.zip took: 274 ms (New or modified files: 54, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/domstubs.zip took: 469 ms (New or modified files: 133, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 3 source roots took: 904 ms (New or modified files: 201, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.ui.indexing]: Indexing finished, indexing took 919 ms.

Patched:

     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/tu/edu-udo-itmc-dev-presencetracking took: 342.169 ms (New or modified files: 41.000, Deleted files: 0) [Adding listeners took: 3 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 1 source roots took: 342.169 ms (New or modified files: 41.000, Deleted files: 0) [Adding listeners took: 3 ms]
     [exec] INFO [org.netbeans.ui.indexing]: Indexing finished, indexing took 342.572 ms.
     [exec] INFO [org.netbeans.ui.indexing]: Indexing started, time from last indexing 0 ms.
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Resolving dependencies took: 14 ms
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 0 binary roots took: 0 ms
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/reststubs.zip took: 168 ms (New or modified files: 14, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/corestubs.zip took: 276 ms (New or modified files: 54, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Indexing of: /home/matthias/src/netbeans/nbbuild/netbeans/webcommon/jsstubs/domstubs.zip took: 445 ms (New or modified files: 133, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater]: Complete indexing of 3 source roots took: 889 ms (New or modified files: 201, Deleted files: 0) [Adding listeners took: 0 ms]
     [exec] INFO [org.netbeans.ui.indexing]: Indexing finished, indexing took 903 ms.

There could be pathological cases (the ones where tosString called on a CharSequence is the bottleneck, but I expect the string case the common case.

@dbalek what do you think?

@sdedic sdedic requested a review from dbalek April 13, 2022 08:37
@@ -131,7 +131,7 @@ public final class MimePath {
"video" //NOI18N
));

private static final Map<String,Reference<MimePath>> string2mimePath = new HashMap<String,Reference<MimePath>>();
private static final Map<String,Reference<MimePath>> string2mimePath = new HashMap<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Synchronization looks ok to me, but what about Collections.synchronizedMap and forgetting about those synchronized blocks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Valid point, I switched to ConcurrentHashMap which should be better than the suggested synchronizedMap (finer Locking). What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthiasblaesing ConcurrentHashMap is more performant. I assume we won't be storying nulls as values though.

@@ -278,6 +278,12 @@ public static boolean validate(CharSequence type, CharSequence subtype) {
* @since 1.7
*/
public static boolean validate(CharSequence path) {
String pathString = path == null ? null : path.toString();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't parseImpl() immediately blow up with null anyway? Could simplify and let it throw NPE here?

Copy link
Contributor

@vieiro vieiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, not keys nor values (Weak References) seem to be null. ConcurrentHashMap will do fine!

…e attempting to parse

For the changes regarding the switch from a synchronized access to a
HashMap to a ConcurrentHashMap:

The existing synchronisation only protected the get and set operations
on the map and not some secondary functionality (iteration for example).
Using a ConcurrentHashMap should thus improve performance and not
change the semantics.
@matthiasblaesing
Copy link
Contributor Author

@vieiro thanks for the additional check and yes this should be safe. Tests are happy, so lets get this in.

@matthiasblaesing matthiasblaesing merged commit e623b46 into apache:master Apr 15, 2022
@matthiasblaesing matthiasblaesing deleted the mime_parser_performance branch April 16, 2022 08:53
@neilcsmith-net neilcsmith-net added this to the NB15 milestone Jul 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants