Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memoized implementation for PathMatchingResourcePatternResolver [SPR-17344] #21878

Closed
spring-projects-issues opened this issue Oct 5, 2018 · 8 comments
Labels
in: core

Comments

@spring-projects-issues
Copy link
Collaborator

@spring-projects-issues spring-projects-issues commented Oct 5, 2018

Rahul Shinde opened SPR-17344 and commented

For applications that get bundled as an uber jar, while benchmarking the startup timings, it was found that there is always a fixed cost associated with every lookup that is performed by PathMatchingResourcePatternResolver (PMRPR) while component scanning.

This is a result of opening up the jar file and traversing the entries each time when PMRPR is used to find the resources.

With 10K+ beans and large number of packages for component-scanning, this adds up pretty quick. 

After adding a Memoized implementation of the PathMatchingResourcePatternResolver, the refresh timings dropped around 2-4 minutes depending on the hardware spec.

The Memoized implementation builds a cache when a given jar is looked up the very first time. For subsequent lookups, it simply does a binary search over the cached entries. 

This memoized implementation is attached to the ApplicationContext by overriding getResourcePatternResolver(..)

It would be good to have this as an option for applications that want to use a memoized version of PMRPR.

 

 


Affects: 5.0.9, 5.1 GA

Referenced from: pull request #1981

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 5, 2018

Stéphane Nicoll commented

That's interesting. Have you considered using the components index on this application?

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 8, 2018

Rahul Shinde commented

The index approach though I agree would have helped doesn't work for us at the moment.

We have use-cases where @ComponentScan uses filters on the beans that need to be discovered that reside in other modules. The spring-context-indexer I think gathers all the beans in the module and not necessarily the ones specified by @ComponentScan.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 9, 2018

Stéphane Nicoll commented

It does work regardless of the modules you have. You just need to make sure to apply the indexer for every module that defines components.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 10, 2018

Rahul Shinde commented

It is not really a limitation of the indexer but how each of the modules gets built and what information they have at build time.
Since the source of @ComponentScan resides in another module, the indexer for the module that has the components wont even know about it and would build up the full list as their build lifecycle is managed independently by separate teams.
At runtime however, this gets resolved correctly when all the modules come together and only components matching the criteria get discovered.
We also happen to auto-generate components that currently cannot take advantage of the indexer (limitation of our incremental build process).

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 10, 2018

Stéphane Nicoll commented

That's the wrong place to discuss this but it doesn't matter where @ComponentScan is. As for generating components at runtime, this should be possible if you use an AP and they are ordered properly. I'd be interested to know more offline to see if there is something we can improve in the indexer for your use cases: snicoll at pivotal dot io.

@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Oct 12, 2018

Rahul Shinde commented

Let me run and see if this is reproducible with the latest version. Will contact you offline. Thanks

@spring-projects-issues spring-projects-issues added status: waiting-for-triage type: enhancement in: core and removed type: enhancement labels Jan 11, 2019
@bclozel bclozel added the status: waiting-for-feedback label Apr 1, 2020
@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Apr 8, 2020

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

@spring-projects-issues spring-projects-issues added the status: feedback-reminder label Apr 8, 2020
@spring-projects-issues
Copy link
Collaborator Author

@spring-projects-issues spring-projects-issues commented Apr 15, 2020

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

@spring-projects-issues spring-projects-issues removed status: waiting-for-feedback status: feedback-reminder status: waiting-for-triage labels Apr 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: core
Projects
None yet
Development

No branches or pull requests

2 participants