Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache #8023
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hsbt
changed the title
|
Is it possible to include the current PR in Ruby 3.2.3? |
|
Thanks, created https://bugs.ruby-lang.org/issues/19771 |
Rebuilding the loaded feature index slowed down with the bug fix for #17885 in 79a4484. The slowdown was extreme if realpath emulation was used, but even when not emulated, it could be about 10x slower. This adds loaded_features_realpath_map to rb_vm_struct. This is a hidden hash mapping loaded feature paths to realpaths. When rebuilding the loaded feature index, look at this hash to get cached realpath values, and skip calling rb_check_realpath if a cached value is found. Fixes [Bug #19246]
Number of lstat and stat syscalls for each 'require'd file is doubled, because rb_realpath_internal is called from two places with the same arguments in require_internal; once for checking the realpaths cache, and once in load_iseq_eval when iseq is not found. Introduce rb_realpath_internal_cached function to reuse the realpath_map cache which memoizes rb_realpath_internal function, leading to less syscalls and increased startup performance depending on the cost of the syscalls in a particular environment.
nagachika
force-pushed
the
faster-loading-32
branch
from
58fd9dd
to
332e063
Compare
nagachika
approved these changes
Jul 29, 2023
matzbot
pushed a commit
that referenced
this pull request
Jul 29, 2023
…ealpath cache (#8023)" This reverts commit 8346d16. It seems break CI on MinGW. https://github.com/ruby/ruby/actions/runs/5698775895/job/15447455988
MSP-Greg
added a commit
to MSP-Greg/ruby
that referenced
this pull request
Aug 21, 2023
… realpath cache (ruby#8023)"
MSP-Greg
added a commit
to MSP-Greg/ruby
that referenced
this pull request
Aug 21, 2023
… realpath cache (ruby#8023)"
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 21, 2023
… realpath cache (#8023)" (#8252) * Re-apply "Ruby 3.2 - Speed up rebuilding the loaded feature index and realpath cache (#8023)" * [CI] mingw.yml - remove IBM437 encoding for test-all, use cmd shell for test & test-all * Skip failing test on mingw with readline.so Co-authored-by: nagachika <nagachika@ruby-lang.org> --------- Co-authored-by: nagachika <nagachika@ruby-lang.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport "Speed up rebuilding the loaded feature index" (1f8823f) and "Add a realpath cache to reduce number of syscalls" (#8017) to Ruby 3.2.