Implement jar directory snapshot caching. #1546

Merged
merged 2 commits into from Mar 12, 2014

Projects

None yet

2 participants

@ratnikov
Contributor
ratnikov commented Mar 5, 2014

Currently, when we try to load a jar resource, we go through all the jar entries to figure out if there are any that match the directory information. When doing globbing or Dir.entries it also currently scans the jar again.

This PR optimizes this by creating a static JarCache reference that caches information about loaded jars, including representation of the directory structure. The inform1.7.10: user 1m24.332s
HEAD: user 0m19.604sation is stored in a WeakHashMap which should allow it to be garbage collected. Benchmarking on a jar containing files from jruby.git/core (just over 9000 files) shows about 4x improvement:

time bin/jruby -e '5.times { Dir.glob("file:/home/ratnikov/jruby.git/stuff.jar!core/**/*") }'

Results for 1.7.10: 1m24.332s
Results for HEAD: 0m19.604s

I have considered doing a per Ruby runtime cache, but that would require resources to have a reference to a Runtime and caching cross-runtime jar contents is also probably okay.

@ratnikov
Contributor

I've updated the PR to make sure that caching respects modification time and added a test to verify it.

This didn't really affect runtime for my stuff.jar test:

$ time bin/jruby -e '5.times { Dir.glob("file:/home/ratnikov/jruby.git/stuff.jar!/core/**/*") }'

real    0m11.145s
user    0m21.404s
sys     0m1.048s

Please review/merge. :)

@ratnikov
Contributor

Seems like the break is due to jruby-1_7 being broken.

ratnikov added some commits Mar 6, 2014
@ratnikov ratnikov Implement jar directory snapshot caching.
This improves jar globbing performance by about 4x on a simple benchmark.

For a jar containing 9156 files, 5.times { Dir.glob("file.jar!/**/*") } is:

1.7.10: user    1m24.332s
HEAD:   user    0m19.604s
1ac6e61
@ratnikov ratnikov Made the jar caching respect changing of jars on the filesystem. f780bce
@enebo enebo merged commit 585f05b into jruby:jruby-1_7 Mar 12, 2014
@enebo enebo added this to the JRuby 1.7.12 milestone Mar 12, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment