plenv rehash is so slow

[miyagawa]

plenv rehash is so slow (takes about 10 seconds) as I have many plenv versions installed. rbenv rehash is about 300ms.

It annoys the hell out of me because plenv install-cpanm adds a rehash hook for every invocation for cpanm, even for --info usage etc.

Turns out there're lots of shims like xsubpp5.20.1, and that is causing the extra step.

➜  .plenv  ls shims/ | grep  5. | wc -l
     792
➜  .plenv  ls shims/ | grep  -v 5. | wc -l
     320

These executables are completely useless because plenv takes care of switching versions.

I'm sure if plenv stops symlinking these executables plenv rehash is much faster.

[tokuhirom]

Hm. But how do I detect these files?

[kazuho]

Why do you have to regenerate symlinks again? Can't you just have a bin/ directory for each version of Perl and alter the link that points to the directory, instead of creating symlinks for each of the executables under bin/?

disclaimer: I am newcomer to this topic, please forgive me if I am missing the context.

[miyagawa]

@kazuho they are not symlinks.

[miyagawa]

@tokuhirom:

I'm sure you can reproduce this on your env, but in case it isn't here's what it loos like:

> ls ~/.plenv/shims | grep zipdetails                                                                            
zipdetails
zipdetails5.17.10
zipdetails5.17.11
zipdetails5.18.1
zipdetails5.18.2
zipdetails5.18.3
zipdetails5.18.4
zipdetails5.19.0
zipdetails5.19.1
zipdetails5.19.3
zipdetails5.19.9
zipdetails5.20.0
zipdetails5.20.1
zipdetails5.21.4
zipdetails5.21.7
zipdetails5.21.8

> ls ~/.plenv/versions/5.20.1/bin/ | grep zipdetails
zipdetails
zipdetails5.20.1

> zipdetails5.20.0 
plenv: zipdetails5.20.0: command not found

The `zipdetails5.20.0' command exists in these Perl versions:
  5.20.0
  5.20.0-RC1

Because these commands exist for each perl, these shims are generated to keep track of the aggregated bin files. However this executable conflicts with plenv/perlbrew in the first place (they're there in /usr/bin for example to make sure you'll run it with the right perl version, in case you have multiple perl versions in /usr/bin).

I don't know what is the right thing to do, but an easy patch would be just ignore files that are a) known to come with perl core and b) ends with [5\.\d+\.d+]$.

[miyagawa]

Actually, i was about to go ahead and delete all these command5.* files, and found out that these are actually symlinks inside of their own perl bin.

 ll ~/.plenv/versions/*/bin | grep zipdetails
-r-xr-xr-x+ 1 miyagawa  staff    47K Apr 22  2013 zipdetails
-rwxr-xr-x+ 1 miyagawa  staff    47K Apr 13  2013 zipdetails
-r-xr-xr-x+ 1 miyagawa  staff    47K Aug 18  2013 zipdetails
lrwxr-xr-x  1 miyagawa  staff    61B Apr  7  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.17.10/bin/zipdetails5.17.10
-rwxr-xr-x+ 1 miyagawa  staff    47K Apr  7  2013 zipdetails5.17.10
lrwxr-xr-x  1 miyagawa  staff    61B Apr 22  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.17.11/bin/zipdetails5.17.11
-rwxr-xr-x+ 1 miyagawa  staff    47K Apr 22  2013 zipdetails5.17.11
-rwxr-xr-x+ 1 miyagawa  staff    47K May 12  2013 zipdetails
-rwxr-xr-x+ 1 miyagawa  staff    47K May 13  2013 zipdetails
-r-xr-xr-x+ 1 miyagawa  staff    47K May 27  2013 zipdetails
-r-xr-xr-x+ 1 miyagawa  staff    47K Aug 12  2013 zipdetails
-rw-rw-rw-+ 1 miyagawa  staff    47K Aug 12  2013 zipdetails5.18.1
lrwxr-xr-x  1 miyagawa  staff    63B Dec  5  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.18.2-RC1/bin/zipdetails5.18.2
-rwxr-xr-x+ 1 miyagawa  staff    47K Dec  5  2013 zipdetails5.18.2
-r-xr-xr-x+ 1 miyagawa  staff    47K Jan  8  2014 zipdetails
-rw-rw-rw-+ 1 miyagawa  staff    47K Jan  8  2014 zipdetails5.18.2
lrwxr-xr-x  1 miyagawa  staff    63B Sep 29  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.18.3-RC2/bin/zipdetails5.18.3
-rwxr-xr-x+ 1 miyagawa  staff    47K Sep 29  2014 zipdetails5.18.3
lrwxr-xr-x  1 miyagawa  staff    59B Oct  2  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.18.4/bin/zipdetails5.18.4
-rwxr-xr-x+ 1 miyagawa  staff    47K Oct  2  2014 zipdetails5.18.4
lrwxr-xr-x  1 miyagawa  staff    59B May 25  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.19.0/bin/zipdetails5.19.0
-rwxr-xr-x+ 1 miyagawa  staff    47K May 25  2013 zipdetails5.19.0
lrwxr-xr-x  1 miyagawa  staff    59B Jun 22  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.19.1/bin/zipdetails5.19.1
-rwxr-xr-x+ 1 miyagawa  staff    47K Jun 22  2013 zipdetails5.19.1
lrwxr-xr-x  1 miyagawa  staff    59B Sep  9  2013 zipdetails -> /Users/miyagawa/.plenv/versions/5.19.3/bin/zipdetails5.19.3
-rwxr-xr-x+ 1 miyagawa  staff    47K Sep  9  2013 zipdetails5.19.3
lrwxr-xr-x  1 miyagawa  staff    59B Feb 24  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.19.9/bin/zipdetails5.19.9
-rwxr-xr-x+ 1 miyagawa  staff    47K Feb 24  2014 zipdetails5.19.9
lrwxr-xr-x  1 miyagawa  staff    63B May 17  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.20.0-RC1/bin/zipdetails5.20.0
-rwxr-xr-x+ 1 miyagawa  staff    47K May 17  2014 zipdetails5.20.0
lrwxr-xr-x  1 miyagawa  staff    59B May 27  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.20.0/bin/zipdetails5.20.0
-rwxr-xr-x+ 1 miyagawa  staff    47K May 27  2014 zipdetails5.20.0
-r-xr-xr-x+ 1 miyagawa  staff    47K Dec  8 18:27 zipdetails
-rwxr-xr-x+ 1 miyagawa  staff    47K Sep 17  2014 zipdetails5.20.1
lrwxr-xr-x  1 miyagawa  staff    59B Oct  3  2014 zipdetails -> /Users/miyagawa/.plenv/versions/5.21.4/bin/zipdetails5.21.4
-rwxr-xr-x+ 1 miyagawa  staff    47K Oct  3  2014 zipdetails5.21.4
lrwxr-xr-x  1 miyagawa  staff    59B Jan 25 18:51 zipdetails -> /Users/miyagawa/.plenv/versions/5.21.7/bin/zipdetails5.21.7
-rwxr-xr-x+ 1 miyagawa  staff    47K Jan 25 18:51 zipdetails5.21.7
lrwxr-xr-x  1 miyagawa  staff    59B Jan 25 18:47 zipdetails -> /Users/miyagawa/.plenv/versions/5.21.8/bin/zipdetails5.21.8
-rwxr-xr-x+ 1 miyagawa  staff    47K Jan 25 18:46 zipdetails5.21.8
-r-xr-xr-x+ 1 miyagawa  staff    47K Oct  2  2014 zipdetails

It is fairly annoying, and I wonder if there's a perl build option to disable them in the first place...

[miyagawa]

That being said, this whole plenv commands including this rehash are just ported over from rbenv, and @kazuho is right that regenerating the whole files don't seem to make sense.

unless i'm stupid, what I've found out so far is that bash for loop is so slow.

I have 3716 bin files across various perl versions I have,

ls -l ~/.plenv/versions/*/bin/* | wc -l
    3716

and just iterating them over in register_shims which calls register_shim is so slow. I replaced it with an inline invocation and the process is now 4 sec faster.

at this point i'm fairly convinced that I should just uninstall these dev versions and call it a day :/