Improve and consolidate distribution lookup

[ugexe]

Adds method candidates to CompUnit::Repository::Installation ( CURI ) and CompUnit::Repository::FileSystem ( CURFS ). All distribution searching/resolving goes through this method. This gives developers insight into how a CURI/CURFS/CUR??? resolves a module request, which was not possible with .resolve -- it would chain to the next CUR and only returned a single result. The centralization of the recommendation-manager-like functionality should aid in implementing additional NYI s22 features.

Raku/roast@cab3dd0
https://design.perl6.org/S22.html#candidates

CURI and CURFS changes

!matching-dist is now backed by .candidates, but also caches a single 'best match' so it acts similar to before.

.candidates ( and by extension !matching-dist ) now returns a Distribution object and not ($dist-id, $dist-data). This is possible because of a role that allows our lazy distribution to avoid IO calls to obtain meta data. Previously this was done by passing around a hash internally, but now we dogfood our Distribution interface throughout CURI.

Deprecates .script, and reverts to using just .files - .script was just a fast path of .files under certain conditions and lacked the ability to filter by name. Now .files can provide the faster path for other files automatically. Additionally this fixes an issue where .script would invoke older version bin/ scripts from ~/.perl6

CURFS changes

Creates full Distribution for both -I. and -Ilib, where the later will use the directory recursion it was already doing ( to determine id ) to automatically populate meta data for provides and files ( bin/, resources/ ).

Gives bin scripts a way to more robustly know their own version ( although nyi $*DISTRIBUTION would be the more appropriate way, but would still be helped by this work ):

# bin/zef
use Zef::Client;

sub MAIN("version") {
    my $spec = CompUnit::DependencySpecification.new(:short-name<Zef::Client>);
    say $*REPO.need($spec).distribution.meta<ver version>.first
}

The above currently only works for perl6 bin/zef (See: ugexe/zef#122 (comment)), and would blow up on perl6 -I. bin/zef and perl6 -Ilib bin/zef. This PR works for all 3; -I. ( which we assume is pointing at a directory with a META file ) will pull its version from the meta file, and -Ilib (a directory without a META file) is version 0.

Note this does look at ../ to build the meta data for -Ilib, but this is already the case for CURFS.resources.

[zoffixznet]

ping is this mergeable or what is needed? It's blocking auth tests that I believe are needed to spec the auth key

[ugexe]

Progress is blocked by a consensus ( and possible associated code changes ) for #1853 since it adds the same auth/ver/api lookup/verification to CURFS / -I. -Ilib

Additionally the performance of the CURFS changes probably need to be improved, as it slowed down the roast a fair bit ( although this appeared to be because of how poorly the roast organizes the various modules it uses/tests, resulting in having to iterate and recurse over significantly more files ).

EDIT: Actually performance is probably ok... the speed is exactly the same with TEST_JOBS=1.

[ugexe]

Perhaps the candidates method name should be renamed, since that is already used to do something completely different:

$ perl6 -e 'say &gethostname.candidates()'
(sub gethostname ( --> Str:D) { #`(Sub+{Callable[Str:D]}|140424549133624) ... })

[ugexe]

I was going to wait until after the release was cut, but I only have the next 5 days to keep a close watch on this which is already less than I wanted. So hopefully the release can still be handled without blocking master. @AlexDaniel

[niner]

Yeah!

[AlexDaniel]

@ugexe 👍 Sorry for not writing a response here, I simply didn't see the notification (getting too many of them…). Please feel free to ping me on IRC any time you need a quicker answer from me. Yes, I can easily cut a release from a branch so generally it's not an issue to commit to master (especially once we know that there's a stable revision at some point), and that's exactly what I did for 2018.12 (the release does not include cur-candidates merge, even though the release happened after the branch was merged).

Thank you for your work!

BTW there's a regression associated with this merge, though there's not much info about it currently. See https://colabti.org/irclogger/irclogger_log/perl6-dev?date=2018-12-21#l181