feat(scope): add scope stanza for library visibility control

[samoht]

The scope stanza controls which libraries from a subdirectory are visible to the current project. This allows for selectively exposing libraries when vendoring external projects.

Syntax:

  (scope
   (dir <directory>)
   (packages <pkg1> <pkg2> ...))

The (dir ...) field specifies an immediate subdirectory containing libraries. The (packages ...) field lists packages whose libraries should be visible. All public libraries from the specified packages become accessible.

[samoht]

For context: this is a trimmed-down version of #13325 with a new name and probably targeting the right part of the codebase.

I'm not super happy with the huge package_db global memo, so any advice on how to make this look better is very welcome!

[samoht]

Ok I think this starts looking a bit better.

Now load_dune.ml don't do anything clever anymore with package and masks. That's now scope.ml that do the proper checks.

[rgrinberg]

Yes, the code looks a lot more sensible now. A few points worth pointing out quickly:

1. Could you preserve all the old names in Dune_load.packages? These changes aren't really required and make the diff noisy. In general, let's just focus on the absolute essentials so that I can manage going through review in a reasonable amount of time. We can rename things if it's needed later. If there are other changes that are gratuitous that I haven't noticed, it's a good opportunity to remove them as well.

2. All of the new scoping code should be in its own module eg. Package_scope. This is again done for ease of review, and as far as I can tell you aren't sharing any implementation with Scope anyway.

3. All full source traversals need to go into Dune_load so that they fused with other such traversals. Otherwise, users that aren't using this feature will get have slower null builds.

[samoht]

Thanks for the very useful directions, this is really helpful!

I think I have something that starts to look better now. I am still not super confident that the logic in package_scope.ml is the most efficient one, but at least I feel the rest of the code fits a bit better with what I understand Dune is doing (I might be wrong, though).

Happy to get another quick review when you have time :-)