deno vendor as cache override

[nayeemrmn]

Proposed new design for vendoring in Deno:

• Support { "vendor": true } option in deno.json. (Maybe a --vendor flag as well for completeness.)
• If it is set, use ./vendor in place of $DENO_DIR/deps.
• If it is set, use readable cache paths for URLs instead of encoding to hashes. ./vendor should look the same as with deno vendor.
• Remove deno vendor subcommand.

Advantages:

• Adding { "vendor": true } once to set up a vendor directory is much easier than calling deno vendor on every dep bump. It will be updated on normal usage.
• Import maps are non-composable; the import map slot is a valuable resource for users which would no longer be occupied by the vendoring mechanism.
• The vendor directory needs to be known at runtime to capture true dynamic imports.
• Runtime behaviour will be better preserved (see current import.meta values from vendored code).
• No subcommand necessary.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[marvinhagemeister]

+1 Just came across this issue and I really like the proposal. It's a much more ergonomic workflow than the existing vendor command.

[dsherret]

I'm working on this. Current thought is to keep deno vendor for cases where people need portable vendoring with an import map, but then also have this ability to have a local cache folder that gets updated automatically.

1. Add { "remoteModulesDir": true } to deno.json and --remote-modules-dir
   • Compliments "nodeModulesDir": true and --node-modules-dir
   • Doesn't confuse it as much with deno vendor
2. A remote_modules directory is created on first run or by running deno cache <entrypoint>
   • remote_modules/manifest.json
     • Contains the local paths <-> remote urls
       • I initially thought a reverse mapping wouldn't be necessary, but it is for lsp support. In the lsp we need a way to be in a file and know what url it's for.
     • Contains the header data for certain urls
   • Should work well with multiple processes editing this directory at the same time
   • Strive to have somewhat human readable paths so people can find their way around easily
   • Maybe allows for a file path to be added that's not in the manifest as long as one of its ancestor directories is in the manifest
     • Probably this would match a file path as if it were on the remote server using the ancestor directory in the manifest for the base url. Probably if the same path is in the manifest with different casing, then the file won't be matched on the file system as this wouldn't be cross platform and wouldn't work with case insensitive file systems. More thought necessary here.
   • This will run slightly slower than the global cache due to its more user friendly directory structure

[nayeemrmn]

• Contains the header data for certain urls

Yes! The way I imagined it is we would store metadata only for modules which have metadata which is different from the default/specifier-inferred value. e.g. a .ts URL which has a application/javascript header for whatever reason (transpiled by CDN?).

+1 on "remoteModulesDir" and remote_modules/.

I hope we can deprecate deno vendor though... the remaining use case for it sounds appropriate for a third-party tool.

[dsherret]

This issue will close once #19977 merges, but I opened #20015 (remote to local specifier mapping) as a followup to improve the experience in the editor.