Resource loading #135
Merged
Resource loading #135
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
squaremo
force-pushed
the
module-relative-reads
branch
3 times, most recently
from
d1a0f6e
to
81d6116
Compare
This implements a "magic" module '@jkcfg/std/resource'. It's magic because it's generated each time it's imported, so that it can take into account the path of the importing module (known at import time, and not thereafter). The implementations is this: create a module that embeds the base path of the module, and exports a procedure that does reads relative to the base path. This has some flaws: - at present, jk (v8worker2, really) gives imports using the same specifier (in this case `@jkcfg/std/resource`) the same module, which is not much use for magic modules. - it relies on the module base path being readable (if you change the input path when you run jk, it might not be)
This commit adds a test that resource() can succeed even if the module directory is not under the input directory. The test does not pass, since all reads are currently required to be under the input directory. It also corrects the guard against reading outside of the input directory: it's possible for os/path/filepath.Rel to return a relative path that starts with parent directories, i.e., with `..`.
Just using std.read() for resources won't work in general, because the desired path may not be under the input directory (and reads are restricted to paths under the read directory). To enable reads from module directories, while keeping the guard against reading outside the allowed paths, keep track of each module that imports `@jkcfg/std/resource' and look up the module path when its `resource` procedure is called, to use as a base path.
squaremo
force-pushed
the
module-relative-reads
branch
from
81d6116
to
d07e6c4
Compare
To be able to identify the base path of a resource read, I added a `module` option to `read`. That is now "generally available", meaning any read might include it, if it can guess a module hash, and read from some other module's base path. To prevent abuse of this mechanism, salt the module hash using random bytes generated when starting up. The randomness does not hinder repeatability, since the hashes are opaque and only used in the scope of a run anyway.
|
I totally misread the test case :) all good :) |
dlespiau
approved these changes
Mar 4, 2019
| // as generating the magic modules when they are imported. | ||
| type ModuleResources struct { | ||
| // module hash -> basePath for resource reads | ||
| modules map[string]string |
There was a problem hiding this comment.
I think we can't manipulate this map from different goroutines, so we indeed don't need locking?
There was a problem hiding this comment.
At present it won't happen, but I could be defensive and put a mutex on it
squaremo
force-pushed
the
module-relative-reads
branch
from
fed3321
to
a2bd768
Compare
Closed
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This implements a "magic" module '@jkcfg/std/resource'. It's magic
because it's generated each time it's imported, so that it can take
into account the path of the importing module (known at import time,
and not thereafter).
The implementation is this:
modulefor specifying a module to use as the base path (a missing value means use the input directory);@jkcfg/std/resourceis imported, hash the current import base path (i.e., the directory of the importing module), then load module code exporting a procedure that reads using the hash.This has some flaws:at present, jk (v8worker2, really) gives imports using the samespecifier (in this case
@jkcfg/std/resource) the same module,which is not much use for magic modules.
it relies on the module base path being readable (if you change theinput path when you run jk, it might not be)
(EDIT: strike out flaws, which have been excised, and explain how module base paths are identified.)