Integration tests for XS Node.js bundler parity #2252
Comments
29 tasks
kriskowal
changed the title
kriskowal
added a commit
that referenced
this issue
Jun 4, 2024
Closes: #2295 Refs: #400, #2252 ## Description This change adds a mode to the `bundle-source` command with the initial flag `--no-transforms` that generates “endo zip base64” style bundles without applying the module-to-program transform and SES shim censorship evasion transforms, such that the original files on disk appear in the zip file. This is a preparatory step, necessary for building test artifacts, in advance of full support for this bundle style. ### Security Considerations `bundle-source` is part of the Endo and Agoric toolkit and it, or its surrogate, participate in the toolchain for generating content that can be confined by Hardened JavaScript, but is not trusted by Hardened JavaScript at runtime. It does however _currently_ run with all the authority of the developer in their development environment and its integrity must be carefully guarded. ### Scaling Considerations No improvements expected at this time, but in pursuit of #400, it may be possible to move the heavy and performance sensitive JavaScript transform components from `bundle-source` to `import-bundle` and only suffer the performance cost of these transforms on Node.js, where those costs are more readily born by some runtimes. Precompiled bundles may continue to be the preferred medium for deployment to the web, for example. ### Documentation Considerations We will need to advertise the `--no-transforms` flag eventually, since there will be a period where it is advisable if not necessary to generate contracts and caplets targeting the XS runtime. ### Testing Considerations I have included a test that verifies the API behavior and manually run the following to verify behavior for the CLI: ```sh rm -rf bundles yarn bundle-source --no-transforms --cache-json bundles demo/circular/a.js circular-a rm -rf circular-a mkdir -p circular-a jq -r .endoZipBase64 bundles/bundle-circular-a.json | base64 -d > circular-a/circular-a.zip (cd circular-a; unzip circular-a.zip) jq . circular-a/compartment-map.json # verifying the final module entires have parser: 'mjs' ``` ### Compatibility Considerations This flag is opt-in and breaks no prior behaviors. This introduces a new entry to the build cache meta-data and may cause some bundles to be regenerated one extra time after upgrading. ### Upgrade Considerations This should not impact upgrade, though it participates in the greater #400 story which will require xsnap upgrades to come to bear.
kriskowal
added a commit
that referenced
this issue
Sep 18, 2024
SES exposes `ses/console-shim.js` in `package.json`, but lacked the “thunk” module needed to entrain the actual implementation. Being able to compose all the layers of `ses` *except* `ses/assert-shim.js` proves to be necessary to make progress on parity testing with XS #2252
kriskowal
added a commit
that referenced
this issue
Sep 24, 2024
Refs: #2252 ## Description To test feature the degree of compatibility between a version of XS, XS with Endo shims, and Node.js with Endo shims, we need a shim for ModuleSource proper. This lets us create an environment with a global `ModuleSource`, where `ModuleSource` is a shared intrinsic of all Compartments. This change both introduces the shim and the necessary accommodations for the existence of a global ModuleSource shared intrinsic in SES. ### Security Considerations The new ModuleSource is subject to hardening of shared intrinsics during SES lockdown. A failure to harden the shared intrinsic would lead to a potential for interference or communication between isolated compartments. ### Scaling Considerations None. ### Documentation Considerations None. ### Testing Considerations This includes a test for SES that verifies that ModuleSource is properly propagated and hardened. ### Compatibility Considerations None. ### Upgrade Considerations None.
kriskowal
added a commit
that referenced
this issue
Sep 26, 2024
Refs: #2463, #2252 ## Description In #2463 we introduced ModuleSource to the SES permits, but this interacted catastrophically with the native XS ModuleSource. We reverted this change #2468 to unbreak Agoric SDK integration. This change reintroduces the ModuleSource permits, such that they are compatible with both XS and the `@endo/module-source/shim.js`, which anticipates the introduction of an AbstractModuleSource base class. Because SES can more gracefully tolerate the absence of an permitted [[Proto]] than the presence of a non-permitted [[Proto]], this adjusts the shim to ensure that the AbstractModuleSource shape exists as a side-effect of repairs/taming, before permits are applied. ### Security Considerations Increase in memory safety exposure in native code implementation of ModuleSource where applicable. ### Scaling Considerations None. ### Documentation Considerations This change reintroduces a note in NEWS.md for the next release. ### Testing Considerations The prior regression went unnoticed because we did not yet have CI for XS #2465. With this change, `yarn test:xs` in SES validates the shim on XS. We also test `@endo/module-source/shim.js` in `ses/test/module-source.test.js` on Node.js. ### Compatibility Considerations This change is designed to tolerate either path forward for the language, whether or not it accepts an AbstractModuleSource base class. ### Upgrade Considerations None.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Refs: #400
Establish tests that demonstrate that SES and XS native compartments have parity in integration with the kitchen sink behaviors of Endo/Agoric bundles.
Design
It should be possible to use
@endo/compartment-mapper/bundle.jswith a new"xs"tag to create a script that entrains a JSON bundle andimportBundlewith all its dependencies such thatxstcan run to completion and compare the output against an oracle. In combination with #2250, this will remove Babel from the transitive dependencies ofimportBundle.The text was updated successfully, but these errors were encountered: