Obviate Rollup #9686
Assignees
Labels
enhancement
New feature or request
Comments
kriskowal
added a commit
to endojs/endo
that referenced
this issue
Jul 17, 2024
Refs: Agoric/agoric-sdk#9686 ## Description Endo’s `bundle-source` currently generates bundles with the `endoZipBase64` format, which requires the `importBundle` runtime. We typically bootstrap `importBundle` using the `getExport` or `nestedEvaluate` bundle format, which only need `eval` or `compartment.evaluate` to run. These use Rollup internally. Elsewhere, for distributing SES and toward bootstrapping XS and Hermes variants of SES, we have our own bundler. In this change, we expose this format as `endoScript`, an alternative to `getExport` and `nestedEvaluate` which do not depend on Rollup. We introduce a `--format` flag to `bundle-source` so any of the old and the new bundle formats are addressable from the CLI. For easy hacking, we introduce a no-cache mode to `bundle-source` so we can just write a bundle to `stdout`. This required a minor fix to the Compartment Mapper. ### Security Considerations This will introduce a bundle format that is somewhat easier to audit because it preserves the lexical encapsulation of each module. ### Scaling Considerations This may not perform as well as `nestedEvaluate` for bootstrapping, or will at least perform differently. ### Documentation Considerations With this change, we can treat `bundle-source` as a one-stop-shop for all bundling concerns. ### Testing Considerations To manually verify these changes, in `packages/bundle-source`: ``` yarn bundle-source demo/circular/a.js --format endoScript | jq .source -r node -e "$(yarn bundle-source demo/circular/a.js --format endoScript | jq .source -r)" yarn bundle-source -f endoScript --cache-json bundles/ demo/circular/a.js a jq . bundles/bundle-a.json jq -r .source bundles/bundle-a.json | less ``` ### Compatibility Considerations No changes to prior workflows. ### Upgrade Considerations None.
kriskowal
added a commit
to endojs/endo
that referenced
this issue
Jul 19, 2024
Refs: #400, Agoric/agoric-sdk#9686 ## Description Endo’s zip bundle format and Rollup both support importing JSON modules. This change adds JSON module support to Endo’s script bundle format. We need this to have Rollup parity for Agoric Core Eval scripts and also to unblock test262 for XS native compartment parity tests. ### Security Considerations Obviates Rollup, increases legibility of script bundles. ### Scaling Considerations None. ### Documentation Considerations None. JSON module support is assumed. No one will notice unless it doesn’t work. ### Testing Considerations This change adds a JSON module to the existing bundle test fixture. This also verifies parity with the Zip bundle format. ### Compatibility Considerations This make Endo more compatible with Rollup, but Endo continues to not support `import / with { type: "json" }`, which would be necessary for parity with Node.js proper and newer bundlers. ### Upgrade Considerations None.
|
This is integrated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the Problem Being Solved?
Rollup takes an entry point ESM and converts it into a single script. This operation erases the boundaries between packages so an auditor cannot reason about lexical accessibility of powerful objects. We currently use Rollup for Core Eval proposals and bootstrapping scripts in the kernel. Contracts use a Zip archive bundle format, but we have to bootstrap the runtime for that by evaluating a script.
Description of the Design
Long ago, we used Rollup to bundle SES. However, the SES emulation of ESM includes a mechanism for converting individual modules into scripts, and employs a calling convention to link them. In a feat of meta-circularity, we are now a bootstrap SES script by using its ESM emulator to generate a single script with inline linkage. MetaMask grew that bundler out to support CJS. It’s a hair’s breadth away from supporting JSON modules too. It also supports all the Endo compartment mapper support for Node.js “build tags” or “predicates” in
package.json"exports". We also already have a bundler CLI,bundle-source.We can:
bundle-sourceto support other bundle formats includingendoZipBase64(default), and our bootstrapping formatsnestedEvaluteandgetExport.endoScriptformat that uses the Endo bundler to generate a script.In the end, we would replace Rollup commands with:
We would continue to also support a cache mode:
Security Considerations
Increases ceremony in the reviewable core eval scripts in order to make lexical access to powers within individual modules more evident.
Scaling Considerations
None.
Test Plan
Manual.
Upgrade Considerations
None.
Estimate
Should take an afternoon to implement.
Implications
With this implemented,
bundle-sourcebecomes a one-stop shop for all bundling needs for proposing contractsnestedEvaluatefor bootstrapping SwingSet workers--tagflag so--tag=xscan be specified, we make it easier to generate endo test262 scripts for integration testing in Using XS native Compartments for Agoric Vat Workers endojs/endo#400The text was updated successfully, but these errors were encountered: