Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC First cut support for sandboxing procmacros with wasm #7297

Open
wants to merge 2 commits into
base: master
from

Conversation

@jsgf
Copy link
Contributor

commented Aug 25, 2019

This is a first cut experiment at building proc-macros with wasm in order to sandbox them.

It adds a new Kind: HostSandbox, in addition to Host and Target. This is treated as a host build, but its always wasm32-unknown-unknown.

In Cargo.toml, you can set wasm_sandbox = true in addition to proc-macro = true, which enables the use of HostSandbox.

When the compilation actually happens, it's treated as a cross-build to wasm32-unknown-unknown.

Lightly tested on serde, which worked until it couldn't find proc_macro from rustc (well, proc-macro2's build.rs assumes it can't exist and so fakes it out).

Assuming this basic approach is sound, I'm thinking that the wasm_sandbox flag will become a marker that the procmacro could be sandboxed, and then a command-line or cargo config option would request that procmacros be built sandboxed (with further config to forbid non-sandboxed).

TODO:

  • the rustc side
  • what happens if we're cross-compiling to wasm32? Does it conflict or is it ok?

cc @alexcrichton, @dtolnay

@rust-highfive

This comment has been minimized.

Copy link

commented Aug 25, 2019

r? @Eh2406

(rust_highfive has picked a reviewer for you, use r? to override)

@jsgf jsgf changed the title RFC First cut support for sandboxing procmacros with WASM RFC First cut support for sandboxing procmacros with wasm Aug 25, 2019

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 25, 2019

@rust-highfive rust-highfive assigned alexcrichton and unassigned Eh2406 Aug 25, 2019

@jsgf jsgf force-pushed the jsgf:wasm-procmacro branch from f44c081 to c7ad50b Aug 25, 2019

@dtolnay
Copy link
Member

left a comment

Nice!

In the medium term, I would like for crates.io to distribute proc macros as precompiled wasm binaries -- which immediately eliminates all complaints about proc macros taking long to compile.

Is the implementation here forward compatible with a world where serde_derive = { version = "1.0", features = ["some", "features"] } serves me a precompiled wasm with the requested feature set? or do we need to ask that the macro author make some additional guarantees about the possible behavior of the macro when they opt in to wasm-sandbox = true.

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 25, 2019

If you assume features are additive then you could just set them all for prebuilt. Or if some are rare and bloaty then prebuilt could exclude them and you'd need to build locally if you want them.

IOW the prebuilt would have associated features and resolution would work out whether it's usable.

Of course it breaks if features are used wrongly.

@jsgf jsgf force-pushed the jsgf:wasm-procmacro branch from d809d3a to d393d5a Aug 25, 2019

@killercup
Copy link
Member

left a comment

Very cool!

In the medium term, I would like for crates.io to distribute proc macros as precompiled wasm binaries

I'm looking forward to that!

src/cargo/util/toml/mod.rs Show resolved Hide resolved

@jsgf jsgf force-pushed the jsgf:wasm-procmacro branch 2 times, most recently from d843872 to f5e49fe Aug 26, 2019

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Aug 26, 2019

Thanks for getting started on this @jsgf! I think this has the rough shape of what will be necessary, but I think that we'll probably want to pursue the rustc changes first since they're likely the much more meaty ones for now.

Some updates we'll need on the Cargo side eventually are:

  • The wasm-sandbox flag in the manifest will need a feature gate
  • This'll need to still work when the wasm32-unknown-unknown target isn't installed and no crates use wasm-sandbox
  • We'll either want to auto-fix or improve the error with wasm32-unknown-unknown missing and a crate using wasm-sandbox
  • I think we'll want to refactor Kind to just be InternedString instead of a Host or Target variant where the InternedString is just the target of compilation. Ideally Cargo could build all sorts of targets all at once but this "only kind or only host" limitation has been around for a bit too long at this point

I suspect we'll have some sort of different --crate-type feature in rustc itself to indicate this as well, and that'd be keyed off proc macros compiled for wasm. I think we've still got aways to go with this, but it's hopefully not that far away!


In terms of overall feature design I personally feel that precompiled wasm files distributed on crates.io is one of the biggest wins here and is one we'd want to flesh out before stabilizing everything. That doesn't necessarily needed to happen right here on this PR, but would likely be accompanied in the eventual RFC to flesh all this out.

For example I do have concerns about what @dtolnay was mentioning with features but I also have concerns about transitive dependencies. For example the precompiled wasm file might use syn 1.0.3 but if syn 1.0.4 is released later what would happen to the wasm file?

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 26, 2019

@alexcrichton I'm thinking that wasm-sandbox will become advisory - ie, this procmacro supports being sandboxed. Then something like cargo build --sandbox-procmacros would enable it (I guess feature gated, or just unstable), along with --require-sandboxed-procmacros to prevent unsandboxed procmacros.

I think we'll want to refactor Kind to just be InternedString instead of a Host or Target variant where the InternedString is just the target of compilation.

Maybe, but is some logic being applied to Kind in this diff - for example, the build script for a Kind::HostSandbox target will be Host. I'm not how that would look if it were just strings.

Now that this seems to have minimal correct functionality, I've moved into rustc. I'll put up a WIP RFC PR once I've made some headway. I expect it require further changes here.

@jsgf jsgf force-pushed the jsgf:wasm-procmacro branch from f5e49fe to a2b76bd Aug 27, 2019

@jsgf jsgf force-pushed the jsgf:wasm-procmacro branch from a2b76bd to 30a01ed Aug 27, 2019

Use cdylib for wasm32 procmacros
The rationale for this is that rustc doesn't support proc-macro as
a crate type for wasm32, so use cdylib since its functioanlly
equivalent to the crate type we'd want. But it doesn't work because
a proc-macro crate also has language features that we still need
to make available.
@joshtriplett

This comment has been minimized.

Copy link
Member

commented Aug 28, 2019

Based on discussion in the Cargo meeting, note that we don't want to consider this in any way a security feature, only an isolation feature to make it easier to treat macros as functions from inputs to outputs.

@jsgf

This comment has been minimized.

Copy link
Contributor Author

commented Aug 29, 2019

@joshtriplett Totally on board with that characterization.

@bors

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2019

☔️ The latest upstream changes (presumably #7216) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.