Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable snapshots when cross compiling #13948

Closed
littledivy opened this issue Mar 14, 2022 · 5 comments
Closed

Disable snapshots when cross compiling #13948

littledivy opened this issue Mar 14, 2022 · 5 comments
Labels
build build system or continuous integration related suggestion suggestions for new features (yet to be agreed)

Comments

@littledivy
Copy link
Member

littledivy commented Mar 14, 2022

When cross-compiling, snapshots are built for the host architecture instead of the target. There is no easy workaround for this.

Unblocks our CI to produce the following targets:

@mdekstrand
Copy link

mdekstrand commented Oct 31, 2023

I've been looking into this to support the Conda packages (conda-forge/deno-feedstock#68), and have successfully cross-built Deno for Linux aarch64 (the Conda CI is currently failing, but I have working local builds).

The things I see needing to be done (or nice to have) here:

  • Fix feature defaults to enable a build without unconditionally activating include_js_files_for_snapshotting on dependencies [mandatory, the patch in conda-forge does this crudely]
  • Correct cli/ops/mod.rs to include source code when snapshotting is disabled [mandatory, patch in conda-forge does this]
  • Revisit feature flags for selecting or disabling snapshotting, perhaps to make snapshotting behind a feature flag that is enabled by default [patch in conda-forge has vv. incomplete start on this]
  • Modularize cli/build.rs and gate more things behind feature flags, so it doesn't depend on Deno & the runtime when it isn't snapshotting (otherwise cross-builds compile Deno twice, once for the build arch to support the snapshots that aren't created, and again for the target arch)

Relatedly: one of the challenges is on the things behind include_js_files_for_snapshotting, which causes binaries to include files by reference instead of including their source code. Would it be worth considering always including the sources? A rough count finds 18M of JavaScript & TypeScript source in the runtime, cli, and ext trees; that would be a substantial add. Brotli-compressing the whole bundle brings it down to 2.2M while separate Deflate compression yields a 4.3M zip, suggesting compressed versions of the source code could be unconditionally included with a modest increase in binary size.

@mdekstrand
Copy link

A further hitch has arisen: with Deno 1.39.0, snapshotting is more deeply baked into Deno's working, so the patches I currently have to work without snapshotting cause Deno.build to be incorrectly populated.

@littledivy
Copy link
Member Author

Deno.build fields are being populated during snapshot time (improving startup time perf):

#[derive(Serialize, Default)]
#[serde(rename_all = "camelCase")]
pub struct SnapshotOptions {
pub deno_version: String,
pub ts_version: String,
pub v8_version: &'static str,
pub target: String,
}
// Note: Called at snapshot time, op perf is not a concern.
#[op2]
#[serde]
pub fn op_snapshot_options(state: &mut OpState) -> SnapshotOptions {
state.take::<SnapshotOptions>()
}

which means its currently not possible to compile Deno CLI without snapshots.


Discussed this offline and will be working towards aarch64 linux builds with snapshots.

@mdekstrand
Copy link

mdekstrand commented Dec 22, 2023

Thanks — looking forward to official aarch64 builds!

It would be useful for packagers, however, to support building the Deno CLI without snapshots — packaging environments often require cross-builds. The conda-forge CI infrastructure, for example, only supports native builds for x86-64 (I co-maintain the package there).

@littledivy
Copy link
Member Author

Official aarch64 linux builds are available. Closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build system or continuous integration related suggestion suggestions for new features (yet to be agreed)
Projects
None yet
Development

No branches or pull requests

2 participants