Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use a parcel_filesystem crate #9682

Merged
merged 2 commits into from
May 1, 2024
Merged

Use a parcel_filesystem crate #9682

merged 2 commits into from
May 1, 2024

Conversation

yamadapc
Copy link
Contributor

@yamadapc yamadapc commented May 1, 2024

  • Move the parcel_resolver::fs module into its own crate
  • Move our in-memory and JSDelegate implementations into that crate
  • Move the napi helper functions from node-bindings into another crate
  • Change all usages to use these functions

@yamadapc
Use a parcel_filesystem crate
44abeab 
* Move the `parcel_resolver::fs` module into its own crate
* Move our in-memory and JSDelegate implementations into that crate
* Move the napi helper functions from node-bindings into another crate
* Change all usages to use these functions
yamadapc
yamadapc commented May 1, 2024
View reviewed changes
crates/node-bindings/src/image.rs
@@ -32,7 +32,12 @@ pub fn optimize_image(kind: String, buf: Buffer, env: Env) -> Result<JsBuffer> {
match optimize_jpeg(slice) {
Ok(res) => Ok(
env
.create_buffer_with_borrowed_data(res.as_ptr(), res.len(), res.as_mut_ptr(), finalize)?
.create_buffer_with_borrowed_data(
res.as_mut_ptr(),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required due to napi bump

@yamadapc yamadapc enabled auto-merge May 1, 2024 02:58
@yamadapc yamadapc added this pull request to the merge queue May 1, 2024
Merged via the queue into v2 with commit 3227fc4 May 1, 2024
16 of 17 checks passed
@darknoon
Copy link

darknoon commented May 2, 2024

Did this break the REPL build?
I did a clean checkout of parcel and yarn build-repl was failing for me with something that seems related:

logs 
$ node scripts/build-native.js --release --wasm
Building packages/core/rust...
$ CARGO_PROFILE_RELEASE_LTO=true cargo build -p parcel-node-bindings --target wasm32-unknown-unknown --release && wasm-opt --strip-debug -O ../../../target/wasm32-unknown-unknown/release/parcel_node_bindings.wasm -o parcel_node_bindings.wasm
   Compiling proc-macro2 v1.0.79
   Compiling unicode-ident v1.0.5
   Compiling serde v1.0.198
   Compiling autocfg v1.1.0
   Compiling cfg-if v1.0.0
   Compiling libc v0.2.150
   Compiling version_check v0.9.4
   Compiling serde_json v1.0.116
   Compiling once_cell v1.19.0
   Compiling siphasher v0.3.10
   Compiling ppv-lite86 v0.2.17
   Compiling smallvec v1.11.1
   Compiling memchr v2.6.3
   Compiling regex-syntax v0.6.28
   Compiling itoa v1.0.4
   Compiling ryu v1.0.11
   Compiling either v1.11.0
   Compiling aho-corasick v0.7.20
   Compiling num-traits v0.2.15
   Compiling tinyvec_macros v0.1.0
   Compiling tinyvec v1.6.0
   Compiling ahash v0.8.11
   Compiling phf_shared v0.11.2
   Compiling getrandom v0.2.11
   Compiling zerocopy v0.7.32
   Compiling regex v1.7.0
   Compiling unicode-normalization v0.1.22
   Compiling new_debug_unreachable v1.0.4
   Compiling percent-encoding v2.3.0
   Compiling unicode-bidi v0.3.13
   Compiling form_urlencoded v1.2.0
   Compiling idna v0.4.0
   Compiling quote v1.0.35
   Compiling syn v2.0.53
   Compiling hashbrown v0.14.3
   Compiling lazy_static v1.4.0
   Compiling radium v0.7.0
   Compiling Inflector v0.11.4
   Compiling url v2.4.0
   Compiling num-integer v0.1.45
   Compiling outref v0.1.0
   Compiling rustc-hash v1.1.0
   Compiling rand_core v0.6.4
   Compiling tap v1.0.1
   Compiling uuid v1.5.0
   Compiling rand_chacha v0.3.1
   Compiling rand v0.8.5
   Compiling wyz v0.5.1
   Compiling simd-abstraction v0.7.1
   Compiling phf_generator v0.11.2
   Compiling num-bigint v0.4.3
   Compiling tracing-core v0.1.32
   Compiling funty v2.0.0
   Compiling data-encoding v2.3.3
   Compiling pin-project-lite v0.2.14
   Compiling bitflags v2.5.0
   Compiling if_chain v1.0.2
   Compiling base64-simd v0.7.0
   Compiling swc_macros_common v0.3.11
   Compiling unicode-id-start v1.1.2
   Compiling scoped-tls v1.0.1
   Compiling better_scoped_tls v0.1.1
   Compiling unicode-width v0.1.10
   Compiling unicode-id v0.3.3
   Compiling equivalent v1.0.1
   Compiling indexmap v2.1.0
   Compiling anyhow v1.0.82
   Compiling smartstring v1.0.1
   Compiling static_assertions v1.1.0
   Compiling typed-arena v2.0.2
   Compiling lock_api v0.4.12
   Compiling parking_lot_core v0.9.9
   Compiling serde_derive v1.0.198
   Compiling phf_macros v0.11.2
   Compiling phf v0.11.2
   Compiling tracing-attributes v0.1.27
   Compiling tracing v0.1.40
   Compiling bitvec v1.0.1
   Compiling from_variant v0.1.8
   Compiling swc_visit_macros v0.5.12
   Compiling ast_node v0.9.8
   Compiling swc_eq_ignore_macros v0.1.3
   Compiling is-macro v0.3.5
   Compiling swc_visit v0.5.14
   Compiling string_enum v0.4.4
   Compiling num_cpus v1.14.0
   Compiling scopeguard v1.1.0
   Compiling swc_trace_macro v0.1.3
   Compiling swc_ecma_transforms_macros v0.5.4
   Compiling dashmap v5.5.3
   Compiling hstr v0.2.6
   Compiling typenum v1.16.0
   Compiling swc_config_macro v0.1.3
   Compiling generic-array v0.14.6
   Compiling phf_shared v0.10.0
   Compiling arrayvec v0.7.4
   Compiling rustversion v1.0.14
   Compiling debugid v0.8.0
   Compiling sourcemap v8.0.1
   Compiling swc_atoms v0.6.5
   Compiling swc_cached v0.3.19
   Compiling phf_generator v0.10.0
   Compiling swc_config v0.1.10
   Compiling indexmap v1.9.3
   Compiling swc_common v0.33.26
   Compiling string_cache_codegen v0.5.2
   Compiling semver v1.0.20
   Compiling hashbrown v0.12.3
   Compiling thiserror v1.0.43
   Compiling browserslist-rs v0.14.0
   Compiling block-buffer v0.10.3
   Compiling crypto-common v0.1.6
   Compiling parking_lot v0.12.1
   Compiling thiserror-impl v1.0.43
   Compiling pmutil v0.6.1
   Compiling minimal-lexical v0.2.1
   Compiling vergen v8.2.6
   Compiling unicode-segmentation v1.10.0
   Compiling precomputed-hash v0.1.1
   Compiling string_cache v0.8.7
   Compiling convert_case v0.6.0
   Compiling nom v7.1.3
   Compiling static-map-macro v0.3.0
   Compiling digest v0.10.6
   Compiling chrono v0.4.31
   Compiling itertools v0.12.0
   Compiling fixedbitset v0.4.2
   Compiling petgraph v0.6.3
   Compiling sha-1 v0.10.0
   Compiling st-map v0.2.0
   Compiling napi-derive-backend v1.0.65
   Compiling base64 v0.21.5
   Compiling sourcemap v6.4.1
   Compiling napi-derive v2.16.3
   Compiling ctor v0.2.0
   Compiling swc_ecma_codegen_macros v0.7.4
   Compiling napi-sys v2.4.0
   Compiling ryu-js v1.0.0
   Compiling napi v2.16.4
   Compiling predicates-core v1.0.6
   Compiling serde_spanned v0.6.5
   Compiling toml_datetime v0.6.5
   Compiling itertools v0.11.0
   Compiling stable_deref_trait v1.2.0
   Compiling anstyle v1.0.4
   Compiling termtree v0.4.0
   Compiling napi-build v2.1.3
   Compiling winnow v0.6.6
   Compiling predicates v3.0.4
   Compiling parcel-node-bindings v0.1.0 (/Users/andrew/Developer/Web/_hacks/parcel/crates/node-bindings)
   Compiling predicates-tree v1.0.7
   Compiling toml_edit v0.22.12
   Compiling elsa v1.7.0
   Compiling serde_bytes v0.11.7
   Compiling mockall_derive v0.12.1
   Compiling itertools v0.10.5
   Compiling pathdiff v0.2.1
   Compiling downcast v0.11.0
   Compiling dunce v1.0.3
   Compiling xxhash-rust v0.8.7
   Compiling bitflags v1.3.2
   Compiling glob-match v0.2.1
   Compiling path-slash v0.1.5
   Compiling fragile v2.0.0
   Compiling indoc v1.0.7
   Compiling json_comments v0.2.1 (/Users/andrew/Developer/Web/_hacks/parcel/crates/json-comments-rs)
   Compiling toml v0.8.12
   Compiling mockall v0.12.1
   Compiling glob v0.3.1
   Compiling swc_core v0.89.6
   Compiling log v0.4.21
   Compiling swc_ecma_ast v0.111.1
   Compiling swc_fast_graph v0.21.15
   Compiling parcel_napi_helpers v0.1.0 (/Users/andrew/Developer/Web/_hacks/parcel/crates/parcel_napi_helpers)
   Compiling parcel_filesystem v0.1.0 (/Users/andrew/Developer/Web/_hacks/parcel/crates/parcel_filesystem)
error[E0432]: unresolved import `canonicalize::canonicalize`
  --> crates/parcel_filesystem/src/os_file_system/mod.rs:4:5
   |
4  | use canonicalize::canonicalize;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^ no `canonicalize` in `os_file_system::canonicalize`
   |
note: found an item that was configured out
  --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:10:8
   |
10 | pub fn canonicalize(
   |        ^^^^^^^^^^^^

warning: unused import: `std::path::Path`
 --> crates/parcel_filesystem/src/os_file_system/mod.rs:1:5
  |
1 | use std::path::Path;
  |     ^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

warning: unused import: `std::path::PathBuf`
 --> crates/parcel_filesystem/src/os_file_system/mod.rs:2:5
  |
2 | use std::path::PathBuf;
  |     ^^^^^^^^^^^^^^^^^^

warning: unused import: `dashmap::DashMap`
 --> crates/parcel_filesystem/src/os_file_system/mod.rs:5:5
  |
5 | use dashmap::DashMap;
  |     ^^^^^^^^^^^^^^^^

warning: unused import: `crate::FileSystem`
 --> crates/parcel_filesystem/src/os_file_system/mod.rs:7:5
  |
7 | use crate::FileSystem;
  |     ^^^^^^^^^^^^^^^^^

warning: unused import: `std::collections::VecDeque`
 --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:1:5
  |
1 | use std::collections::VecDeque;
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^

warning: unused import: `std::path::Component`
 --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:2:5
  |
2 | use std::path::Component;
  |     ^^^^^^^^^^^^^^^^^^^^

warning: unused import: `std::path::Path`
 --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:3:5
  |
3 | use std::path::Path;
  |     ^^^^^^^^^^^^^^^

warning: unused import: `std::path::PathBuf`
 --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:4:5
  |
4 | use std::path::PathBuf;
  |     ^^^^^^^^^^^^^^^^^^

warning: unused import: `dashmap::DashMap`
 --> crates/parcel_filesystem/src/os_file_system/canonicalize.rs:6:5
  |
6 | use dashmap::DashMap;
  |     ^^^^^^^^^^^^^^^^

For more information about this error, try `rustc --explain E0432`.
warning: `parcel_filesystem` (lib) generated 9 warnings
error: could not compile `parcel_filesystem` (lib) due to 1 previous error; 9 warnings emitted
warning: build failed, waiting for other jobs to finish...
error Command failed with exit code 101.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

When I tried rewinding further back, I ran into some other errors, ie on 0eaf926

…much log output…

$ yarn prepare && PARCEL_BUILD_REPL=1 parcel build src/index.html --no-source-maps
$ node write-commit.js
node:internal/modules/cjs/loader:1147
  throw err;
  ^

Error: Cannot find module '../lib/bin'

@mischnic
Copy link
Member

mischnic commented May 2, 2024

Probably yes 😞
There were many Rust changes recently and the REPL CI build was always broken because the Vercel publishing token is somehow invalid, so everyone ignored the REPL CI job.

@yamadapc
Copy link
Contributor Author

yamadapc commented May 2, 2024

Let me fix this, thank you for reporting

yamadapc added a commit that referenced this pull request May 2, 2024
@yamadapc
Fix wasm builds
5b3e782 
This fixes broken WASM builds since #9682 and previously added sentry changes.
@yamadapc yamadapc mentioned this pull request May 2, 2024
Merged
@yamadapc
Copy link
Contributor Author

yamadapc commented May 2, 2024

When I tried rewinding further back, I ran into some other errors, ie on 0eaf926

In order to work on the parcel tree you'll need to run lerna run dev:prepare. This will set-up a few symlinks so that parcel can self-build.

github-merge-queue bot pushed a commit that referenced this pull request May 6, 2024
@yamadapc
Fix wasm builds (#9688)
6c6b879 
This fixes broken WASM builds since #9682 and previously added sentry changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MonicaOlejniczak MonicaOlejniczak MonicaOlejniczak approved these changes

@mattcompiles mattcompiles Awaiting requested review from mattcompiles

@alshdavid alshdavid Awaiting requested review from alshdavid

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yamadapc @darknoon @mischnic @MonicaOlejniczak