-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decouple serde
from its derive
crate
#6917
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! One comment below.
#6921 provides the |
That PR merged, so this can be rebased now. |
By not activating the `derive` feature on `serde`, the compilation speed can be improved by a lot. This is because `serde` can then compile in parallel to `serde_derive`, allowing it to finish compilation possibly even before `serde_derive`, unblocking all the crates waiting for `serde` to start compiling much sooner. As it turns out the main deciding factor for how long the compile time of a project is, is primarly determined by the depth of dependencies rather than the width. In other words, a crate's compile times aren't affected by how many crates it depends on, but rather by the longest chain of dependencies that it needs to wait on. In many cases `serde` is part of that long chain, as it is part of a long chain if the `derive` feature is active: `proc-macro2` compile build script > `proc-macro2` run build script > `proc-macro2` > `quote` > `syn` > `serde_derive` > `serde` > `serde_json` (or any crate that depends on serde) By decoupling it from `serde_derive`, the chain is shortened and compile times get much better. Check this issue for a deeper elaboration: serde-rs/serde#2584 For `wasmtime` I'm seeing a reduction from 24.75s to 22.45s when compiling in `release` mode. This is because wasmtime through `gimli` has a dependency on `indexmap` which can only start compiling when `serde` is finished, which you want to happen as early as possible so some of wasmtime's dependencies can start compiling. To measure the full effect, the dependencies can't by themselves activate the `derive` feature. I've upstreamed a patch for `fxprof-processed-profile` which was the only dependency that activated it for `wasmtime` (not yet published to crates.io). `wasmtime-cli` and co. may need patches for their dependencies to see a similar improvement.
3791b5d
to
22a1266
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
By not activating the `derive` feature on `serde`, the compilation speed can be improved by a lot. This is because `serde` can then compile in parallel to `serde_derive`, allowing it to finish compilation possibly even before `serde_derive`, unblocking all the crates waiting for `serde` to start compiling much sooner. As it turns out the main deciding factor for how long the compile time of a project is, is primarly determined by the depth of dependencies rather than the width. In other words, a crate's compile times aren't affected by how many crates it depends on, but rather by the longest chain of dependencies that it needs to wait on. In many cases `serde` is part of that long chain, as it is part of a long chain if the `derive` feature is active: `proc-macro2` compile build script > `proc-macro2` run build script > `proc-macro2` > `quote` > `syn` > `serde_derive` > `serde` > `serde_json` (or any crate that depends on serde) By decoupling it from `serde_derive`, the chain is shortened and compile times get much better. Check this issue for a deeper elaboration: serde-rs/serde#2584 For `wasmtime` I'm seeing a reduction from 24.75s to 22.45s when compiling in `release` mode. This is because wasmtime through `gimli` has a dependency on `indexmap` which can only start compiling when `serde` is finished, which you want to happen as early as possible so some of wasmtime's dependencies can start compiling. To measure the full effect, the dependencies can't by themselves activate the `derive` feature. I've upstreamed a patch for `fxprof-processed-profile` which was the only dependency that activated it for `wasmtime` (not yet published to crates.io). `wasmtime-cli` and co. may need patches for their dependencies to see a similar improvement.
I noticed recently that `wit-component` was relatively slow to compile and it's going to be an introductory requirement for any Rust crates so I wanted to optimize its build a little bit. This is the first optimization which is to disable the `derive` feature of the `serde` crate to enable build build pipelining opportunities. This is the same as bytecodealliance/wasmtime#6917
* Use `serde_derive` instead of the `derive` feature I noticed recently that `wit-component` was relatively slow to compile and it's going to be an introductory requirement for any Rust crates so I wanted to optimize its build a little bit. This is the first optimization which is to disable the `derive` feature of the `serde` crate to enable build build pipelining opportunities. This is the same as bytecodealliance/wasmtime#6917 * Fix build issue
By not activating the
derive
feature onserde
, the compilation speed can be improved by a lot. This is becauseserde
can then compile in parallel toserde_derive
, allowing it to finish compilation possibly even beforeserde_derive
, unblocking all the crates waiting forserde
to start compiling much sooner.As it turns out the main deciding factor for how long the compile time of a project is, is primarly determined by the depth of dependencies rather than the width. In other words, a crate's compile times aren't affected by how many crates it depends on, but rather by the longest chain of dependencies that it needs to wait on. In many cases
serde
is part of that long chain, as it is part of a long chain if thederive
feature is active:proc-macro2
compile build script >proc-macro2
run build script >proc-macro2
>quote
>syn
>serde_derive
>serde
>serde_json
(or any crate that depends on serde)By decoupling it from
serde_derive
, the chain is shortened and compile times get much better.Check this issue for a deeper elaboration:
serde-rs/serde#2584
For
wasmtime
I'm seeing a reduction from 24.75s to 22.45s when compiling inrelease
mode. This is because wasmtime throughgimli
has a dependency onindexmap
which can only start compiling whenserde
is finished, which you want to happen as early as possible so some of wasmtime's dependencies can start compiling.To measure the full effect, the dependencies can't by themselves activate the
derive
feature. I've upstreamed a patch forfxprof-processed-profile
which was the only dependency that activated it forwasmtime
(not yet published to crates.io).wasmtime-cli
and co. may need patches for their dependencies to see a similar improvement.