[pull] canary from vercel:canary#1053
Merged
Merged
Conversation
## What? Converts more tests that use `createNext` to `nextTestSetup` Follow-up to #93767 --------- Co-authored-by: Cursor <cursoragent@cursor.com>
…l.cssChunking: "graph" (#93606) ### What? Adds an alternative CSS chunking algorithm to Turbopack, opted into via: ```js // next.config.js module.exports = { experimental: { cssChunking: 'graph', // or, with explicit cost overrides: // cssChunking: { type: 'graph', requestCost: 20_000, moduleFactorCost: 1 }, }, } ``` The new algorithm is **off by default** — Turbopack still uses the existing "loose"/dependencies algorithm unless this flag is set, so this PR is a pure addition for users that don't opt in. While we were here, the `experimental.cssChunking` shape was also generalized so every existing string accepts an object form too: | Value | Bundler | Notes | |---|---|---| | `true` / `'loose'` / `{ type: 'loose' }` | both | default heuristic-based chunking | | `'strict'` / `{ type: 'strict' }` | webpack | unchanged | | `false` | webpack | unchanged (one chunk per CSS module) | | `'graph'` / `{ type: 'graph', requestCost?, moduleFactorCost? }` | Turbopack | new | Cross-bundler combinations are rejected at config-validation time: - `'graph'` with webpack throws. - `'strict'` and `false` with Turbopack throw. ### Why? The existing Turbopack CSS chunker (loose / dependencies) is good at preserving CSS ordering but doesn't share chunks across pages well — every page tends to load its own chunk per CSS module, which scales poorly for apps with many pages and shared component libraries. The new "graph" algorithm models the per-chunk-group CSS ordering as a weighted DAG over modules, then greedily merges adjacent runs in the global topological order whenever the merge reduces total cost. The cost model charges every CSS request and overshipped byte, with two tunable knobs (`requestCost` and `moduleFactorCost`). **Trade-off vs. the loose default.** With the default cost parameters (`requestCost: 20_000`, `moduleFactorCost: 1`) the graph algorithm typically ships **less CSS per chunk group at the cost of more requests** than the loose algorithm. The cost model is tuned to avoid overshipping unrelated CSS into pages that don't need it; on apps where the loose algorithm was already collapsing a lot into one big chunk that some pages didn't actually use, the graph algorithm will split it. Apps that prefer fewer requests can raise `requestCost`; apps that prefer less overshipping can raise `moduleFactorCost`. This is opt-in and Turbopack-only because: - The cost model is sensitive to per-app properties (number of pages, size distribution of CSS modules, …) — keeping it experimental gives us room to tune defaults from real usage. - Webpack already has its own `CssChunkingPlugin` and `'strict'` mode that cover the equivalent design space; we don't want to fork that. ### Performance Measured on `vercel.com` (the full graph algorithm spans `create_graph → make_acyclic → linearize → split_into_chunks → assemble`): - **~3s** end-to-end for the synchronous chunking pipeline on a realistic production input. Implementation choices that matter for that throughput: - Tarjan SCC uses `Vec<u32>` / `Vec<bool>` scratch arrays indexed by `NodeIndex` — no hashing on `indices` / `lowlinks` / `on_stack`. - `make_acyclic` batches multiple cuts per SCC pass by seeding successive short-cycle searches at the previous cut's target, only re-running Tarjan when no further cycle is reachable from the seed. - `find_short_cycle` is a bidirectional Dijkstra over a `BinaryHeap` with predecessor pointers (no path cloning) and skips its refinement loop for trivial 2-cycles. - `split_into_chunks` picks the next merge from a `BinaryHeap` keyed on the cost delta instead of an O(N) linear scan per merge. - `chunk_cost` reads a once-built `module_to_groups` inverse index instead of scanning every chunk group on every call; the GlobalStyle leakage check uses binary search on the inverse index rather than scanning each group's module list. ### How? #### Module layout (`turbopack/crates/turbopack-core/src/module_graph/`) The two algorithms are deliberately split so neither imports from the other: - `style_groups/` — algorithm-neutral output types (`StyleGroups`, `StyleItemInfo`, `make_style_groups`). Both algorithms produce these. - `style_groups_loose/` — the existing ("loose") algorithm plus the shared config types (`StyleGroupsAlgorithm`, `StyleGroupsConfig`, `F32TaskInput`). - `style_groups_graph/` — the new algorithm. Pure Rust, no `Vc`, with `petgraph::DiGraph` plus a thin `SubgraphView` wrapper and a small `ReadonlyGraph` trait that lets the same pipeline run against either a `&DiGraph` or a filtered view of one SCC. #### Algorithm ```text create_graph → make_acyclic → linearize → split_into_chunks → assemble batches ``` 1. **`create_graph`** — for each chunk group, every `(later, earlier)` pair inside the group's CSS-module list becomes an edge `later → earlier` (weight 1, accumulated). Heavy edges = strong co-occurrence. 2. **`make_acyclic`** — co-occurrence almost always introduces cycles; each multi-node SCC has its lowest-weight cycle edge cut until the graph is a DAG. 3. **`linearize`** — Kahn-style topological sort with a tie-break on edge weight, so strongly co-occurring modules end up adjacent in the global order. 4. **`split_into_chunks`** — greedy bottom-up merger over the global order. At every active split point we score the merge as `cost(merged) - cost(left) - cost(right)`, take the most-negative score from a min-heap, and repeat until no merge would reduce cost. `max_chunk_size` and "global CSS must not leak into unrelated chunk groups" are enforced as `+infinity` cost. The cost model is: ```text cost_per_group(chunk, group) = chunk_size + (chunk_size / group_total_size) * module_factor_cost + request_cost ``` summed over the chunk groups that load the chunk. #### Wiring - `StyleGroups::shared_chunk_items` is a `FxIndexMap<ChunkItemWithAsyncModuleInfo, StyleItemInfo>` where `StyleItemInfo { order: Option<u32>, batch: Option<…> }`. The graph algorithm fills `order` so `style_production.rs` can stable-sort chunks globally; the legacy algorithm leaves `order = None`, which makes the sort a no-op for it. `flatten_and_sort` returns the `StyleItemInfo` references alongside each chunk item so the per-item loop doesn't re-query the map. - A new `StyleGroupsAlgorithm` enum on `ChunkingConfig` selects the algorithm at chunking time; `ModuleGraph::style_groups` dispatches to either `compute_style_groups` (existing) or `compute_style_groups_graph` (new). - `next-core` exposes `NextConfig::css_chunking() -> Vc<CssChunkingAlgorithm>` resolving the JS `experimental.cssChunking` to the Rust enum, with cost defaults applied (`requestCost: 20_000`, `moduleFactorCost: 1`). All three chunking-context constructors (`next_client`, `next_edge`, `next_server`) thread it through. #### Configuration - `experimental.cssChunking` zod schema accepts the new shapes; cost params are `z.number().nonnegative().finite().optional()`. - `config-shared.ts` exports a `CssChunkingConfig` type alias and a `resolveCssChunkingMode(value)` helper that normalizes any input to one of `'off' | 'loose' | 'strict' | 'graph'`. Both `webpack-config.ts` (plugin wiring) and `config.ts` (bundler-compat validation) use the helper. - New `errors.json` entries for the three bundler-compatibility validation errors (E1193 graph-on-webpack, E1194 strict-on-Turbopack, E1195 false-on-Turbopack). #### Tests - 53 Rust unit tests in `style_groups_graph/tests.rs` cover `create_graph`, Tarjan SCC, `find_short_cycle` (bidirectional Dijkstra), `make_acyclic`, `linearize`, `split_into_chunks`, and end-to-end pipeline scenarios. - `test/e2e/app-dir/css-order/css-order.test.ts` is parametrised over `[label, value]` pairs. The Turbopack matrix now includes `'graph'` and an object-form `{ type: 'graph', requestCost: 1, moduleFactorCost: 1 }` in addition to the existing default. Per-page expectations grew a `requests` object encoding distinct request counts for `loose` and `graph` where they differ. - A new `sandwich` e2e fixture (`/sandwich/a`, `/sandwich/b`) exercises the case where two pages share a leading and trailing chunk around a unique middle stylesheet — including a global stylesheet that the algorithm must not leak into unrelated chunk groups. The graph algorithm hits the optimal 3 chunks per page on this fixture; loose mode falls short. #### Documentation - `ExperimentalConfig.cssChunking` JSDoc describes every accepted shape and what each cost knob does. - The `style_groups_graph` module-level docs describe the pipeline, cost model and constraints with diagrams. Closes NEXT- <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>
### What?
Reworks `emit_assets` in `crates/next-core/src/emit.rs` so that:
- The actual `emit()` / `emit_rebase()` write runs **before**
`check_duplicates()`.
- Per-duplicate diff checks inside `check_duplicates` run in parallel.
- At most one `EmitConflictIssue` is emitted per asset path, even when
multiple duplicates conflict.
- The conflict-issue emission is wrapped in a `turbo_tasks::function`
and reported via `as_side_effect()`.
### Why?
Under eventual consistency, all children might be removed from
`emit_assets`. This causes all `content()` and `emit()` tasks being
inactive and keeping the stale value (which is potentially an error).
On recomputation of `emit_assets()` this only makes very few `content()`
tasks active, as tasks are still in a stale error state and make
`emit_assets()` exit early. So it takes N recomputations of
`emit_assets()` to be up-to-date, where N is the number of duplicate
assets. This can slow down builds drastically and did cause build time
up to 30min.
Emitting the asset first ensures the file is written to disk even if the
subsequent duplicate diagnostic crashes under eventual consistency.
Doing the diff checks in parallel and folding conflicts into a single
issue keeps the diagnostic path cheap and avoids noisy duplicate output
for the same path. Tracking the issue emission as a turbo-tasks side
effect means it is properly attributed to a task instead of being
emitted ad-hoc from `emit_assets`.
### How?
- Swap the order in both branches of `emit_assets` (server-side `emit`
and client-side `emit_rebase`) so the write happens first, then
`check_duplicates` runs as a diagnostic afterwards. A comment is left
explaining the ordering requirement.
- `check_duplicates` now returns `Result<()>` instead of returning the
picked asset. The caller picks `assets.first()` directly, since
`check_duplicates` only validated equivalence — it didn't choose a
different asset.
- The inner loop in `check_duplicates` uses `iter.map(async |next| { ...
}).try_flat_join().await?` to run the diff comparisons in parallel and
collect conflict details. `ext` is computed once before the loop and
cloned into each task.
- Only the first conflict from the collected list is reported, instead
of one issue per conflicting asset.
- The `EmitConflictIssue` emission is moved into a local
`#[turbo_tasks::function] fn emit_conflict_issue(...)` invoked via
`.as_side_effect().await?`, so the issue is tracked as a task side
effect.
<!-- NEXT_JS_LLM_PR -->
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#93915) The fixture's `next.config.js` had `output: 'standalone'` hardcoded, which is meaningless on Vercel deployments (the standalone directory is ignored) but became a build-time crash after #93684 made the Turbopack NFT generator skip `next-server.js.nft.json` whenever an adapter is configured. With `output: 'standalone'` still active, `writeStandaloneDirectory` then fails with `ENOENT` when it tries to read that file ([x-ref](https://github.com/vercel/next.js/actions/runs/26006483168/job/76441905221)). This PR splits the `og-api` E2E suite into a default variant and a `standalone.test.ts` wrapper that enables `output: 'standalone'` via the `TEST_OUTPUT_STANDALONE` env var, and excludes the standalone wrapper from the deploy test manifest. The default variant still runs in `deploy` mode and preserves coverage of `next/og` from Pages Router API routes, App Router route handlers (edge and node), and middleware — paths not covered by the metadata-* fixtures. The standalone variant continues to exercise the trace-copy assertion in `next start` mode. --------- Co-authored-by: Tim Neutkens <tim@timneutkens.nl>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )