Implement Turbopack trace server bindings #65410

timneutkens · 2024-05-06T10:55:24Z

What?

Implements support for running the Turbopack trace server, which is the websocket server that powers https://turbo-trace-viewer.vercel.app/ when using NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1.

Currently you have to manually run the server through the Turbo repository which in practice means that only people working on Turbopack are able to run it.

With the bindings implemented anyone should be able to run the trace server.

Note that the traces that come out of Turbopack are very low level, they're meant for optimizing Turbopack like finding slowdowns / large memory usage / optimizing performance.

However, it's useful for people that want to peek into why their application is slow to compile. I.e. we've used https://turbo-trace-viewer.vercel.app to investigate reports in #48748.

This PR adds support for trace.log by default, so if you add NEXT_TURBOPACK_TRACING=1 NEXT_TURBOPACK_TRACE_SERVER=1 it will automatically select the trace.log for the current instance of Next.js. You can only have one trace server running at the same time.

`next internal`

In order to support running the trace server standalone, which is useful for investigating trace files other people have shared, I've added a new subcommand internal that is not covered by semver / use at your own risk. It's meant for internal tools that are useful to be bound to the version of Next.js, the turbo-trace-server is a great example of that as it has an internal binary format for storing data that needs to match the trace.log file.

If you want to take a look at .next/trace instead the new next internal subcommand can be used for that:

# Replace [path] with a path to a file.
next internal turbo-trace-server [path]

For example:

next internal turbo-trace-server ~/Downloads/trace

Currently the trace server does not support loading multiple files, just hasn't been implemented yet. Once we can load two or more files we can load both .next/trace and trace.log when NEXT_TURBOPACK_TRACE_SERVER=1 and support multiple paths passed to next internal turbo-trace-server.

Turbopack upgrade

PR includes a Turbopack upgrade:

Closes NEXT-3328

timneutkens · 2024-05-06T10:55:44Z

Implement Turbopack trace server bindings #65410 👈
canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @timneutkens and the rest of your teammates on Graphite

ijjk · 2024-05-06T11:05:59Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
buildDuration	16.6s	13.9s	N/A
buildDurationCached	7.7s	6.7s	N/A
nodeModulesSize	360 MB	360 MB	⚠️ +6.15 kB
nextStartRea..uration (ms)	410ms	407ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
1103-HASH.js gzip	31.9 kB	31.9 kB	N/A
1a9f679d-HASH.js gzip	53.5 kB	53.5 kB	N/A
335-HASH.js gzip	5.05 kB	5.05 kB	✓
7953.HASH.js gzip	169 B	169 B	✓
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	231 B	228 B	N/A
main-HASH.js gzip	31.6 kB	31.6 kB	N/A
webpack-HASH.js gzip	1.65 kB	1.65 kB	N/A
Overall change	50.4 kB	50.4 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
_app-HASH.js gzip	192 B	193 B	N/A
_error-HASH.js gzip	192 B	193 B	N/A
amp-HASH.js gzip	507 B	511 B	N/A
css-HASH.js gzip	341 B	343 B	N/A
dynamic-HASH.js gzip	2.52 kB	2.52 kB	✓
edge-ssr-HASH.js gzip	266 B	265 B	N/A
head-HASH.js gzip	362 B	365 B	N/A
hooks-HASH.js gzip	392 B	392 B	✓
image-HASH.js gzip	4.27 kB	4.27 kB	✓
index-HASH.js gzip	268 B	268 B	✓
link-HASH.js gzip	2.69 kB	2.7 kB	N/A
routerDirect..HASH.js gzip	329 B	328 B	N/A
script-HASH.js gzip	397 B	397 B	✓
withRouter-HASH.js gzip	324 B	324 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	8.27 kB	8.27 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
_buildManifest.js gzip	483 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
index.html gzip	527 B	526 B	N/A
link.html gzip	540 B	539 B	N/A
withRouter.html gzip	523 B	522 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
edge-ssr.js gzip	94.8 kB	94.8 kB	N/A
page.js gzip	182 kB	182 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
middleware-b..fest.js gzip	623 B	623 B	✓
middleware-r..fest.js gzip	156 B	156 B	✓
middleware.js gzip	25.7 kB	25.7 kB	N/A
edge-runtime..pack.js gzip	839 B	839 B	✓
Overall change	1.62 kB	1.62 kB	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
app-page-exp...dev.js gzip	171 kB	171 kB	✓
app-page-exp..prod.js gzip	98.8 kB	98.8 kB	✓
app-page-tur..prod.js gzip	100 kB	100 kB	✓
app-page-tur..prod.js gzip	94.6 kB	94.6 kB	✓
app-page.run...dev.js gzip	157 kB	157 kB	✓
app-page.run..prod.js gzip	93.3 kB	93.3 kB	✓
app-route-ex...dev.js gzip	21.5 kB	21.5 kB	✓
app-route-ex..prod.js gzip	15.3 kB	15.3 kB	✓
app-route-tu..prod.js gzip	15.3 kB	15.3 kB	✓
app-route-tu..prod.js gzip	15 kB	15 kB	✓
app-route.ru...dev.js gzip	21.3 kB	21.3 kB	✓
app-route.ru..prod.js gzip	15 kB	15 kB	✓
pages-api-tu..prod.js gzip	9.55 kB	9.55 kB	✓
pages-api.ru...dev.js gzip	9.82 kB	9.82 kB	✓
pages-api.ru..prod.js gzip	9.55 kB	9.55 kB	✓
pages-turbo...prod.js gzip	21.5 kB	21.5 kB	✓
pages.runtim...dev.js gzip	22.1 kB	22.1 kB	✓
pages.runtim..prod.js gzip	21.5 kB	21.5 kB	✓
server.runti..prod.js gzip	51.7 kB	51.7 kB	✓
Overall change	964 kB	964 kB	✓

build cache Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 05-06-Implement_Turbopack_trace_server_bindings	Change
0.pack gzip	1.62 MB	1.62 MB	N/A
index.pack gzip	113 kB	114 kB	⚠️ +417 B
Overall change	113 kB	114 kB	⚠️ +417 B

Diff details

Diff for edge-ssr.js

Diff too large to display

Commit: ab36b11

packages/next-swc/crates/napi/src/turbo_trace_server.rs

packages/next/src/server/dev/hot-reloader-turbopack.ts

…_TRACE_SERVER

packages/next-swc/crates/napi/src/turbo_trace_server.rs

This reverts commit b85210d.

This is failing to compile for some of our targets to reverting to unblock CI Reverts #65410 Closes NEXT-3331

Implement Turbopack trace server bindings

caa8acd

timneutkens requested review from ijjk, shuding, huozhi, a team, ztanner, feedthejim and wyattjoh as code owners May 6, 2024 10:55

ijjk added created-by: Next.js team PRs by the Next.js team Turbopack Related to Turbopack with Next.js. type: next labels May 6, 2024

sokra reviewed May 6, 2024

View reviewed changes

packages/next-swc/crates/napi/src/turbo_trace_server.rs Show resolved Hide resolved

sokra reviewed May 6, 2024

View reviewed changes

packages/next/src/server/dev/hot-reloader-turbopack.ts Outdated Show resolved Hide resolved

timneutkens added 2 commits May 6, 2024 14:46

Move trace server boot to project.rs, only enable with NEXT_TURBOPACK…

16ffaa8

…_TRACE_SERVER

Add "next internal turbo-trace-server" command to run other trace files

bf870e0

timneutkens requested a review from samcx as a code owner May 6, 2024 12:48

Update next.ts

48dc4d6

sokra reviewed May 6, 2024

View reviewed changes

packages/next-swc/crates/napi/src/turbo_trace_server.rs Outdated Show resolved Hide resolved

sokra reviewed May 6, 2024

View reviewed changes

packages/next-swc/crates/napi/src/turbo_trace_server.rs Outdated Show resolved Hide resolved

sokra approved these changes May 6, 2024

View reviewed changes

Remove inProcess option as it's not used

ab36b11

timneutkens enabled auto-merge (squash) May 6, 2024 15:09

timneutkens merged commit b85210d into canary May 6, 2024
77 of 81 checks passed

timneutkens deleted the 05-06-Implement_Turbopack_trace_server_bindings branch May 6, 2024 15:23

ijjk added a commit that referenced this pull request May 6, 2024

Revert "Implement Turbopack trace server bindings (#65410)"

ba6b671

This reverts commit b85210d.

ijjk mentioned this pull request May 6, 2024

Revert "Implement Turbopack trace server bindings" #65419

Merged

ijjk added a commit that referenced this pull request May 6, 2024

Revert "Implement Turbopack trace server bindings" (#65419)

aa51bed

This is failing to compile for some of our targets to reverting to unblock CI Reverts #65410 Closes NEXT-3331

github-actions bot added the locked label May 21, 2024

github-actions bot locked as resolved and limited conversation to collaborators May 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Turbopack trace server bindings #65410

Implement Turbopack trace server bindings #65410

timneutkens commented May 6, 2024 •

edited

timneutkens commented May 6, 2024

ijjk commented May 6, 2024 •

edited

Implement Turbopack trace server bindings #65410

Implement Turbopack trace server bindings #65410

Conversation

timneutkens commented May 6, 2024 • edited

What?

next internal

Turbopack upgrade

timneutkens commented May 6, 2024

ijjk commented May 6, 2024 • edited

Stats from current PR

timneutkens commented May 6, 2024 •

edited

`next internal`

ijjk commented May 6, 2024 •

edited