Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Snapshot issues in next-dev-tests #3774

Merged
merged 36 commits into from Feb 27, 2023

Conversation

wbinnssmith
Copy link
Member

This:

  • Adds issue snapshot support to next-dev-tests. This will allow us to assert that certain issues are raised in tests that require next-dev.
  • Extracts common snapshot code into a new crates, turbopack-test-utils, which is shared between snapshot tests and next-dev-tests.
  • Implements an issue reporter that emits issues in a channel to the the integration test code, where they are snapshotted.
  • Fixes an issue where next-dev tests that were not Next.js apps would emit non-fatal issues for a missing pages/ directory.

Co-authored-by: Justin Ridgewell justin@ridgewell.name

This:
* Adds issue snapshot support to next-dev-tests. This will allow us to assert that certain issues are raised in tests that require next-dev.
* Extracts common snapshot code into a new crates, `turbopack-test-utils`, which is shared between snapshot tests and next-dev-tests.
* Implements an issue reporter that emits issues in a channel to the the integration test code, where they are snapshotted.
* Fixes an issue where next-dev tests that were not Next.js apps would emit non-fatal issues for a missing `pages/` directory.

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
@vercel
Copy link

vercel bot commented Feb 13, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
examples-basic-web 🔄 Building (Inspect) Feb 27, 2023 at 4:31PM (UTC)
examples-native-web 🔄 Building (Inspect) Feb 27, 2023 at 4:31PM (UTC)
examples-nonmonorepo 🔄 Building (Inspect) Feb 27, 2023 at 4:31PM (UTC)
7 Ignored Deployments
Name Status Preview Comments Updated
examples-cra-web ⬜️ Ignored (Inspect) Feb 27, 2023 at 4:31PM (UTC)
examples-designsystem-docs ⬜️ Ignored (Inspect) Feb 27, 2023 at 4:31PM (UTC)
examples-kitchensink-blog ⬜️ Ignored (Inspect) Visit Preview Feb 27, 2023 at 4:31PM (UTC)
examples-svelte-web ⬜️ Ignored (Inspect) Visit Preview Feb 27, 2023 at 4:31PM (UTC)
examples-tailwind-web ⬜️ Ignored (Inspect) Feb 27, 2023 at 4:31PM (UTC)
examples-vite-web ⬜️ Ignored (Inspect) Feb 27, 2023 at 4:31PM (UTC)
turbo-site ⬜️ Ignored (Inspect) Visit Preview Feb 27, 2023 at 4:31PM (UTC)

@github-actions
Copy link
Contributor

github-actions bot commented Feb 13, 2023

⚠️ CI failed ⚠️

The following steps have failed in CI:

  • Go e2e tests

See workflow summary for details

crates/next-core/src/router_source.rs Outdated Show resolved Hide resolved
crates/turbopack-core/src/issue/mod.rs Outdated Show resolved Hide resolved
crates/turbo-tasks/src/state.rs Show resolved Hide resolved
crates/turbopack-core/src/issue/mod.rs Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Benchmark for 97d1b38

Test Base PR % Significant %
bench_startup/Turbopack CSR/1000 modules 2795.49ms ± 7.22ms 2831.73ms ± 9.39ms +1.30% +0.11%
bench_startup/Turbopack SSR/1000 modules 2138.61ms ± 3.89ms 2171.83ms ± 4.72ms +1.55% +0.74%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 10.21ms ± 0.09ms 10.23ms ± 0.11ms +0.16%
bench_hmr_to_commit/Turbopack RCC/1000 modules 11.96ms ± 0.14ms 12.25ms ± 0.21ms +2.39%
bench_hmr_to_commit/Turbopack RSC/1000 modules 511.10ms ± 2.12ms 511.00ms ± 2.04ms -0.02%
bench_hmr_to_commit/Turbopack SSR/1000 modules 10.12ms ± 0.06ms 10.09ms ± 0.07ms -0.26%
bench_hmr_to_eval/Turbopack CSR/1000 modules 8949.34µs ± 81.77µs 8701.30µs ± 99.40µs -2.77%
bench_hmr_to_eval/Turbopack RCC/1000 modules 11.15ms ± 0.24ms 10.97ms ± 0.13ms -1.65%
bench_hmr_to_eval/Turbopack SSR/1000 modules 9004.75µs ± 105.46µs 8827.81µs ± 86.89µs -1.96%
bench_hydration/Turbopack RCC/1000 modules 4239.06ms ± 11.31ms 4254.56ms ± 14.77ms +0.37%
bench_hydration/Turbopack RSC/1000 modules 3865.53ms ± 18.44ms 3862.89ms ± 14.78ms -0.07%
bench_hydration/Turbopack SSR/1000 modules 3741.36ms ± 22.01ms 3722.93ms ± 9.63ms -0.49%
bench_startup/Turbopack CSR/1000 modules 2795.49ms ± 7.22ms 2831.73ms ± 9.39ms +1.30% +0.11%
bench_startup/Turbopack RCC/1000 modules 2584.44ms ± 5.07ms 2585.92ms ± 4.79ms +0.06%
bench_startup/Turbopack RSC/1000 modules 2483.70ms ± 6.29ms 2486.34ms ± 10.31ms +0.11%
bench_startup/Turbopack SSR/1000 modules 2138.61ms ± 3.89ms 2171.83ms ± 4.72ms +1.55% +0.74%

crates/next-core/src/next_config.rs Show resolved Hide resolved
crates/next-core/src/router_source.rs Outdated Show resolved Hide resolved
@github-actions
Copy link
Contributor

Benchmark for e40ae05

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 10.11ms ± 0.08ms 10.20ms ± 0.07ms +0.92%
bench_hmr_to_commit/Turbopack RCC/1000 modules 12.09ms ± 0.19ms 12.54ms ± 0.39ms +3.78%
bench_hmr_to_commit/Turbopack RSC/1000 modules 513.96ms ± 2.56ms 510.97ms ± 1.98ms -0.58%
bench_hmr_to_commit/Turbopack SSR/1000 modules 10.24ms ± 0.08ms 10.32ms ± 0.08ms +0.80%
bench_hmr_to_eval/Turbopack CSR/1000 modules 9139.56µs ± 93.00µs 9161.43µs ± 78.18µs +0.24%
bench_hmr_to_eval/Turbopack RCC/1000 modules 10.90ms ± 0.09ms 10.83ms ± 0.14ms -0.64%
bench_hmr_to_eval/Turbopack SSR/1000 modules 9223.44µs ± 60.37µs 9172.57µs ± 87.79µs -0.55%
bench_hydration/Turbopack RCC/1000 modules 4249.77ms ± 19.93ms 4208.71ms ± 6.88ms -0.97%
bench_hydration/Turbopack RSC/1000 modules 3875.72ms ± 18.12ms 3855.83ms ± 12.23ms -0.51%
bench_hydration/Turbopack SSR/1000 modules 3753.32ms ± 10.32ms 3740.14ms ± 16.57ms -0.35%
bench_startup/Turbopack CSR/1000 modules 2811.22ms ± 7.85ms 2796.25ms ± 6.62ms -0.53%
bench_startup/Turbopack RCC/1000 modules 2592.59ms ± 9.51ms 2582.98ms ± 6.59ms -0.37%
bench_startup/Turbopack RSC/1000 modules 2500.35ms ± 10.89ms 2482.34ms ± 8.84ms -0.72%
bench_startup/Turbopack SSR/1000 modules 2147.08ms ± 2.51ms 2147.20ms ± 2.18ms +0.01%

@nathanhammond nathanhammond removed their request for review February 17, 2023 04:33
@github-actions
Copy link
Contributor

Benchmark for 56ab7d9

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 12.58ms ± 0.09ms 12.54ms ± 0.14ms -0.25%
bench_hmr_to_commit/Turbopack RCC/1000 modules 17.02ms ± 0.22ms 16.95ms ± 0.18ms -0.40%
bench_hmr_to_commit/Turbopack RSC/1000 modules 515.80ms ± 2.17ms 513.50ms ± 1.42ms -0.45%
bench_hmr_to_commit/Turbopack SSR/1000 modules 12.58ms ± 0.07ms 12.70ms ± 0.11ms +0.92%
bench_hmr_to_eval/Turbopack CSR/1000 modules 11.57ms ± 0.08ms 11.58ms ± 0.12ms +0.08%
bench_hmr_to_eval/Turbopack RCC/1000 modules 15.12ms ± 0.27ms 14.84ms ± 0.24ms -1.82%
bench_hmr_to_eval/Turbopack SSR/1000 modules 11.56ms ± 0.13ms 11.56ms ± 0.08ms +0.06%
bench_hydration/Turbopack RCC/1000 modules 3918.53ms ± 13.69ms 3920.65ms ± 15.22ms +0.05%
bench_hydration/Turbopack RSC/1000 modules 3559.53ms ± 14.78ms 3514.86ms ± 11.88ms -1.25%
bench_hydration/Turbopack SSR/1000 modules 3423.31ms ± 9.57ms 3402.91ms ± 10.45ms -0.60%
bench_startup/Turbopack CSR/1000 modules 2608.53ms ± 8.59ms 2585.02ms ± 8.12ms -0.90%
bench_startup/Turbopack RCC/1000 modules 2374.69ms ± 6.55ms 2376.49ms ± 5.76ms +0.08%
bench_startup/Turbopack RSC/1000 modules 2366.74ms ± 5.90ms 2344.04ms ± 8.62ms -0.96%
bench_startup/Turbopack SSR/1000 modules 2084.54ms ± 2.88ms 2084.41ms ± 2.92ms -0.01%

@github-actions
Copy link
Contributor

Benchmark for ee092a6

Test Base PR % Significant %
bench_hydration/Turbopack SSR/1000 modules 3352.23ms ± 8.61ms 3312.25ms ± 9.34ms -1.19% -0.12%
bench_startup/Turbopack RCC/1000 modules 2334.23ms ± 4.12ms 2370.80ms ± 9.76ms +1.57% +0.38%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 11.54ms ± 0.09ms 11.47ms ± 0.07ms -0.63%
bench_hmr_to_commit/Turbopack RCC/1000 modules 15.52ms ± 0.29ms 15.43ms ± 0.19ms -0.59%
bench_hmr_to_commit/Turbopack RSC/1000 modules 506.26ms ± 0.99ms 502.31ms ± 2.15ms -0.78%
bench_hmr_to_commit/Turbopack SSR/1000 modules 11.53ms ± 0.09ms 11.49ms ± 0.10ms -0.34%
bench_hmr_to_eval/Turbopack CSR/1000 modules 10.66ms ± 0.10ms 10.56ms ± 0.09ms -0.94%
bench_hmr_to_eval/Turbopack RCC/1000 modules 14.41ms ± 0.14ms 14.35ms ± 0.13ms -0.41%
bench_hmr_to_eval/Turbopack SSR/1000 modules 10.50ms ± 0.10ms 10.58ms ± 0.09ms +0.76%
bench_hydration/Turbopack RCC/1000 modules 3818.90ms ± 12.48ms 3835.36ms ± 8.65ms +0.43%
bench_hydration/Turbopack RSC/1000 modules 3510.96ms ± 14.50ms 3497.42ms ± 13.49ms -0.39%
bench_hydration/Turbopack SSR/1000 modules 3352.23ms ± 8.61ms 3312.25ms ± 9.34ms -1.19% -0.12%
bench_startup/Turbopack CSR/1000 modules 2600.23ms ± 10.44ms 2585.34ms ± 7.48ms -0.57%
bench_startup/Turbopack RCC/1000 modules 2334.23ms ± 4.12ms 2370.80ms ± 9.76ms +1.57% +0.38%
bench_startup/Turbopack RSC/1000 modules 2304.64ms ± 11.11ms 2325.02ms ± 8.62ms +0.88%
bench_startup/Turbopack SSR/1000 modules 2079.69ms ± 2.99ms 2074.67ms ± 3.11ms -0.24%

@github-actions
Copy link
Contributor

Benchmark for 79950ae

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 12.60ms ± 0.11ms 12.59ms ± 0.04ms -0.10%
bench_hmr_to_commit/Turbopack RCC/1000 modules 15.36ms ± 0.18ms 15.32ms ± 0.17ms -0.26%
bench_hmr_to_commit/Turbopack RSC/1000 modules 531.84ms ± 1.03ms 536.20ms ± 1.29ms +0.82%
bench_hmr_to_commit/Turbopack SSR/1000 modules 12.70ms ± 0.10ms 12.81ms ± 0.10ms +0.84%
bench_hmr_to_eval/Turbopack CSR/1000 modules 11.67ms ± 0.09ms 11.75ms ± 0.08ms +0.71%
bench_hmr_to_eval/Turbopack RCC/1000 modules 14.00ms ± 0.13ms 13.96ms ± 0.18ms -0.27%
bench_hmr_to_eval/Turbopack SSR/1000 modules 11.54ms ± 0.08ms 11.57ms ± 0.16ms +0.20%
bench_hydration/Turbopack RCC/1000 modules 4062.22ms ± 16.13ms 4025.78ms ± 14.75ms -0.90%
bench_hydration/Turbopack RSC/1000 modules 3664.98ms ± 14.15ms 3663.08ms ± 20.97ms -0.05%
bench_hydration/Turbopack SSR/1000 modules 3498.02ms ± 7.60ms 3473.13ms ± 10.38ms -0.71%
bench_startup/Turbopack CSR/1000 modules 2706.26ms ± 7.71ms 2690.71ms ± 7.60ms -0.57%
bench_startup/Turbopack RCC/1000 modules 2494.08ms ± 10.58ms 2464.25ms ± 9.96ms -1.20%
bench_startup/Turbopack RSC/1000 modules 2425.98ms ± 15.06ms 2415.19ms ± 8.65ms -0.44%
bench_startup/Turbopack SSR/1000 modules 2167.02ms ± 7.48ms 2157.57ms ± 5.15ms -0.44%

@github-actions
Copy link
Contributor

Benchmark for 38b9cd0

Test Base PR % Significant %
bench_hmr_to_commit/Turbopack SSR/1000 modules 11.69ms ± 0.12ms 10.82ms ± 0.10ms -7.40% -3.72%
bench_hydration/Turbopack SSR/1000 modules 3398.17ms ± 12.00ms 3444.71ms ± 9.81ms +1.37% +0.09%
bench_startup/Turbopack RCC/1000 modules 2365.60ms ± 9.07ms 2335.42ms ± 3.56ms -1.28% -0.21%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 11.15ms ± 0.14ms 11.45ms ± 0.12ms +2.67%
bench_hmr_to_commit/Turbopack RCC/1000 modules 15.17ms ± 0.25ms 15.75ms ± 0.50ms +3.81%
bench_hmr_to_commit/Turbopack RSC/1000 modules 515.03ms ± 1.29ms 515.05ms ± 1.37ms +0.00%
bench_hmr_to_commit/Turbopack SSR/1000 modules 11.69ms ± 0.12ms 10.82ms ± 0.10ms -7.40% -3.72%
bench_hmr_to_eval/Turbopack CSR/1000 modules 10.82ms ± 0.25ms 11.37ms ± 0.09ms +5.10%
bench_hmr_to_eval/Turbopack RCC/1000 modules 13.89ms ± 0.23ms 13.76ms ± 0.15ms -0.98%
bench_hmr_to_eval/Turbopack SSR/1000 modules 10.17ms ± 0.16ms 10.58ms ± 0.24ms +4.01%
bench_hydration/Turbopack RCC/1000 modules 3882.36ms ± 14.75ms 3907.94ms ± 18.17ms +0.66%
bench_hydration/Turbopack RSC/1000 modules 3606.65ms ± 12.01ms 3566.80ms ± 13.36ms -1.10%
bench_hydration/Turbopack SSR/1000 modules 3398.17ms ± 12.00ms 3444.71ms ± 9.81ms +1.37% +0.09%
bench_startup/Turbopack CSR/1000 modules 2568.01ms ± 6.05ms 2559.02ms ± 8.45ms -0.35%
bench_startup/Turbopack RCC/1000 modules 2365.60ms ± 9.07ms 2335.42ms ± 3.56ms -1.28% -0.21%
bench_startup/Turbopack RSC/1000 modules 2282.71ms ± 7.81ms 2283.23ms ± 6.38ms +0.02%
bench_startup/Turbopack SSR/1000 modules 2064.96ms ± 4.72ms 2058.02ms ± 3.42ms -0.34%

@wbinnssmith wbinnssmith merged commit e4a4aa3 into main Feb 27, 2023
@wbinnssmith wbinnssmith deleted the wbinnssmith/snapshot-issues-next-dev-tests branch February 27, 2023 18:04
wbinnssmith added a commit that referenced this pull request Feb 27, 2023
This implements webpack loader context api for `emitError` and `emitWarning`, each of which emit Turbopack issues at the appropriate error level. As with webpack’s implementation, `emitError` does _not_ cause builds to fail [0].

This uses next-dev issue snapshots (#3774) to test that issues are emitted.

[0] https://webpack.js.org/api/loaders/#thisemiterror
ijjk pushed a commit to vercel/next.js that referenced this pull request Mar 1, 2023
wbinnssmith added a commit that referenced this pull request Mar 2, 2023
This implements webpack loader context api for `emitError` and
`emitWarning`, each of which emit Turbopack issues at the appropriate
error level. As with webpack’s implementation, `emitError` does _not_
cause builds to fail [0].

This uses next-dev issue snapshots (#3774) to test that issues are
emitted.

[0] https://webpack.js.org/api/loaders/#thisemiterror

---------

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
jridgewell added a commit to vercel/next.js that referenced this pull request Mar 10, 2023
This:
* Adds issue snapshot support to next-dev-tests. This will allow us to
assert that certain issues are raised in tests that require next-dev.
* Extracts common snapshot code into a new crates,
`turbopack-test-utils`, which is shared between snapshot tests and
next-dev-tests.
* Implements an issue reporter that emits issues in a channel to the the
integration test code, where they are snapshotted.
* Fixes an issue where next-dev tests that were not Next.js apps would
emit non-fatal issues for a missing `pages/` directory.

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>

---------

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
jridgewell added a commit to vercel/next.js that referenced this pull request Mar 10, 2023
…o#3983)

This implements webpack loader context api for `emitError` and
`emitWarning`, each of which emit Turbopack issues at the appropriate
error level. As with webpack’s implementation, `emitError` does _not_
cause builds to fail [0].

This uses next-dev issue snapshots (vercel/turbo#3774) to test that issues are
emitted.

[0] https://webpack.js.org/api/loaders/#thisemiterror

---------

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
sokra pushed a commit to vercel/next.js that referenced this pull request Mar 13, 2023
This:
* Adds issue snapshot support to next-dev-tests. This will allow us to
assert that certain issues are raised in tests that require next-dev.
* Extracts common snapshot code into a new crates,
`turbopack-test-utils`, which is shared between snapshot tests and
next-dev-tests.
* Implements an issue reporter that emits issues in a channel to the the
integration test code, where they are snapshotted.
* Fixes an issue where next-dev tests that were not Next.js apps would
emit non-fatal issues for a missing `pages/` directory.

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>

---------

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
sokra pushed a commit to vercel/next.js that referenced this pull request Mar 13, 2023
…o#3983)

This implements webpack loader context api for `emitError` and
`emitWarning`, each of which emit Turbopack issues at the appropriate
error level. As with webpack’s implementation, `emitError` does _not_
cause builds to fail [0].

This uses next-dev issue snapshots (vercel/turbo#3774) to test that issues are
emitted.

[0] https://webpack.js.org/api/loaders/#thisemiterror

---------

Co-authored-by: Justin Ridgewell <justin@ridgewell.name>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants