Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow CombinedContentSource to need data without locking up #3124

Merged
merged 12 commits into from Dec 23, 2022

Conversation

jridgewell
Copy link
Contributor

@jridgewell jridgewell commented Dec 22, 2022

Currently when an individual ContentSource inside a CombinedContentSource requests additional vary data, the combined source will treat that source as the winner of the query. But, sometimes the vary data is necessary to determine that the content is not found by this source. And if that's the case, we'll hit a 404 when we should actually keep iterating.

This is exactly the case we're going to hit with redirect/rewrites. Redirects need to be the very first content source, and I also need query/header/cookie/hostname data in order to determine if a redirect applies. Without this, every request becomes a 404.

@jridgewell jridgewell requested a review from a team as a code owner December 22, 2022 08:41
@vercel
Copy link

vercel bot commented Dec 22, 2022

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated
examples-basic-web 🔄 Building (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-cra-web 🔄 Building (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-designsystem-docs ❌ Failed (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-native-web 🔄 Building (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-svelte-web 🔄 Building (Inspect) Dec 23, 2022 at 3:16AM (UTC)
turbo-vite-web 🔄 Building (Inspect) Dec 23, 2022 at 3:16AM (UTC)
4 Ignored Deployments
Name Status Preview Comments Updated
examples-kitchensink-blog ⬜️ Ignored (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-nonmonorepo ⬜️ Ignored (Inspect) Dec 23, 2022 at 3:16AM (UTC)
examples-tailwind-web ⬜️ Ignored (Inspect) Dec 23, 2022 at 3:16AM (UTC)
turbo-site ⬜️ Ignored (Inspect) Visit Preview Dec 23, 2022 at 3:16AM (UTC)

@github-actions
Copy link
Contributor

github-actions bot commented Dec 22, 2022

🟢 CI successful 🟢

Thanks

@github-actions
Copy link
Contributor

Benchmark for 2ca490c

Test Base PR % Significant %
bench_hydration/Turbopack RCC/1000 modules 3440.64ms ± 6.40ms 3405.68ms ± 8.53ms -1.02% -0.15%
bench_startup/Turbopack CSR/1000 modules 1651.48ms ± 6.01ms 1677.23ms ± 6.64ms +1.56% +0.03%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8539.83µs ± 52.65µs 8582.74µs ± 68.10µs +0.50%
bench_hmr_to_commit/Turbopack RCC/1000 modules 8791.03µs ± 59.94µs 8750.47µs ± 80.52µs -0.46%
bench_hmr_to_commit/Turbopack RSC/1000 modules 482.88ms ± 1.22ms 482.13ms ± 4.29ms -0.16%
bench_hmr_to_commit/Turbopack SSR/1000 modules 8555.38µs ± 83.84µs 8597.68µs ± 65.54µs +0.49%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7516.55µs ± 46.78µs 7489.62µs ± 58.55µs -0.36%
bench_hmr_to_eval/Turbopack RCC/1000 modules 7626.48µs ± 44.25µs 7650.97µs ± 66.72µs +0.32%
bench_hmr_to_eval/Turbopack SSR/1000 modules 7509.72µs ± 42.37µs 7545.27µs ± 43.66µs +0.47%
bench_hydration/Turbopack RCC/1000 modules 3440.64ms ± 6.40ms 3405.68ms ± 8.53ms -1.02% -0.15%
bench_hydration/Turbopack RSC/1000 modules 2911.46ms ± 12.27ms 2899.84ms ± 11.42ms -0.40%
bench_hydration/Turbopack SSR/1000 modules 2696.89ms ± 12.98ms 2684.30ms ± 13.02ms -0.47%
bench_startup/Turbopack CSR/1000 modules 1651.48ms ± 6.01ms 1677.23ms ± 6.64ms +1.56% +0.03%
bench_startup/Turbopack RCC/1000 modules 2592.77ms ± 10.01ms 2566.64ms ± 10.70ms -1.01%
bench_startup/Turbopack RSC/1000 modules 2464.69ms ± 9.65ms 2452.69ms ± 11.57ms -0.49%
bench_startup/Turbopack SSR/1000 modules 2087.37ms ± 5.70ms 2111.33ms ± 9.67ms +1.15%


/// The paused state (partially processed path, content source, vary data)
/// of the internal content source which asked for vary data.
pending: Option<NeededData>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Since we don't actually use pending.vary, this should be its own PausableCombinedContentSourcePendingState struct with only a path and a source.

That way, you would not need the fiddling below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@jridgewell jridgewell added the pr: automerge Kodiak will merge these automatically after checks pass label Dec 22, 2022
@github-actions
Copy link
Contributor

Benchmark for afc9836

Click to view benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 8000.88µs ± 29.97µs 7916.60µs ± 57.18µs -1.05%
bench_hmr_to_commit/Turbopack RCC/1000 modules 8274.71µs ± 42.13µs 8206.24µs ± 44.72µs -0.83%
bench_hmr_to_commit/Turbopack RSC/1000 modules 467.64ms ± 1.99ms 469.68ms ± 2.47ms +0.44%
bench_hmr_to_commit/Turbopack SSR/1000 modules 8018.83µs ± 54.56µs 8082.00µs ± 66.54µs +0.79%
bench_hmr_to_eval/Turbopack CSR/1000 modules 7125.70µs ± 47.53µs 7032.29µs ± 55.53µs -1.31%
bench_hmr_to_eval/Turbopack RCC/1000 modules 7188.61µs ± 66.05µs 7199.52µs ± 49.66µs +0.15%
bench_hmr_to_eval/Turbopack SSR/1000 modules 7103.21µs ± 47.59µs 7141.77µs ± 42.74µs +0.54%
bench_hydration/Turbopack RCC/1000 modules 3300.16ms ± 12.19ms 3294.28ms ± 8.80ms -0.18%
bench_hydration/Turbopack RSC/1000 modules 2790.41ms ± 11.91ms 2766.94ms ± 8.29ms -0.84%
bench_hydration/Turbopack SSR/1000 modules 2610.70ms ± 8.77ms 2599.61ms ± 11.05ms -0.42%
bench_startup/Turbopack CSR/1000 modules 1615.35ms ± 5.79ms 1614.60ms ± 5.98ms -0.05%
bench_startup/Turbopack RCC/1000 modules 2475.47ms ± 10.87ms 2467.08ms ± 9.40ms -0.34%
bench_startup/Turbopack RSC/1000 modules 2374.05ms ± 8.50ms 2354.35ms ± 8.21ms -0.83%
bench_startup/Turbopack SSR/1000 modules 2025.46ms ± 3.54ms 2012.22ms ± 4.12ms -0.65%

@github-actions
Copy link
Contributor

Benchmark for a9d1ec0

Test Base PR % Significant %
bench_hydration/Turbopack RCC/1000 modules 3255.53ms ± 9.31ms 3214.13ms ± 6.78ms -1.27% -0.28%
Click to view full benchmark
Test Base PR % Significant %
bench_hmr_to_commit/Turbopack CSR/1000 modules 7811.07µs ± 49.38µs 7798.19µs ± 47.23µs -0.16%
bench_hmr_to_commit/Turbopack RCC/1000 modules 8249.71µs ± 57.06µs 8148.77µs ± 48.69µs -1.22%
bench_hmr_to_commit/Turbopack RSC/1000 modules 462.79ms ± 1.30ms 460.96ms ± 2.07ms -0.40%
bench_hmr_to_commit/Turbopack SSR/1000 modules 7881.85µs ± 63.74µs 7879.79µs ± 63.02µs -0.03%
bench_hmr_to_eval/Turbopack CSR/1000 modules 6969.20µs ± 44.60µs 6941.95µs ± 41.03µs -0.39%
bench_hmr_to_eval/Turbopack RCC/1000 modules 7092.37µs ± 71.20µs 7048.74µs ± 69.43µs -0.62%
bench_hmr_to_eval/Turbopack SSR/1000 modules 6909.58µs ± 41.74µs 6980.71µs ± 70.37µs +1.03%
bench_hydration/Turbopack RCC/1000 modules 3255.53ms ± 9.31ms 3214.13ms ± 6.78ms -1.27% -0.28%
bench_hydration/Turbopack RSC/1000 modules 2737.23ms ± 7.18ms 2733.25ms ± 6.78ms -0.15%
bench_hydration/Turbopack SSR/1000 modules 2585.80ms ± 9.86ms 2567.43ms ± 12.87ms -0.71%
bench_startup/Turbopack CSR/1000 modules 1591.18ms ± 2.58ms 1589.34ms ± 5.73ms -0.12%
bench_startup/Turbopack RCC/1000 modules 2441.49ms ± 8.25ms 2416.13ms ± 4.60ms -1.04%
bench_startup/Turbopack RSC/1000 modules 2336.21ms ± 9.43ms 2317.29ms ± 7.62ms -0.81%
bench_startup/Turbopack SSR/1000 modules 1997.29ms ± 5.88ms 1996.82ms ± 5.91ms -0.02%

jridgewell added a commit to vercel/next.js that referenced this pull request Mar 10, 2023
…urbo#3124)

Currently when an individual `ContentSource` inside a `CombinedContentSource` requests additional vary data, the combined source will treat that source as the winner of the query. But, sometimes the vary data is necessary to determine that the content is not found by this source. And if that's the case, we'll hit a 404 when we should actually keep iterating.

This is exactly the case we're going to hit with redirect/rewrites. Redirects need to be the very first content source, and I also need query/header/cookie/hostname data in order to determine if a redirect applies. Without this, **every** request becomes a 404.
sokra pushed a commit to vercel/next.js that referenced this pull request Mar 13, 2023
…urbo#3124)

Currently when an individual `ContentSource` inside a `CombinedContentSource` requests additional vary data, the combined source will treat that source as the winner of the query. But, sometimes the vary data is necessary to determine that the content is not found by this source. And if that's the case, we'll hit a 404 when we should actually keep iterating.

This is exactly the case we're going to hit with redirect/rewrites. Redirects need to be the very first content source, and I also need query/header/cookie/hostname data in order to determine if a redirect applies. Without this, **every** request becomes a 404.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: automerge Kodiak will merge these automatically after checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants