Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: make spaces requests in parallel #6536

Merged
merged 2 commits into from
Nov 21, 2023

Conversation

chris-olszewski
Copy link
Contributor

@chris-olszewski chris-olszewski commented Nov 21, 2023

Description

Realized that we were sending spaces requests serially and on a >>> FULL TURBO run this was very noticeable as sending the summaries took longer than restoring from cache.

Testing Instructions

EXPERIMENTAL_RUST_CODEPATH=true turbo_dev build --filter=docs --output-logs=none -vv
Before:

2023-11-21T11:57:59.278-0800 [DEBUG] turborepo_lib::run::summary::spaces: created run: SpaceRun { id: "space_run_xTnqFacHOWXEGCTV1lHGPhJR", url: "https://vercel.com/teams/vercel/repos/turbo-monorepo/runs/space_run_xTnqFacHOWXEGCTV1lHGPhJR" }
2023-11-21T11:57:59.278-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "docs#schema", name: "schema", workspace: "docs", hash: "46782ac6b6dc082a", start_time: 1700596678433, end_time: 1700596678441, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: ["@turbo/types#topo"], dependents: ["docs#build"], logs: "" }
2023-11-21T11:57:59.279-0800 [DEBUG] hyper::client::pool: reuse idle connection for ("https", vercel.com)
2023-11-21T11:57:59.280-0800 [DEBUG] h2::codec::framed_write: send
2023-11-21T11:57:59.280-0800 [DEBUG] h2::codec::framed_write: send
>   ...sending run summary...                                                                                                                
2023-11-21T11:58:00.120-0800 [DEBUG] h2::codec::framed_read: received
2023-11-21T11:58:00.120-0800 [DEBUG] h2::codec::framed_read: received
2023-11-21T11:58:00.120-0800 [DEBUG] h2::codec::framed_read: received
2023-11-21T11:58:00.121-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "docs#rss", name: "rss", workspace: "docs", hash: "67433a1dc94d54a2", start_time: 1700596678427, end_time: 1700596678441, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: [], dependents: ["docs#build"], logs: "" }
2023-11-21T11:58:00.121-0800 [DEBUG] hyper::client::pool: reuse idle connection for ("https", vercel.com)
2023-11-21T11:58:00.122-0800 [DEBUG] h2::codec::framed_write: send
2023-11-21T11:58:00.122-0800 [DEBUG] h2::codec::framed_write: send
>>  ...sending run summary...                                                                                                                
2023-11-21T11:58:00.732-0800 [DEBUG] h2::codec::framed_read: received
2023-11-21T11:58:00.732-0800 [DEBUG] h2::codec::framed_read: received
2023-11-21T11:58:00.732-0800 [DEBUG] h2::codec::framed_read: received
...

After:
See that all spaces requests get sent before previous ones finish:

...
2023-11-21T11:52:03.394-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "docs#schema", name: "schema", workspace: "docs", hash: "46782ac6b6dc082a", start_time: 1700596322692, end_time: 1700596322701, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: ["@turbo/types#topo"], dependents: ["docs#build"], logs: "" }
2023-11-21T11:52:03.395-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "docs#rss", name: "rss", workspace: "docs", hash: "67433a1dc94d54a2", start_time: 1700596322687, end_time: 1700596322701, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: [], dependents: ["docs#build"], logs: "" }
2023-11-21T11:52:03.395-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "@turbo/workspaces#build", name: "build", workspace: "@turbo/workspaces", hash: "75aa63b2eb51862b", start_time: 1700596322695, end_time: 1700596322701, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: ["@turbo/eslint-config#build", "@turbo/test-utils#build", "@turbo/tsconfig#build", "@turbo/utils#build"], dependents: ["@turbo/gen#build"], logs: "" }
2023-11-21T11:52:03.395-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "@turbo/gen#build", name: "build", workspace: "@turbo/gen", hash: "788339cd0cf19964", start_time: 1700596322703, end_time: 1700596322715, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: ["@turbo/eslint-config#build", "@turbo/test-utils#build", "@turbo/tsconfig#build", "@turbo/utils#build", "@turbo/workspaces#build"], dependents: ["docs#build"], logs: "" }
2023-11-21T11:52:03.396-0800 [DEBUG] turborepo_lib::run::summary::spaces: sending task: SpaceTaskSummary { key: "docs#build", name: "build", workspace: "docs", hash: "6c83c3a154c658be", start_time: 1700596322719, end_time: 1700596322732, cache: SpacesCacheStatus { status: Hit, source: Some(Local), time_saved: 0 }, exit_code: Some(0), dependencies: ["@turbo/eslint-config#build", "@turbo/gen#build", "@turbo/types#build", "docs#rss", "docs#schema"], dependents: [], logs: "" }
...

Closes TURBO-1731

Copy link

vercel bot commented Nov 21, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

11 Ignored Deployments
Name Status Preview Updated (UTC)
examples-basic-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-cra-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-designsystem-docs ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-gatsby-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-kitchensink-blog ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-native-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-nonmonorepo ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-svelte-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-tailwind-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
examples-vite-web ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm
turbo-site ⬜️ Ignored (Inspect) Nov 21, 2023 7:59pm

@chris-olszewski chris-olszewski marked this pull request as ready for review November 21, 2023 20:00
@chris-olszewski chris-olszewski requested a review from a team as a code owner November 21, 2023 20:00
Copy link
Contributor

🟢 Turbopack Benchmark CI successful 🟢

Thanks

Copy link
Contributor

⚠️ CI failed ⚠️

The following steps have failed in CI:

  • Turborepo integration tests

See workflow summary for details

@chris-olszewski chris-olszewski merged commit 94d4617 into main Nov 21, 2023
59 of 66 checks passed
@chris-olszewski chris-olszewski deleted the olszewski/fix_summary_serial branch November 21, 2023 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants