Transient tasks #8764

sokra · 2024-07-16T08:32:12Z

Description

Track which tasks are transient and which are persistent. We need to track that for persistent caching.

We used a different task id space for transient tasks so they don't use up persistent ids and to easily identify transient tasks.

Testing Instructions

vercel · 2024-07-16T08:32:18Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
examples-nonmonorepo	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 26, 2024 6:53am
rust-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 26, 2024 6:53am

8 Skipped Deployments

Name	Status	Preview	Updated (UTC)
examples-basic-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-designsystem-docs	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-gatsby-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-kitchensink-blog	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-native-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-svelte-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-tailwind-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am
examples-vite-web	⬜️ Ignored (Inspect)	Visit Preview	Jul 26, 2024 6:53am

sokra · 2024-07-16T08:32:32Z

Transient tasks #8764 👈
remove ConcreteTaskInput in favor of the original rust datatypes #8737
Refactor task arguments to be a single one #8736
turbo-tasks this calls #8735
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @sokra and the rest of your teammates on Graphite

github-actions · 2024-07-16T08:34:15Z

🟢 Turbopack Benchmark CI successful 🟢

Thanks

github-actions · 2024-07-16T08:35:12Z

✅ This change can build next-swc

github-actions · 2024-07-16T10:50:24Z

⚠️ CI failed ⚠️

The following steps have failed in CI:

Turbopack Rust tests (mac/win, non-blocking)
Turbopack Rust benchmark tests (linux)

See workflow summary for details

sokra · 2024-07-24T16:11:54Z

crates/turbo-tasks/src/id.rs

@@ -77,6 +77,14 @@ impl Debug for TaskId {
    }
 }

+pub const TRANSIENT_TASK_BIT: u32 = 0x8000_0000;


We could go for 0xc000_0000 if we need more space for persistent tasks. That would work with the system, but I keep it in the middle for now

Rename PersistentTaskType -> CachedTaskType

bgw

I don't love that this duplicates a lot of functions and logic, but I'm okay with it, and there's probably not much more that can be done for that.

bgw · 2024-07-26T23:03:11Z

crates/turbo-tasks-memory/src/memory_backend.rs

@@ -292,6 +319,74 @@ impl MemoryBackend {
    pub fn task_statistics(&self) -> &TaskStatisticsApi {
        &self.task_statistics
    }
+
+    fn track_cache_hit(


Thanks for moving my garbage out into a separate function!

turbo-orchestrator bot added created-by: turbopack owned-by: turbopack labels Jul 16, 2024

This was referenced Jul 16, 2024

turbo-tasks this calls #8735

Merged

Refactor task arguments to be a single one #8736

Merged

sokra mentioned this pull request Jul 16, 2024

remove ConcreteTaskInput in favor of the original rust datatypes #8737

Merged

vercel bot deployed to Preview – examples-nonmonorepo July 16, 2024 08:32 View deployment

sokra changed the title ~~use different task ids for persistent and transient tasks~~ Transient tasks Jul 16, 2024

vercel bot deployed to Preview – rust-docs July 16, 2024 08:34 View deployment

sokra force-pushed the sokra/improve-task-input branch from 8f0000c to d13a797 Compare July 16, 2024 08:43

sokra force-pushed the sokra/transient-tasks branch from 5e4b8d7 to 3488490 Compare July 16, 2024 08:43

vercel bot deployed to Preview – examples-nonmonorepo July 16, 2024 08:44 View deployment

vercel bot deployed to Preview – rust-docs July 16, 2024 08:45 View deployment

sokra force-pushed the sokra/improve-task-input branch from d13a797 to f2892be Compare July 17, 2024 20:03

sokra force-pushed the sokra/transient-tasks branch from 3488490 to 8b2e404 Compare July 17, 2024 20:03

vercel bot deployed to Preview – examples-nonmonorepo July 17, 2024 20:04 View deployment

vercel bot deployed to Preview – rust-docs July 17, 2024 20:06 View deployment

sokra force-pushed the sokra/transient-tasks branch from 8b2e404 to 79ae4b8 Compare July 17, 2024 21:05

vercel bot deployed to Preview – examples-nonmonorepo July 17, 2024 21:05 View deployment

vercel bot had a problem deploying to Preview – rust-docs July 17, 2024 21:05 Failure

sokra force-pushed the sokra/improve-task-input branch from 6a2b023 to 4c95906 Compare July 17, 2024 21:08

sokra force-pushed the sokra/transient-tasks branch from 79ae4b8 to 84301f5 Compare July 17, 2024 21:09

vercel bot deployed to Preview – examples-nonmonorepo July 17, 2024 21:09 View deployment

vercel bot had a problem deploying to Preview – rust-docs July 17, 2024 21:09 Failure

sokra force-pushed the sokra/improve-task-input branch from 4c95906 to 47efbda Compare July 17, 2024 21:21

sokra force-pushed the sokra/transient-tasks branch from 84301f5 to d43625e Compare July 17, 2024 21:21

vercel bot deployed to Preview – examples-nonmonorepo July 17, 2024 23:46 View deployment

vercel bot deployed to Preview – rust-docs July 17, 2024 23:48 View deployment

sokra force-pushed the sokra/improve-task-input branch from 647470f to a97d97a Compare July 18, 2024 00:20

sokra force-pushed the sokra/transient-tasks branch from f0f3d2a to 6fdd5a5 Compare July 18, 2024 00:21

vercel bot deployed to Preview – examples-nonmonorepo July 18, 2024 00:21 View deployment

vercel bot deployed to Preview – rust-docs July 18, 2024 00:23 View deployment

Base automatically changed from sokra/improve-task-input to main July 18, 2024 00:35

sokra force-pushed the sokra/transient-tasks branch from 6fdd5a5 to 01eecb3 Compare July 18, 2024 06:43

vercel bot deployed to Preview – examples-nonmonorepo July 18, 2024 06:44 View deployment

vercel bot deployed to Preview – rust-docs July 18, 2024 06:46 View deployment

sokra force-pushed the sokra/transient-tasks branch from 01eecb3 to e3616fc Compare July 22, 2024 14:26

vercel bot deployed to Preview – examples-nonmonorepo July 22, 2024 14:26 View deployment

vercel bot deployed to Preview – rust-docs July 22, 2024 14:28 View deployment

sokra force-pushed the sokra/transient-tasks branch from e3616fc to e3e5fc7 Compare July 24, 2024 15:01

vercel bot deployed to Preview – examples-nonmonorepo July 24, 2024 15:01 View deployment

vercel bot deployed to Preview – rust-docs July 24, 2024 15:03 View deployment

sokra commented Jul 24, 2024

View reviewed changes

sokra requested a review from bgw July 24, 2024 16:18

sokra added 3 commits July 26, 2024 08:26

use different task ids for persistent and transient tasks

d09922a

Store transient tasks separately

1faed5f

Rename PersistentTaskType -> CachedTaskType

clippy

5422e84

sokra force-pushed the sokra/transient-tasks branch from e3e5fc7 to 5422e84 Compare July 26, 2024 06:48

vercel bot deployed to Preview – examples-nonmonorepo July 26, 2024 06:48 View deployment

remove unused code

5584336

vercel bot deployed to Preview – examples-nonmonorepo July 26, 2024 06:51 View deployment

vercel bot deployed to Preview – rust-docs July 26, 2024 06:53 View deployment

sokra marked this pull request as ready for review July 26, 2024 07:09

sokra requested a review from a team as a code owner July 26, 2024 07:09

bgw approved these changes Jul 26, 2024

View reviewed changes

sokra enabled auto-merge (squash) July 29, 2024 07:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transient tasks #8764

Transient tasks #8764

sokra commented Jul 16, 2024 •

edited

Loading

vercel bot commented Jul 16, 2024 •

edited

Loading

sokra commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

sokra Jul 24, 2024

bgw left a comment

bgw Jul 26, 2024

Transient tasks #8764

Are you sure you want to change the base?

Transient tasks #8764

Conversation

sokra commented Jul 16, 2024 • edited Loading

Description

Testing Instructions

vercel bot commented Jul 16, 2024 • edited Loading

sokra commented Jul 16, 2024 • edited Loading

github-actions bot commented Jul 16, 2024 • edited Loading

🟢 Turbopack Benchmark CI successful 🟢

github-actions bot commented Jul 16, 2024 • edited Loading

github-actions bot commented Jul 16, 2024 • edited Loading

⚠️ CI failed ⚠️

sokra Jul 24, 2024

Choose a reason for hiding this comment

bgw left a comment

Choose a reason for hiding this comment

bgw Jul 26, 2024

Choose a reason for hiding this comment

sokra commented Jul 16, 2024 •

edited

Loading

vercel bot commented Jul 16, 2024 •

edited

Loading

sokra commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading

github-actions bot commented Jul 16, 2024 •

edited

Loading