Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow jobs to depend on jobs #69

Merged
merged 46 commits into from
Sep 30, 2022
Merged

allow jobs to depend on jobs #69

merged 46 commits into from
Sep 30, 2022

Conversation

BrianHicks
Copy link
Member

@BrianHicks BrianHicks commented Sep 24, 2022

Allows jobs to depend on other jobs, making a build graph! Finally! Yay!

Of course, job execution is still single-threaded, but now we can do more-or-less the same jobs as make --jobs 1! (Although we don't do the magic dynamic discovery thing that make does.)

Fixes #67

@BrianHicks BrianHicks requested a review from a team September 27, 2022 22:08
@BrianHicks
Copy link
Member Author

this is ready to review, despite that fact that macOS builds are failing. Gonna ask about that on Zulip later… maybe it's a nightly thing? I haven't really dug in yet, though.

@BrianHicks BrianHicks marked this pull request as ready for review September 27, 2022 22:08
src/coordinator.rs Outdated Show resolved Hide resolved
src/coordinator.rs Outdated Show resolved Hide resolved
src/coordinator.rs Outdated Show resolved Hide resolved
src/coordinator.rs Show resolved Hide resolved
src/coordinator.rs Show resolved Hide resolved
}

for key in self.input_jobs.keys().sorted() {
let dep = job_to_content_hash.get(key).context("could not look up output hash for dependency. This is a bug in rbt's coordinator. Please file it!")?.hash();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What hasher does this hash() use?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a getter for content-addressable store key, whose hashing has already been done. (It uses blake3 there.)

src/store.rs Show resolved Hide resolved
src/workspace.rs Show resolved Hide resolved
tests/end_to_end.rs Show resolved Hide resolved
tests/end_to_end/job_inputs/rbt.roc Show resolved Hide resolved
@BrianHicks BrianHicks mentioned this pull request Sep 30, 2022
@BrianHicks
Copy link
Member Author

@bhansconnect this is ready for another look! You were right that there were a few bugs. Luckily, they were easy to fix. 💯

@BrianHicks
Copy link
Member Author

wheeeee it's finally happening!

@BrianHicks BrianHicks merged commit 6d40a51 into main Sep 30, 2022
@BrianHicks BrianHicks deleted the job-job-ii branch September 30, 2022 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

jobs should be able to depend on other jobs
2 participants