Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add garbage collection to jobs #586

Merged
merged 8 commits into from
Dec 18, 2015
Merged

Add garbage collection to jobs #586

merged 8 commits into from
Dec 18, 2015

Conversation

dadgar
Copy link
Contributor

@dadgar dadgar commented Dec 15, 2015

This PR adds garbage collection to jobs and marks batch jobs as eligible for garbage collection

@diptanu
Copy link
Contributor

diptanu commented Dec 15, 2015

Some thoughts about the design -

a. I am wondering why we have exposed the gc duration configurable for an end user of Nomad. GCing jobs out of Nomad when they are no longer going to be referenced by the system doesn't feel like something an end user needs to configure via the job defn.

b. We could just GC any job which won't be referenced again in the system. Batch Jobs by definition feels GC-able. Service jobs which has count > 1 and num of Tasks > 1 should never be GC-able until the moment the user stops the job. Setting count = 0 or num Tasks = 0 is almost same as the job getting stopped. System jobs, the way they are currently implemented shouldn't be GC-ed as well.

c. Periodic Jobs are interesting - If a periodic job won't be invoked again to submit a child jobs because of the configuration of it's spec(which seems highly unlikely), it's a candidate for being GC-ed, otherwise there shouldn't be any need for GC-ing it.

d. Any of the jobs - periodic, batch or service jobs which are stopped can immediately be GCed.

I think (b) and (c) just explains that when a job can be GC-ed from the system could be inferred by the state of it's current allocations and whether any future evaluations would be created for that job without needing any additional configuration from the end user whose job is running on Nomad.

@dadgar
Copy link
Contributor Author

dadgar commented Dec 15, 2015

@diptanu: Updated to address comments.

c.srv.logger.Printf("[DEBUG] sched.core: job GC: scanning before index %d (%v)",
oldThreshold, c.srv.config.JobGCThreshold)

// Collect the allocations and evaluations to GC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

allocations, evaluations and jobs

@diptanu
Copy link
Contributor

diptanu commented Dec 16, 2015

LGTM overall.

dadgar added a commit that referenced this pull request Dec 18, 2015
Add garbage collection to jobs
@dadgar dadgar merged commit 9daeae7 into master Dec 18, 2015
@dadgar dadgar deleted the f-job-gc branch December 18, 2015 19:21
@github-actions
Copy link

github-actions bot commented May 1, 2023

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 1, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants