Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JENKINS-36479 - clean up hard killed/deleted builds' locks #34

Closed
wants to merge 5 commits into from

Conversation

abayer
Copy link
Member

@abayer abayer commented Jul 6, 2016

JENKINS-36479

  • Successfully clear locks from hard killed/deleted builds on next lock request.
  • Write tests for both hard kill and deleted build scenarios.
  • Decide whether this is sufficient or if we need to automatically clear those locks without needing a new lock request.

cc @reviewbybees, esp @jglick and @amuniz

When we try to lock, check if the resource is locked by either a null
build or a build that's no longer running. If so, unlock the resource
before continuing.

This probably needs a *lot* more work. It's just a first thought.
Note - test needs to be reworked to actually fail if it can't get the
lock. Right now it'll hang forever. I'll figure that out!
JenkinsRule tests time out after 180 seconds anyway, it turns
out. Would prefer to have a shorter timeout than that, but it works,
so hey.
@abayer
Copy link
Member Author

abayer commented Jul 6, 2016

Oh, and I've verified that the tests fail against master, so that's nice. =)

&& (lockingRun == null
|| !lockingRun.isBuilding()
|| lockingRun.getParent().getBuild(lockingRun.getId()) == null)) {
performUnlocking(r, lockingRun, inversePrecedence);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably at least print a warning in this build’s log.

@jglick
Copy link
Member

jglick commented Jul 6, 2016

I think you could use RunListener to immediately unblock waiting builds.

@abayer
Copy link
Member Author

abayer commented Jul 6, 2016

I tried some experiments with RunListener, and, well, as far as I could tell, it's not firing for deleted-in-progress builds. More experiments ahoy!

@abayer
Copy link
Member Author

abayer commented Jul 6, 2016

Oh, yeah, it's not because AbstractBuild.

@abayer
Copy link
Member Author

abayer commented Jul 6, 2016

Hmm - looks like changing all the AbstractBuild and AbstractProject references to Run and Job may do the trick without my change...

@jglick
Copy link
Member

jglick commented Jul 6, 2016

Huh? This code should fire for any Run.

@jglick
Copy link
Member

jglick commented Jul 6, 2016

Oh you mean LockRunListener. Yeah fix that. It might need to have different cases for AbstractBuild vs. other Run, @amuniz would know.

@abayer
Copy link
Member Author

abayer commented Jul 6, 2016

Gonna do an alternate pr for that approach and leave this open for the moment.

WorkflowRun b3 = p3.scheduleBuild2(0).waitForStart();

// Verify that b2 gets the lock.
story.j.waitForMessage("Lock acquired on [resource1]", b2);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, b2 is getting the lock because b3 release it before, right? Weird that a new build is required to unlock builds already waiting for the resource. Acceptable I guess, but weird.

@abayer
Copy link
Member Author

abayer commented Jul 7, 2016

Closing in favor of #35

@abayer abayer closed this Jul 7, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants