Skip to content

[JENKINS-31768] dead lock while removing computer #21606

Closed

Description

@jenkins-infra-bot
"Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]":
  waiting for ownable synchronizer 0x00007f3a5d901708, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
  which is held by "Computer.threadPoolForRemoting [#31767]"
"Computer.threadPoolForRemoting [#31767]":
  waiting to lock monitor 0x00007f38ce077bc8 (object 0x00007f3a7c8a10c8, a hudson.plugins.ec2.EC2Computer),
  which is held by "Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]"

"Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  <0x00007f3a5d901708> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at hudson.model.Queue._withLock(Queue.java:1291)
at hudson.model.Queue.withLock(Queue.java:1176)
at hudson.model.Computer.setNumExecutors(Computer.java:837)
- locked <0x00007f3a7c8a10c8> (a hudson.plugins.ec2.EC2Computer)
at hudson.model.Computer.kill(Computer.java:792)
at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:665)
at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:86)
at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:205)
at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1249)
at jenkins.model.Jenkins.doConfigSubmit(Jenkins.java:2924)
- locked <0x00007f3a5c073f20> (a hudson.model.Hudson)

"Computer.threadPoolForRemoting [#31767]":
at hudson.model.Computer.setNumExecutors(Computer.java:830)
- waiting to lock <0x00007f3a7c8a10c8> (a hudson.plugins.ec2.EC2Computer)
at hudson.model.Computer.inflictMortalWound(Computer.java:809)
at hudson.model.AbstractCIBase$2.run(AbstractCIBase.java:199)
at hudson.model.Queue._withLock(Queue.java:1293)
at hudson.model.Queue.withLock(Queue.java:1176)
at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:169)
at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1249)
at jenkins.model.Nodes$2.run(Nodes.java:136)
at hudson.model.Queue._withLock(Queue.java:1293)
at hudson.model.Queue.withLock(Queue.java:1176)
at jenkins.model.Nodes.addNode(Nodes.java:132)
at jenkins.model.Jenkins.addNode(Jenkins.java:1744)


Originally reported by fengxx, imported from: dead lock while removing computer
  • status: Resolved
  • priority: Critical
  • component(s): core
  • label(s): 2.19.4-fixed
  • resolution: Fixed
  • resolved: 2016-10-17T08:58:16+00:00
  • votes: 0
  • watchers: 3
  • imported: 2025-11-24
Raw content of original issue
"Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]":
  waiting for ownable synchronizer 0x00007f3a5d901708, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
  which is held by "Computer.threadPoolForRemoting [#31767]"
"Computer.threadPoolForRemoting [#31767]":
  waiting to lock monitor 0x00007f38ce077bc8 (object 0x00007f3a7c8a10c8, a hudson.plugins.ec2.EC2Computer),
  which is held by "Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]"

"Handling POST /configSubmit from 10.160.255.74 : RequestHandlerThread[#17137]":
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00007f3a5d901708> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
        at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
        at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
        at hudson.model.Queue._withLock(Queue.java:1291)
        at hudson.model.Queue.withLock(Queue.java:1176)
        at hudson.model.Computer.setNumExecutors(Computer.java:837)
        - locked <0x00007f3a7c8a10c8> (a hudson.plugins.ec2.EC2Computer)
        at hudson.model.Computer.kill(Computer.java:792)
        at hudson.slaves.SlaveComputer.kill(SlaveComputer.java:665)
        at hudson.model.AbstractCIBase.killComputer(AbstractCIBase.java:86)
        at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:205)
        at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1249)
        at jenkins.model.Jenkins.doConfigSubmit(Jenkins.java:2924)
        - locked <0x00007f3a5c073f20> (a hudson.model.Hudson)

"Computer.threadPoolForRemoting [#31767]":
        at hudson.model.Computer.setNumExecutors(Computer.java:830)
        - waiting to lock <0x00007f3a7c8a10c8> (a hudson.plugins.ec2.EC2Computer)
        at hudson.model.Computer.inflictMortalWound(Computer.java:809)
        at hudson.model.AbstractCIBase$2.run(AbstractCIBase.java:199)
        at hudson.model.Queue._withLock(Queue.java:1293)
        at hudson.model.Queue.withLock(Queue.java:1176)
        at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:169)
        at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1249)
        at jenkins.model.Nodes$2.run(Nodes.java:136)
        at hudson.model.Queue._withLock(Queue.java:1293)
        at hudson.model.Queue.withLock(Queue.java:1176)
        at jenkins.model.Nodes.addNode(Nodes.java:132)
        at jenkins.model.Jenkins.addNode(Jenkins.java:1744)
  • environment: jenkins v1.634

Activity

  1. jenkins-infra-bot commented on Oct 17, 2016

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Ted
    Path:
    core/src/main/java/hudson/model/Computer.java
    http://jenkins-ci.org/commit/jenkins/c7fd90a8796fe0ad80bfe37a9837af4515fbf5cb
    Log:
    fix JENKINS-31768 dead lock while removing computer

  2. jenkins-infra-bot commented on Oct 17, 2016

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Stephen Connolly
    Path:
    core/src/main/java/hudson/model/Computer.java
    http://jenkins-ci.org/commit/jenkins/430736ad619c52bde438c63f5c81e50fedd63fdb
    Log:
    Merge pull request #1933 from fengxx/feature/removecomputer

    [FIXED JENKINS-31768] - Remove monitor lock from setNumberExecutors to avoid deadlock

    Compare: 2311ae8...430736a

  3. jenkins-infra-bot commented on Oct 23, 2016

    @jenkins-infra-bot
    Author
    oleg_nenashev:
    • Original comment link
    • Raw content of original comment:

      The fix is integrated towards 2.27. Marking it as lts-candidate

    The fix is integrated towards 2.27. Marking it as lts-candidate

  4. jenkins-infra-bot commented on Jan 19, 2017

    @jenkins-infra-bot
    Author
    scm_issue_link:

    Code changed in jenkins
    User: Ted
    Path:
    core/src/main/java/hudson/model/Computer.java
    http://jenkins-ci.org/commit/jenkins/1875144a072b9a4e388a7e49492bd24b95574b27
    Log:
    fix JENKINS-31768 dead lock while removing computer

    (cherry picked from commit c7fd90a)

  5. removed
    bugFor changelog: Minor bug. Will be listed after features
    on Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions