Skip to content

[JENKINS-936] Matrix parent build shouldn't consume an executor. #311

Closed
Bug
Closed
[JENKINS-936] Matrix parent build shouldn't consume an executor.#311
Bug
Labels
component:matrix-project-pluginimported-jira-issuepriority:majorresolution:fixed

Description

@jenkins-infra-bot
jenkins-infra-bot
opened on Oct 22, 2007

Hi,

On a multi-configuration project, I encounter a systematic deadlock because the
parent job and one of its child are queued on the same node.

The configuration matrix is as follows:
[x] Build on multiple nodes
[-] Individual nodes
[x] APOLLON (apollon)
[x] CHRONOS (chronos)
[x] DEMETER (demeter)
[ ] EOS (eos)
[x] EROS (eros)
[x] PAN (pan)
[x] PONTOS (pontos)
[ ] master (the master Hudson node)
[+] Labels
[ ] Axes

Here is the console output:

started
Building remotely on PONTOS
Triggering label=PAN
Triggering label=APOLLON
Triggering label=DEMETER
Triggering label=PONTOS
Triggering label=EROS
Triggering label=CHRONOS

All jobs except the ones on "PONTOS" successfully terminate.
The parent job keeps executing, waiting for all its children termination, but it
never stops as the child job supposed to run on "PONTOS" remains in the build queue.

Enabling the master Hudson node has no effect. Its corresponding job also
remains in the build queue.

Thank you for reading.

Regards,
Regis.

Originally reported by rdesgroppes, imported from: Matrix parent build shouldn't consume an executor.
  • assignee: rdesgroppes
  • status: Closed
  • priority: Major
  • component(s): matrix-project-plugin
  • resolution: Fixed
  • resolved: 2015-02-22T17:42:32+00:00
  • votes: 24
  • watchers: 10
  • imported: 2025-12-06
Raw content of original issue 
Hi,



On a multi-configuration project, I encounter a systematic deadlock because the
parent job and one of its child are queued on the same node.



The configuration matrix is as follows:
[x] Build on multiple nodes
  [-] Individual nodes
    [x] APOLLON (apollon)
    [x] CHRONOS (chronos)
    [x] DEMETER (demeter)
    [ ] EOS (eos)
    [x] EROS (eros)
    [x] PAN (pan)
    [x] PONTOS (pontos)
    [ ] master (the master Hudson node)
  [+] Labels
[ ] Axes



Here is the console output:



started
Building remotely on PONTOS
Triggering label=PAN
Triggering label=APOLLON
Triggering label=DEMETER
Triggering label=PONTOS
Triggering label=EROS
Triggering label=CHRONOS



All jobs except the ones on "PONTOS" successfully terminate.
The parent job keeps executing, waiting for all its children termination, but it
never stops as the child job supposed to run on "PONTOS" remains in the build queue.



Enabling the master Hudson node has no effect. Its corresponding job also
remains in the build queue.



Thank you for reading.



Regards,
Regis.
  • environment: Platform: All, OS: All
3 attachments

Activity

  1. jenkins-infra-bot commented on Oct 22, 2007

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 22, 2007
    Author
    kohsuke:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Right. The parent build needs to be run outside of the normal executors so that
it can let child builds use that executor.
      


      In the mean time, a work around is to tie the parent build to somewhere where
more executors are available.

    Right. The parent build needs to be run outside of the normal executors so that
    it can let child builds use that executor.

    In the mean time, a work around is to tie the parent build to somewhere where
    more executors are available.

  2. jenkins-infra-bot commented on Oct 23, 2007

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 23, 2007
    Author
    rdesgroppes:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      I don't see how to put in place the workaround you propose, in that the
multi-project configuration page doesn't allow to tie the parent build to a
dedicated node.

    I don't see how to put in place the workaround you propose, in that the
    multi-project configuration page doesn't allow to tie the parent build to a
    dedicated node.

  3. jenkins-infra-bot commented on Oct 29, 2007

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 29, 2007
    Author
    kohsuke:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
        
	
      • 
	
          
		
        • 
		
            
			
          • Issue 961 has been marked as a duplicate of this issue. ***
            • 
		
          
		
          • 
	
        
	
        •
        • Issue 961 has been marked as a duplicate of this issue. ***

  4. jenkins-infra-bot commented on Mar 19, 2008

    @jenkins-infra-bot
    jenkins-infra-bot
    on Mar 19, 2008
    Author
    kohsuke:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
        
	
      • 
	
          
		
        • 
		
            
			
          • Issue 1432 has been marked as a duplicate of this issue. ***
            • 
		
          
		
          • 
	
        
	
        •
        • Issue 1432 has been marked as a duplicate of this issue. ***

  5. jenkins-infra-bot commented on Apr 18, 2008

    @jenkins-infra-bot
    jenkins-infra-bot
    on Apr 18, 2008
    Author
    musilt2:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
        
	
      • 
	
          
		
        • 
		
            
			
          • Issue 1561 has been marked as a duplicate of this issue. ***
            • 
		
          
		
          • 
	
        
	
        •
        • Issue 1561 has been marked as a duplicate of this issue. ***

  6. jenkins-infra-bot commented on Apr 18, 2008

    @jenkins-infra-bot
    jenkins-infra-bot
    on Apr 18, 2008
    Author
    musilt2:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      what's the current status of this issue? any time-frame when the fix will be
available?

    what's the current status of this issue? any time-frame when the fix will be
    available?

  7. jenkins-infra-bot commented on Apr 18, 2008

    @jenkins-infra-bot
    jenkins-infra-bot
    on Apr 18, 2008
    Author
    mirilovic:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      It would be great to have a possibility to tie the parent job to particular
machine - and I personally think the best one would be Master.

    It would be great to have a possibility to tie the parent job to particular
    machine - and I personally think the best one would be Master.

  8. jenkins-infra-bot commented on Jul 6, 2008

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 6, 2008
    Author
    lloydchang:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Kohsuke, unless I'm mistaken, a code change in the Hudson Core is needed to even
try the work-around.
      


      Like others, I'm not sure how to configure Hudson to tie the parent build to a
specific node.  In node selection, I selected 1 label for multiple axes to use,
but I don't see any options for the parent build.

    Kohsuke, unless I'm mistaken, a code change in the Hudson Core is needed to even
    try the work-around.

    Like others, I'm not sure how to configure Hudson to tie the parent build to a
    specific node. In node selection, I selected 1 label for multiple axes to use,
    but I don't see any options for the parent build.

  9. jenkins-infra-bot commented on Jul 18, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 18, 2009
    Author
    scm_issue_link:

    Code changed in hudson
    User: : kohsuke
    Path:
    http://fisheye4.cenqua.com/changelog/hudson/?cs=19911
    Log:
    JENKINS-936 Created a branch to experiment with the solution.

  10. jenkins-infra-bot commented on Jul 18, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 18, 2009
    Author
    kohsuke:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Note to myself. There's two ways to do this.
      


      One is to just add one more executor temporarily to compensate the effect. This
is easy, but the downside is that the added executor may end up doing something
else, and it might take a bit of time before they get released. Plus this will
cause a UI discrepancy between the user setting vs what they see.
      


      Another is to add subtype of Executor and let it run just so that it can execute
the parent build. The trick is to find a situation where an executor shouldn't
run (like Hudson is shutting down) so that we won't lose the item.

    Note to myself. There's two ways to do this.

    One is to just add one more executor temporarily to compensate the effect. This
    is easy, but the downside is that the added executor may end up doing something
    else, and it might take a bit of time before they get released. Plus this will
    cause a UI discrepancy between the user setting vs what they see.

    Another is to add subtype of Executor and let it run just so that it can execute
    the parent build. The trick is to find a situation where an executor shouldn't
    run (like Hudson is shutting down) so that we won't lose the item.

  11. jenkins-infra-bot commented on Jul 19, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 19, 2009
    Author
    ridesmet:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Kohsuke, from your last comment, I think your second suggestion is a better solution. Just add a "virtual" 
executor to the master that can run the composite builds, and also making the composite build of type 
"virtual".

    Kohsuke, from your last comment, I think your second suggestion is a better solution. Just add a "virtual"
    executor to the master that can run the composite builds, and also making the composite build of type
    "virtual".

  12. jenkins-infra-bot commented on Jul 20, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 20, 2009
    Author
    scm_issue_link:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Code changed in hudson
User: : kohsuke
Path:
 branches/matrix-parent/core/src/main/java/hudson/matrix/MatrixProject.java
 branches/matrix-parent/core/src/main/java/hudson/model/Computer.java
 branches/matrix-parent/core/src/main/java/hudson/model/Executor.java
 branches/matrix-parent/core/src/main/java/hudson/model/OneOffExecutor.java
 branches/matrix-parent/core/src/main/java/hudson/model/Queue.java
 branches/matrix-parent/core/src/main/resources/lib/hudson/executors.jelly
http://fisheye4.cenqua.com/changelog/hudson/?cs=19998
Log:
JENKINS-936 I believe this should do.

    Code changed in hudson
    User: : kohsuke
    Path:
    branches/matrix-parent/core/src/main/java/hudson/matrix/MatrixProject.java
    branches/matrix-parent/core/src/main/java/hudson/model/Computer.java
    branches/matrix-parent/core/src/main/java/hudson/model/Executor.java
    branches/matrix-parent/core/src/main/java/hudson/model/OneOffExecutor.java
    branches/matrix-parent/core/src/main/java/hudson/model/Queue.java
    branches/matrix-parent/core/src/main/resources/lib/hudson/executors.jelly
    http://fisheye4.cenqua.com/changelog/hudson/?cs=19998
    Log:
    JENKINS-936 I believe this should do.

  13. jenkins-infra-bot commented on Jul 21, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Jul 21, 2009
    Author
    kohsuke:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      Hudson 1.317 will include the fix for this, but because of a potential impact to
users, the fix is disabled by default for now. I'd like interested parties to
enable this (by setting the system property
hudson.model.Hudson.flyweightSupport=true on Hudson JVM), and report back if
this is working OK for you.
      


      If the fix appears to work without any side effect, I'll enable the fix by default.

    Hudson 1.317 will include the fix for this, but because of a potential impact to
    users, the fix is disabled by default for now. I'd like interested parties to
    enable this (by setting the system property
    hudson.model.Hudson.flyweightSupport=true on Hudson JVM), and report back if
    this is working OK for you.

    If the fix appears to work without any side effect, I'll enable the fix by default.

  14. jenkins-infra-bot commented on Aug 27, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Aug 27, 2009
    Author
    hydraswitch:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      I set the property as described and hit Build Now for my multi configuration job.
The one Node that I did not select the job for is now showing a Dead  thread?
or something.  Not sure what happened.  It goes away if I stop and restart hudson.
It comes back again each time the job runs.  It seems to run correctly on the nodes
that I checked it on for.

    I set the property as described and hit Build Now for my multi configuration job.
    The one Node that I did not select the job for is now showing a Dead thread?
    or something. Not sure what happened. It goes away if I stop and restart hudson.
    It comes back again each time the job runs. It seems to run correctly on the nodes
    that I checked it on for.

  15. jenkins-infra-bot commented on Sep 23, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Sep 23, 2009
    Author
    emmulator:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      I was hoping this might help with the problems I've encountered in Issue 1022. 
It does seem to address the problem of the deadlock, but without the ability to
have the parent job run on a different slave node from any of the children, the
interaction with perforce still causes one of the children to not sync its
workspace.
      


      I found this issue when looking for how I would expose the
assignedNode/hasSlaveAffinity property of a MatrixProject, in the hopes that if
I could tie the parent to a particular node, it would no longer interfere with
the children.  I've noticed that a few other people in this ticket were thinking
along the same lines, but you have chosen this virtual executor approach
instead.  Is there a reason it would not be desirable to be able to tie the
parent to a particular node?  And even if you don't like that approach for the
general release, could you please point me to where in the source I would go to
expose that property?  I realize that the 'real' solution to Issue 1022 probably
involves modifying the perforce plugin, but that seems more complicated, and I
was hoping this would work as a workaround in the meantime.
      


      Thanks!

    I was hoping this might help with the problems I've encountered in Issue 1022.
    It does seem to address the problem of the deadlock, but without the ability to
    have the parent job run on a different slave node from any of the children, the
    interaction with perforce still causes one of the children to not sync its
    workspace.

    I found this issue when looking for how I would expose the
    assignedNode/hasSlaveAffinity property of a MatrixProject, in the hopes that if
    I could tie the parent to a particular node, it would no longer interfere with
    the children. I've noticed that a few other people in this ticket were thinking
    along the same lines, but you have chosen this virtual executor approach
    instead. Is there a reason it would not be desirable to be able to tie the
    parent to a particular node? And even if you don't like that approach for the
    general release, could you please point me to where in the source I would go to
    expose that property? I realize that the 'real' solution to Issue 1022 probably
    involves modifying the perforce plugin, but that seems more complicated, and I
    was hoping this would work as a workaround in the meantime.

    Thanks!

  16. 32 remaining items

  17. jenkins-infra-bot commented on Feb 22, 2015

    @jenkins-infra-bot
    jenkins-infra-bot
    on Feb 22, 2015
    Author
    imakowski:
    • Comment link
    • Original comment link (no redirect)
    • Raw content of original comment: 
      I still have this issue on latest LTS. It blocks our executors when Restrict label for matrix job is used!

    I still have this issue on latest LTS. It blocks our executors when Restrict label for matrix job is used!

  18. jenkins-infra-bot commented on Feb 22, 2015

    @jenkins-infra-bot
    jenkins-infra-bot
    on Feb 22, 2015
    Author
    imakowski:

    See my latest comment

  19. jenkins-infra-bot commented on Feb 22, 2015

    @jenkins-infra-bot
    jenkins-infra-bot
    on Feb 22, 2015
    Author
    danielbeck:

    It blocks our executors when Restrict label for matrix job is used!

    Please file a new issue, and include specific information on how to reproduce this.

    https://wiki.jenkins-ci.org/display/JENKINS/How+to+report+an+issue

  20. jenkins-infra-bot
    closed this as completedon Feb 22, 2015
  21. jenkins-infra-bot
    added
    bug
    imported-jira-issue
    component:matrix-project-plugin
    priority:major
    resolution:fixed
    on Dec 6, 2025

  22. jenkins-infra-bot commented on Dec 6, 2025

    @jenkins-infra-bot
    jenkins-infra-bot
    on Dec 6, 2025
    Author

    [Original is_duplicated_by from Jira: JENKINS-4552]

  23. jenkins-infra-bot commented on Dec 6, 2025

    @jenkins-infra-bot
    jenkins-infra-bot
    on Dec 6, 2025
    Author

    [Original is_duplicated_by from Jira: JENKINS-1561]

  24. jenkins-infra-bot commented on Dec 6, 2025

    @jenkins-infra-bot
    jenkins-infra-bot
    on Dec 6, 2025
    Author

    [Original is_duplicated_by from Jira: JENKINS-961]

  25. jenkins-infra-bot commented on Dec 6, 2025

    @jenkins-infra-bot
    jenkins-infra-bot
    on Dec 6, 2025
    Author

    [Original is_duplicated_by from Jira: JENKINS-1432]

  26. jenkins-infra-bot
    added the
    Bug
    issue type on Dec 6, 2025
  27. jenkins-infra-bot
    removed
    bug
    on Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:matrix-project-pluginimported-jira-issuepriority:majorresolution:fixed

    Type

    Bug

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions