Skip to content

[JENKINS-3890] Hudson fails to launch the slave agent a second time, even though it availablity is to go off line on idle. #945

Closed
Bug
Closed
[JENKINS-3890] Hudson fails to launch the slave agent a second time, even though it availablity is to go off line on idle.#945
Bug
Labels
component:remotingimported-jira-issuepriority:criticalresolution:fixed

Description

@jenkins-infra-bot
jenkins-infra-bot
opened on Jun 18, 2009

I had this working with older versions of Hudson, it broke on upgrade.

I can manually launch the agent, and it is configured as such.

Launch method = ssh
Availability = Take node off line when idle

then when the agent goes off line Hudson fails to restart it. I then have to
manually click the "launch slave agent" to restart it. Once this is done, the
agent runs until the agent is idle, when it will go off line again, and will
never come on line again until I click the button "launch slave agent".

The agents are running on Scientific Linux (Red Hat binary compatible OS) 32 and
64 bit, versions 4 and 5.

The following is a typical log file

[06/16/09 18:43:32] [SSH] Connection closed.

channel stopped
[06/16/09 18:43:32] slave agent was terminated
hudson.remoting.Channel$OrderlyShutdown
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:623)
at hudson.remoting.Channel$ReaderThread.run(Channel.java:758)
Caused by: Command close created at
at hudson.remoting.Command.(Command.java:47)
at hudson.remoting.Channel$CloseCommand.(Channel.java:619)
at hudson.remoting.Channel$CloseCommand.(Channel.java:619)
at hudson.remoting.Channel.close(Channel.java:663)
at hudson.remoting.Channel$CloseCommand.execute(Channel.java:622)
... 1 more

This is 100% repeatable, we are running Hudson ver. 1.310 and this bug has been
around for quiet some releases, maybe 3-4 months.

Originally reported by owensynge, imported from: Hudson fails to launch the slave agent a second time, even though it availablity is to go off line on idle.
  • assignee: mindless
  • status: Closed
  • priority: Critical
  • component(s): remoting
  • resolution: Fixed
  • resolved: 2009-11-19T15:38:32+00:00
  • votes: 0
  • watchers: 2
  • imported: 2025-11-25
Raw content of original issue 
I had this working with older versions of Hudson, it broke on upgrade.



I can manually launch the agent, and it is configured as such.



Launch method = ssh
Availability = Take node off line when idle



then when the agent goes off line Hudson fails to restart it. I then have to
manually click the "launch slave agent" to restart it. Once this is done, the
agent runs until the agent is idle, when it will go off line again, and will
never come on line again until I click the button "launch slave agent".



The agents are running on Scientific Linux (Red Hat binary compatible OS) 32 and
64 bit, versions 4 and 5. 



The following is a typical log file 



[06/16/09 18:43:32] [SSH] Connection closed.



<lots of jibberish characters>



channel stopped
[06/16/09 18:43:32] slave agent was terminated
hudson.remoting.Channel$OrderlyShutdown
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:623)
	at hudson.remoting.Channel$ReaderThread.run(Channel.java:758)
Caused by: Command close created at
	at hudson.remoting.Command.<init>(Command.java:47)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:619)
	at hudson.remoting.Channel$CloseCommand.<init>(Channel.java:619)
	at hudson.remoting.Channel.close(Channel.java:663)
	at hudson.remoting.Channel$CloseCommand.execute(Channel.java:622)
	... 1 more




This is 100% repeatable, we are running Hudson ver. 1.310 and this bug has been
around for quiet some releases, maybe 3-4 months.
  • environment: Platform: All, OS: Linux

Activity

  1. jenkins-infra-bot commented on Oct 20, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Oct 20, 2009
    Author
    mirzmaster:
    • Original comment link
    • Raw content of original comment: 
      I can confirm the occurrence of this issue.  The agents are running on Ubuntu
Hardy, Intrepid and Jaunty, all 32-bit.
      


      The SSH plugin appeared to be working just fine up to Hudson 1.304.  A
subsequent release appears to have caused this regression (perhaps coinciding
with the inclusion of the SSH plugin as a default Hudson plugin).

    I can confirm the occurrence of this issue. The agents are running on Ubuntu
    Hardy, Intrepid and Jaunty, all 32-bit.

    The SSH plugin appeared to be working just fine up to Hudson 1.304. A
    subsequent release appears to have caused this regression (perhaps coinciding
    with the inclusion of the SSH plugin as a default Hudson plugin).

  2. jenkins-infra-bot commented on Nov 4, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 4, 2009
    Author
    mindless:
    • Original comment link
    • Raw content of original comment: 
      add cc

    add cc

  3. jenkins-infra-bot commented on Nov 4, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 4, 2009
    Author
    mindless:
    • Original comment link
    • Raw content of original comment: 
        
	
      • 
	
          
		
        • 
		
            
			
          • Issue 4302 has been marked as a duplicate of this issue. ***
            • 
		
          
		
          • 
	
        
	
        •
        • Issue 4302 has been marked as a duplicate of this issue. ***

  4. jenkins-infra-bot commented on Nov 4, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 4, 2009
    Author
    mindless:
    • Original comment link
    • Raw content of original comment: 
      From #4302:
"We discovered this in Hudson v1.309.  When a slave is configured with an idle
delay, it eventually shuts down due to inactivity.  When this happens, the
Hudson master seems to be unable to distinguish this condition from a genuine
failure, and marks the slave as offline.  If the slave is also configured with
an on-demand delay, then Hudson will attempt to start the slave if its job queue
is non-empty for the specified time, but will fail because it thinks the slave
has failed."

    From #4302:
    "We discovered this in Hudson v1.309. When a slave is configured with an idle
    delay, it eventually shuts down due to inactivity. When this happens, the
    Hudson master seems to be unable to distinguish this condition from a genuine
    failure, and marks the slave as offline. If the slave is also configured with
    an on-demand delay, then Hudson will attempt to start the slave if its job queue
    is non-empty for the specified time, but will fail because it thinks the slave
    has failed."

  5. jenkins-infra-bot commented on Nov 19, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 19, 2009
    Author
    mindless:
    • Original comment link
    • Raw content of original comment: 
      working on this.. problem is that SlaveComputer.lastConnectActivity never gets
reset to null, so when it tries to relaunch the node it thinks it is already
trying to connect.  (the Launch now button, on the other hand, does a
"forceReconnect" so it launches even though it thinks it is already trying to
connect)

    working on this.. problem is that SlaveComputer.lastConnectActivity never gets
    reset to null, so when it tries to relaunch the node it thinks it is already
    trying to connect. (the Launch now button, on the other hand, does a
    "forceReconnect" so it launches even though it thinks it is already trying to
    connect)

  6. jenkins-infra-bot commented on Nov 19, 2009

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 19, 2009
    Author
    scm_issue_link:
    • Original comment link
    • Raw content of original comment: 
      Code changed in hudson
User: : mindless
Path:
 trunk/hudson/main/core/src/main/java/hudson/slaves/SlaveComputer.java
 trunk/www/changelog.html
http://fisheye4.cenqua.com/changelog/hudson/?cs=23875
Log:
[FIXED JENKINS-3890] Fix in-demand RetentionStrategy and more offline-slave fixes.
      

        
	
      • Use isConnecting() instead of lastConnectActivity!=null in checking forceReconnect
  param.. var never resets to null, so this prevented forceReconnect==false from ever
  reconnecting (which broken in-demand strategy since 1.302 when it started using false)
        • 
	
      • Only set ChannelTermination offline cause when the exception param is non-null.
  On orderly shutdown this was replacing the real cause of going offline.
        • 
	
      • In setNode try to delegate to RetentionStrategy rather than calling connect directly.
  Reconfiguring a node was launching slave even if it wasn't needed per its strategy.
        •

    Code changed in hudson
    User: : mindless
    Path:
    trunk/hudson/main/core/src/main/java/hudson/slaves/SlaveComputer.java
    trunk/www/changelog.html
    http://fisheye4.cenqua.com/changelog/hudson/?cs=23875
    Log:
    [FIXED JENKINS-3890] Fix in-demand RetentionStrategy and more offline-slave fixes.

    • Use isConnecting() instead of lastConnectActivity!=null in checking forceReconnect
      param.. var never resets to null, so this prevented forceReconnect==false from ever
      reconnecting (which broken in-demand strategy since 1.302 when it started using false)
    • Only set ChannelTermination offline cause when the exception param is non-null.
      On orderly shutdown this was replacing the real cause of going offline.
    • In setNode try to delegate to RetentionStrategy rather than calling connect directly.
      Reconfiguring a node was launching slave even if it wasn't needed per its strategy.
  7. jenkins-infra-bot
    closed this as completedon Nov 19, 2009

  8. jenkins-infra-bot commented on Nov 25, 2025

    @jenkins-infra-bot
    jenkins-infra-bot
    on Nov 25, 2025
    Author

    [Original is_duplicated_by from Jira: JENKINS-4302]

  9. jenkins-infra-bot
    added
    bugFor changelog: Fixes a bug.
    imported-jira-issue
    component:remoting
    resolution:fixed
    priority:critical
    on Nov 25, 2025
  10. jenkins-infra-bot
    added the
    Bug
    issue type on Nov 25, 2025
  11. jenkins-infra-bot
    removed
    bugFor changelog: Fixes a bug.
    on Nov 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    component:remotingimported-jira-issuepriority:criticalresolution:fixed

    Type

    Bug

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions