[JENKINS-52204] Jenkins Windows Agent failed to start after upgrade to 2.1.129

[jenkins-infra-bot]

Jenkins Windows Agent failed to start after upgrade to version 2.1.129: Jenkins is running behind apache reverse proxy on different host that proxy itself. Parameter “tunnel connection through” is set correctly. From the error message we can see that remote agent was trying to connect to reverse proxy host instead of Jenkins master host.

The version 2.1.128 works without issues.

---

Originally reported by dmazuronak, imported from: Jenkins Windows Agent failed to start after upgrade to 2.1.129

• assignee: oleg_nenashev
• status: Resolved
• priority: Major
• component(s): remoting
• resolution: Fixed
• resolved: 2018-07-03T17:56:47+00:00
• votes: 0
• watchers: 6
• imported: 2025-11-25

Raw content of original issue

Jenkins Windows Agent failed to start after upgrade to version 2.1.129: Jenkins is running behind apache reverse proxy on different host that proxy itself. Parameter “tunnel connection through” is set correctly. From the error message we can see that remote agent was trying to connect to reverse proxy host instead of Jenkins master host.

The version 2.1.128 works without issues.

• environment: Jenkins 2.1.129

[jenkins-infra-bot]

• Remote links associated with this issue:

  • [JENKINS-52204] - Skip port availability check when -tunnel option is set #279
  • [JENKINS-52204] - Update Remoting to 3.23 to fix regressions in 2.129 jenkins#3530

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  Most likely it is related to JENKINS-51818, CC denami
  dmazuronak have you updated Remoting on agents to 3.22?
  

Most likely it is related to JENKINS-51818, CC denami
dmazuronak have you updated Remoting on agents to 3.22?

[jenkins-infra-bot]

dmazuronak:

• Original comment link
• Raw content of original comment:

  Remoting 3.22 has the problem. Remoting 3.21 works OK:
  
  INFORMATION: Setting up agent: MY-CLIENT
  Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main$CuiListener <init>
  INFORMATION: Jenkins agent is running in headless mode.
  Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main createEngine
  WARNUNG: Certificate validation for HTTPs endpoints is disabled
  Jun 26, 2018 11:07:54 PM hudson.remoting.Engine startEngine
  INFORMATION: Using Remoting version: 3.22
  Jun 26, 2018 11:07:54 PM hudson.remoting.Engine startEngine
  WARNUNG: No Working Directory. Using the legacy JAR Cache location: C:\Windows\system32\config\systemprofile\.jenkins\cache\jars
  Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main$CuiListener status
  INFORMATION: Locating server among [https://tools.domain.local/devjenkins/]
  Jun 26, 2018 11:07:54 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver openURLConnection
  WARNUNG: HTTPs certificate check is disabled for the endpoint.
  Jun 26, 2018 11:07:55 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
  INFORMATION: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
  Jun 26, 2018 11:07:56 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver isPortVisible
  WARNUNG: Connection refused: connect
  Jun 26, 2018 11:07:56 PM hudson.remoting.jnlp.Main$CuiListener error
  SCHWERWIEGEND: https://tools.domain.local/devjenkins/ provided port:37367 is not reachable
  java.io.IOException: https://tools.domain.local/devjenkins/ provided port:37367 is not reachable
  	at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:284)
  	at hudson.remoting.Engine.innerRun(Engine.java:518)
  	at hudson.remoting.Engine.run(Engine.java:469)
  
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main createEngine
  INFORMATION: Setting up agent: MY-CLIENT
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener <init>
  INFORMATION: Jenkins agent is running in headless mode.
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main createEngine
  WARNUNG: Certificate validation for HTTPs endpoints is disabled
  Jun 27, 2018 8:46:17 AM hudson.remoting.Engine startEngine
  INFORMATION: Using Remoting version: 3.21
  Jun 27, 2018 8:46:17 AM hudson.remoting.Engine startEngine
  WARNUNG: No Working Directory. Using the legacy JAR Cache location: C:\Windows\system32\config\systemprofile\.jenkins\cache\jars
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
  INFORMATION: Locating server among [https://tools.domain.local/devjenkins/]
  Jun 27, 2018 8:46:17 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver openURLConnection
  WARNUNG: HTTPs certificate check is disabled for the endpoint.
  Jun 27, 2018 8:46:17 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
  INFORMATION: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
  INFORMATION: Agent discovery successful
    Agent address: 10.150.3.9
    Agent port:    44864
    Identity:      f8:2a:3d:5a:d4:22:f4:a0:89:26:4a:a1:34:20:c3:11
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
  INFORMATION: Handshaking
  Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
  INFORMATION: Connecting to 10.150.3.9:44864
  
  
   Port 37367 is open on Jenkins server 10.150.3.9 and not on reverse proxy tools.domain.local.
  

Remoting 3.22 has the problem. Remoting 3.21 works OK:

INFORMATION: Setting up agent: MY-CLIENT
Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFORMATION: Jenkins agent is running in headless mode.
Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main createEngine
WARNUNG: Certificate validation for HTTPs endpoints is disabled
Jun 26, 2018 11:07:54 PM hudson.remoting.Engine startEngine
INFORMATION: Using Remoting version: 3.22
Jun 26, 2018 11:07:54 PM hudson.remoting.Engine startEngine
WARNUNG: No Working Directory. Using the legacy JAR Cache location: C:\Windows\system32\config\systemprofile\.jenkins\cache\jars
Jun 26, 2018 11:07:54 PM hudson.remoting.jnlp.Main$CuiListener status
INFORMATION: Locating server among [https://tools.domain.local/devjenkins/]
Jun 26, 2018 11:07:54 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver openURLConnection
WARNUNG: HTTPs certificate check is disabled for the endpoint.
Jun 26, 2018 11:07:55 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFORMATION: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jun 26, 2018 11:07:56 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver isPortVisible
WARNUNG: Connection refused: connect
Jun 26, 2018 11:07:56 PM hudson.remoting.jnlp.Main$CuiListener error
SCHWERWIEGEND: https://tools.domain.local/devjenkins/ provided port:37367 is not reachable
java.io.IOException: https://tools.domain.local/devjenkins/ provided port:37367 is not reachable
	at org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver.resolve(JnlpAgentEndpointResolver.java:284)
	at hudson.remoting.Engine.innerRun(Engine.java:518)
	at hudson.remoting.Engine.run(Engine.java:469)

Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main createEngine
INFORMATION: Setting up agent: MY-CLIENT
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener <init>
INFORMATION: Jenkins agent is running in headless mode.
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main createEngine
WARNUNG: Certificate validation for HTTPs endpoints is disabled
Jun 27, 2018 8:46:17 AM hudson.remoting.Engine startEngine
INFORMATION: Using Remoting version: 3.21
Jun 27, 2018 8:46:17 AM hudson.remoting.Engine startEngine
WARNUNG: No Working Directory. Using the legacy JAR Cache location: C:\Windows\system32\config\systemprofile\.jenkins\cache\jars
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
INFORMATION: Locating server among [https://tools.domain.local/devjenkins/]
Jun 27, 2018 8:46:17 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver openURLConnection
WARNUNG: HTTPs certificate check is disabled for the endpoint.
Jun 27, 2018 8:46:17 AM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFORMATION: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
INFORMATION: Agent discovery successful
  Agent address: 10.150.3.9
  Agent port:    44864
  Identity:      f8:2a:3d:5a:d4:22:f4:a0:89:26:4a:a1:34:20:c3:11
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
INFORMATION: Handshaking
Jun 27, 2018 8:46:17 AM hudson.remoting.jnlp.Main$CuiListener status
INFORMATION: Connecting to 10.150.3.9:44864

 Port 37367 is open on Jenkins server 10.150.3.9 and not on reverse proxy tools.domain.local.

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  From what I see in the https://github.com/jenkinsci/remoting/pull/275/files code, the logic should be using the resolved hostname for direct connection:
  
  
  host = defaultString(first(header(con, "X-Jenkins-JNLP-Host")), salURL.getHost());
  
  
  Are you sure that "hudson.TcpSlaveAgentListener.hostName" is set on your instance? 
  If no, setting it up on the Master side would be a workaround until we figure out the tunneling issue
  

From what I see in the https://github.com/jenkinsci/remoting/pull/275/files code, the logic should be using the resolved hostname for direct connection:

host = defaultString(first(header(con, "X-Jenkins-JNLP-Host")), salURL.getHost());

Are you sure that "hudson.TcpSlaveAgentListener.hostName" is set on your instance?
If no, setting it up on the Master side would be a workaround until we figure out the tunneling issue

[jenkins-infra-bot]

dmazuronak:

• Original comment link
• Raw content of original comment:

  On master side this slave has the parameter "Tunnel connection through" set to 10.150.3.9.
  

On master side this slave has the parameter "Tunnel connection through" set to 10.150.3.9.

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  dmazuronak Could you please set the "-Dhudson.TcpSlaveAgentListener.hostName" system property on master as I suggested above?
  

dmazuronak Could you please set the "-Dhudson.TcpSlaveAgentListener.hostName" system property on master as I suggested above?

[jenkins-infra-bot]

crule:

• Original comment link
• Raw content of original comment:

  I'm having the same issue. I have  jenkins running on host "j-jenkins" and the apache web-server running as reverse proxy on "eddard". Setting the hudson.TcpSlaveAgentListener.hostName property to "j-jenkins" did not correct the problem. I receive the same error as before "IOException: https://eddard/jenkins/ provided port:41829 is not reachable".
  

I'm having the same issue. I have  jenkins running on host "j-jenkins" and the apache web-server running as reverse proxy on "eddard". Setting the hudson.TcpSlaveAgentListener.hostName property to "j-jenkins" did not correct the problem. I receive the same error as before "IOException: https://eddard/jenkins/ provided port:41829 is not reachable".

[jenkins-infra-bot]

dmazuronak:

• Original comment link
• Raw content of original comment:

  I can only confirm the information from Chris. Unfortunately, using the system property "hudson.TcpSlaveAgentListener.hostName" did not bring any improvement.
  
  There is no such header "X-Jenkins-JNLP-Host" in http response.
  
  The property "hudson.TcpSlaveAgentListener.hostName" was set in the command line. In which configuration file can this property be set? I'm not sure that the command line was processed.
  

I can only confirm the information from Chris. Unfortunately, using the system property "hudson.TcpSlaveAgentListener.hostName" did not bring any improvement.

There is no such header "X-Jenkins-JNLP-Host" in http response.

The property "hudson.TcpSlaveAgentListener.hostName" was set in the command line. In which configuration file can this property be set? I'm not sure that the command line was processed.

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  java -Dhudson.TcpSlaveAgentListener.hostName=fooBar -jar jenkins.war should do the job.
  Cannot suggest config file since I do not know Jenkins configuration
  

java -Dhudson.TcpSlaveAgentListener.hostName=fooBar -jar jenkins.war should do the job.
Cannot suggest config file since I do not know Jenkins configuration

[jenkins-infra-bot]

denami:

• Original comment link
• Raw content of original comment:

  You can receive value of "hudson.TcpSlaveAgentListener.hostName". By Jenkins  script console (System.getProperty("hudson.TcpSlaveAgentListener.hostName") ).  Possible it is override by other  JAVA_OPTS/JENKINS_JAVA_OPTIONS)
  

You can receive value of "hudson.TcpSlaveAgentListener.hostName". By Jenkins  script console (System.getProperty("hudson.TcpSlaveAgentListener.hostName") ).  Possible it is override by other  JAVA_OPTS/JENKINS_JAVA_OPTIONS)

[jenkins-infra-bot]

dmazuronak:

• Original comment link
• Raw content of original comment:

  No, the other options are empty, so no override here.
  
  If I type
  
  println(System.getProperty("hudson.TcpSlaveAgentListener.hostName") )
  
  
  in Jenkins script console, I get only
  
  null
  
  
  as result.
   
  There is an other interresting result:
  
  hudson.TcpSlaveAgentListener.hostName
  
  
  produces following exception:
  
  groovy.lang.MissingPropertyException: No such property: hostName for class: hudson.TcpSlaveAgentListener
  	at groovy.lang.MetaClassImpl.invokeStaticMissingProperty(MetaClassImpl.java:1002)
  	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1857)
  	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1833)
  	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3758)
  	at org.codehaus.groovy.runtime.callsite.ClassMetaClassGetPropertySite.getProperty(ClassMetaClassGetPropertySite.java:51)
  	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:296)
  	at Script1.run(Script1.groovy:1)
  
  
  

No, the other options are empty, so no override here.

If I type

println(System.getProperty("hudson.TcpSlaveAgentListener.hostName") )

in Jenkins script console, I get only

null

as result.

 

There is an other interresting result:

hudson.TcpSlaveAgentListener.hostName

produces following exception:

groovy.lang.MissingPropertyException: No such property: hostName for class: hudson.TcpSlaveAgentListener
	at groovy.lang.MetaClassImpl.invokeStaticMissingProperty(MetaClassImpl.java:1002)
	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1857)
	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:1833)
	at groovy.lang.MetaClassImpl.getProperty(MetaClassImpl.java:3758)
	at org.codehaus.groovy.runtime.callsite.ClassMetaClassGetPropertySite.getProperty(ClassMetaClassGetPropertySite.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:296)
	at Script1.run(Script1.groovy:1)

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  > There is an other interresting result:
  
  As designed. System properties won't be visible in Groovy in such way.
  
  dmazuronak System properties are not passed to your Jenkins master due to whatever reason, so the proposed workaround will not work until the property is passed correctly.
  
  

> There is an other interresting result:

As designed. System properties won't be visible in Groovy in such way.

dmazuronak System properties are not passed to your Jenkins master due to whatever reason, so the proposed workaround will not work until the property is passed correctly.

[jenkins-infra-bot]

oleg_nenashev:

• Original comment link
• Raw content of original comment:

  See my comment in [JENKINS-52204] - Skip port availability check when -tunnel option is set #279 . After the review, this issue happens IFF "-tunnel" is set. I believe that "-tunnel" flag should be deprecated and replaced by offering a custom port in TcpAgentListener response headers like it's already done for hosts
  
  So I propose to just disable checks in that case if denami agrees. We could implement a more advanced behavior and checks for tunnel options as a part of JENKINS-52246.
  
  

See my comment in #279 . After the review, this issue happens IFF "-tunnel" is set. I believe that "-tunnel" flag should be deprecated and replaced by offering a custom port in TcpAgentListener response headers like it's already done for hosts

So I propose to just disable checks in that case if denami agrees. We could implement a more advanced behavior and checks for tunnel options as a part of JENKINS-52246.

[jenkins-infra-bot]

dmazuronak:

• Original comment link
• Raw content of original comment:

  You do it right, and it works: I found my error and placed the option in front of jenkins.jar. Now Windows slave is accessible again.
  

You do it right, and it works: I found my error and placed the option in front of jenkins.jar. Now Windows slave is accessible again.

[jenkins-infra-bot]

crule:

• Original comment link
• Raw content of original comment:

  dmazuronak :
  
  Agreed. I had to modify the Jenkins startup script for the define (-D) to take effect. It appears you can't use the Jenkins console to make the change, even though it showed up in the Jenkins System Information page.
  
  Since I run on Linux Mint (Ubuntu derivative) I modified the file /etc/defaults/jenkins, JAVA_ARGS line, to add the -D as described above. My final line looks like (the first -D... was already there):
  
  JAVA_ARGS="-Djava.awt.headless=true -Dhudson.TcpSlaveAgentListener.hostName=j-jenkins"
  
  
  

dmazuronak :

Agreed. I had to modify the Jenkins startup script for the define (-D) to take effect. It appears you can't use the Jenkins console to make the change, even though it showed up in the Jenkins System Information page.

Since I run on Linux Mint (Ubuntu derivative) I modified the file /etc/defaults/jenkins, JAVA_ARGS line, to add the -D as described above. My final line looks like (the first -D... was already there):

JAVA_ARGS="-Djava.awt.headless=true -Dhudson.TcpSlaveAgentListener.hostName=j-jenkins"