[JENKINS-73760] Updates fail due to invalid JSON from HTTP updates.jenkins.io (HTTPS is ok)

[jenkins-infra-bot]

Updates currently fail due to the corresponding endpoint at http://updates.jenkins-ci.org/update-center.json (redirected to https://westeurope.cloudflare.jenkins.io/dynamic-2.476/update-center.json) returning invalid JSON (first characters shown here):

updateCenter.post(
{"connectionCheckUrl":"https://www.go

Stacktrace on startup:

java.io.IOException: Could not find JSON in http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476
at hudson.model.DownloadService.loadJSON(DownloadService.java:129)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:217)
at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:212)
at hudson.model.UpdateCenter.updateAllSitesNow(UpdateCenter.java:2866)
at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)

Firefox complains about this as well:

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

---

Originally reported by stefan6419846, imported from: Updates fail due to invalid JSON from HTTP updates.jenkins.io (HTTPS is ok)

• assignee: basil
• status: Closed
• priority: Major
• component(s): core
• label(s): 2.479.1-fixed
• resolution: Fixed
• resolved: 2024-10-05T15:06:27+00:00
• votes: 2
• watchers: 4
• imported: 2025-11-24

Raw content of original issue

Updates currently fail due to the corresponding endpoint at http://updates.jenkins-ci.org/update-center.json (redirected to https://westeurope.cloudflare.jenkins.io/dynamic-2.476/update-center.json) returning invalid JSON (first characters shown here):


updateCenter.post(
{"connectionCheckUrl":"https://www.go


Stacktrace on startup:

java.io.IOException: Could not find JSON in http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476
        at hudson.model.DownloadService.loadJSON(DownloadService.java:129)
        at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:217)
        at hudson.model.UpdateSite.updateDirectlyNow(UpdateSite.java:212)
        at hudson.model.UpdateCenter.updateAllSitesNow(UpdateCenter.java:2866)
        at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)


Firefox complains about this as well:

SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

1 attachment

• image-2024-09-12-13-29-13-082.png

  

[jenkins-infra-bot]

• Remote link associated with this issue:

  • jenkinsci/jenkins#​9760

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  Hi! Most probably an unexpected consequence of [INFRA-3100] Migrate updates.jenkins.io from CloudBees AWS account to another Cloud (Azure + Cloudflare) jenkins-infra/helpdesk#2649. We are working on it
  

Hi! Most probably an unexpected consequence of jenkins-infra/helpdesk#2649. We are working on it

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  Hi stefan6419846 could you retry to see if our fixes did correct the problem?
  

Hi stefan6419846 could you retry to see if our fixes did correct the problem?

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  On the Jenkins infra side: we do not see any error when retrieving the UC from our 2.476 Jenkins controllers. New files are published.
  
  Keeping the issue open until stefan6419846can confirm whether or not it works for them
  

On the Jenkins infra side: we do not see any error when retrieving the UC from our 2.476 Jenkins controllers. New files are published.

Keeping the issue open until stefan6419846can confirm whether or not it works for them

[jenkins-infra-bot]

stefan6419846:

• Original comment link
• Raw content of original comment:

  I just logged into the system/machine again and I am still seeing this issue. Is there any caching going on or something which would fix itself after a server restart? (I will probably try again tomorrow when I would be able to reboot the instance as well - at the moment the build queue is full and thus I do not want to reboot it directly.)
  

I just logged into the system/machine again and I am still seeing this issue. Is there any caching going on or something which would fix itself after a server restart? (I will probably try again tomorrow when I would be able to reboot the instance as well - at the moment the build queue is full and thus I do not want to reboot it directly.)

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  stefan6419846Jenkins caches the file for 24 hours, unless you restart OR you tell it to check: Manage Jenkins -> Plugins -> click on the grey button with the rounded arrow on top right, next to "Update" blue button.
  

stefan6419846Jenkins caches the file for 24 hours, unless you restart OR you tell it to check: Manage Jenkins -> Plugins -> click on the grey button with the rounded arrow on top right, next to "Update" blue button.

[jenkins-infra-bot]

stefan6419846:

• Original comment link
• Raw content of original comment:

  Then I used the correct button already for retrying. I just restarted the server and it still fails.
  
  Nevertheless, I have been able to further debug this using the script console and a copy of 
  
    
  
      
  
        jenkins/core/src/main/java/hudson/model/DownloadService.java
      
      
  
          Lines 116 to 132
        in
        65f374c
      
    
    
  
      
  
  
          
  
            
  
            
   public static String loadJSON(URL src) throws IOException { 
          
  
          
  
            
  
            
       URLConnection con = ProxyConfiguration.open(src); 
          
  
          
  
            
  
            
       if (con instanceof HttpURLConnection) { 
          
  
          
  
            
  
            
           // prevent problems from misbehaving plugins disabling redirects by default 
          
  
          
  
            
  
            
           ((HttpURLConnection) con).setInstanceFollowRedirects(true); 
          
  
          
  
            
  
            
       } 
          
  
          
  
            
  
            
       try (InputStream is = con.getInputStream()) { 
          
  
          
  
            
  
            
           String jsonp = IOUtils.toString(is, StandardCharsets.UTF_8); 
          
  
          
  
            
  
            
           int start = jsonp.indexOf('{'); 
          
  
          
  
            
  
            
           int end = jsonp.lastIndexOf('}'); 
          
  
          
  
            
  
            
           if (start >= 0 && end > start) { 
          
  
          
  
            
  
            
               return jsonp.substring(start, end + 1); 
          
  
          
  
            
  
            
           } else { 
          
  
          
  
            
  
            
               throw new IOException("Could not find JSON in " + src); 
          
  
          
  
            
  
            
           } 
          
  
          
  
            
  
            
       } 
          
  
          
  
            
  
            
   } 
          
      
    
  
   and apparently somehow the redirect does not work correctly.
  
  
  
  import org.apache.commons.io.IOUtils;
  import java.nio.charset.StandardCharsets;
  URLConnection con = ProxyConfiguration.open(new URL("http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476"));
  if (con instanceof HttpURLConnection) {
      // prevent problems from misbehaving plugins disabling redirects by default
      ((HttpURLConnection) con).setInstanceFollowRedirects(true);
  }
  InputStream is = con.getInputStream()
  String result = IOUtils.toString(is, StandardCharsets.UTF_8);
  println result
  
  
  
  <html>
  <head><title>308 Permanent Redirect</title></head>
  <body>
  <center><h1>308 Permanent Redirect</h1></center>
  <hr><center>nginx</center>
  </body>
  </html> 
  
  
  

Then I used the correct button already for retrying. I just restarted the server and it still fails.

Nevertheless, I have been able to further debug this using the script console and a copy of

jenkins/core/src/main/java/hudson/model/DownloadService.java

Lines 116 to 132 in 65f374c

	public static String loadJSON(URL src) throws IOException {
	URLConnection con = ProxyConfiguration.open(src);
	if (con instanceof HttpURLConnection) {
	// prevent problems from misbehaving plugins disabling redirects by default
	((HttpURLConnection) con).setInstanceFollowRedirects(true);
	}
	try (InputStream is = con.getInputStream()) {
	String jsonp = IOUtils.toString(is, StandardCharsets.UTF_8);
	int start = jsonp.indexOf('{');
	int end = jsonp.lastIndexOf('}');
	if (start >= 0 && end > start) {
	return jsonp.substring(start, end + 1);
	} else {
	throw new IOException("Could not find JSON in " + src);
	}
	}
	}

and apparently somehow the redirect does not work correctly.

import org.apache.commons.io.IOUtils;
import java.nio.charset.StandardCharsets;
URLConnection con = ProxyConfiguration.open(new URL("http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476"));
if (con instanceof HttpURLConnection) {
    // prevent problems from misbehaving plugins disabling redirects by default
    ((HttpURLConnection) con).setInstanceFollowRedirects(true);
}
InputStream is = con.getInputStream()
String result = IOUtils.toString(is, StandardCharsets.UTF_8);
println result

<title>308 Permanent Redirect</title>

308 Permanent Redirect
nginx

 

[jenkins-infra-bot]

stefan6419846:

• Original comment link
• Raw content of original comment:

  I most likely have found the issue: HttpUrlConnection will not upgrade the connection from HTTP to HTTPS and thus is being stuck. I have been able to verify this using curl:
  
  stefan@localhost:~> curl -I "http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476"
  HTTP/1.1 308 Permanent Redirect
  Date: Thu, 12 Sep 2024 10:29:33 GMT
  Content-Type: text/html
  Content-Length: 164
  Connection: keep-alive
  Location: https://updates.jenkins-ci.org/update-center.json?id=default&version=2.476
  
  
  Corresponding JDK issue: https://bugs.openjdk.org/browse/JDK-4620571?focusedId=12159233&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-12159233
  I understand that it is desirable to do everything over HTTPS, but apparently this has not been enforced with the previous update site and some older Jenkins configurations break now. Judging from doing some git blames, everyone using the same Jenkins (base) configuration since version < 2.77 (released September 3, 2017) is affected. (References: c709b19 https://www.jenkins.io/changelog/2.77/)
  

I most likely have found the issue: HttpUrlConnection will not upgrade the connection from HTTP to HTTPS and thus is being stuck. I have been able to verify this using curl:

stefan@​localhost:~> curl -I "http://updates.jenkins-ci.org/update-center.json?id=default&version=2.476"
HTTP/1.1 308 Permanent Redirect
Date: Thu, 12 Sep 2024 10:29:33 GMT
Content-Type: text/html
Content-Length: 164
Connection: keep-alive
Location: https://updates.jenkins-ci.org/update-center.json?id=default&version=2.476

Corresponding JDK issue: https://bugs.openjdk.org/browse/JDK-4620571?focusedId=12159233&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-12159233

I understand that it is desirable to do everything over HTTPS, but apparently this has not been enforced with the previous update site and some older Jenkins configurations break now. Judging from doing some git blames, everyone using the same Jenkins (base) configuration since version < 2.77 (released September 3, 2017) is affected. (References: c709b19 https://www.jenkins.io/changelog/2.77/)

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  Hi stefan6419846thanks for the analysis and feedbacks!
  
  May I ask where does the `http` URL comes from? I've started a fresh Jenkins 2.476 (with the Docker image) and it's set up by default to use `https://updates.jenkins.io` which works as expected.
  
  I guess you're using a custom UC configuration? Or an old or specific setup?
  
  I'm asking because Jenkins default to HTTPS since a few years so I'm not sure to catch the problem in your setup.
  
  In any case: switch to using HTTPS and the domain updates.jenkins.io.
  
  For reference: some users also had the same issue in https://groups.google.com/g/jenkinsci-dev/c/1XsxkwELTBI
  

Hi stefan6419846thanks for the analysis and feedbacks!

May I ask where does the `http` URL comes from? I've started a fresh Jenkins 2.476 (with the Docker image) and it's set up by default to use `https://updates.jenkins.io` which works as expected.

I guess you're using a custom UC configuration? Or an old or specific setup?

I'm asking because Jenkins default to HTTPS since a few years so I'm not sure to catch the problem in your setup.

In any case: switch to using HTTPS and the domain updates.jenkins.io.

For reference: some users also had the same issue in https://groups.google.com/g/jenkinsci-dev/c/1XsxkwELTBI

[jenkins-infra-bot]

stefan6419846:

• Original comment link
• Raw content of original comment:

  This seems to be the old default setup for Jenkins versions up to 2.76 - see the commit linked in my previous comment. In my specific case, we have been using the same (base) configuration for over ten years now and apparently this has never been updated by us or migrated automatically as there has been no need to do so until now.
  

This seems to be the old default setup for Jenkins versions up to 2.76 - see the commit linked in my previous comment. In my specific case, we have been using the same (base) configuration for over ten years now and apparently this has never been updated by us or migrated automatically as there has been no need to do so until now.

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  Oooh good catch! Looks like we did not took in account the scenario "inherited from old Jenkins setups and not migrated by the controller upgrade". Many thanks for your help!
  

Oooh good catch! Looks like we did not took in account the scenario "inherited from old Jenkins setups and not migrated by the controller upgrade". Many thanks for your help!

[jenkins-infra-bot]

lppedd:

• Original comment link
• Raw content of original comment:

  This is what I see when I open the update page, on 2.462.2.
  
  
  

This is what I see when I open the update page, on 2.462.2.

[jenkins-infra-bot]

stefan6419846:

• Original comment link
• Raw content of original comment:

  Going to "Manage Jenkins" > "Plugins" > "Advanced settings" and clicking the "Reset" link in the "Update site" section successfully fixed this for me by automatically updating the outdated URL.
  

Going to "Manage Jenkins" > "Plugins" > "Advanced settings" and clicking the "Reset" link in the "Update site" section successfully fixed this for me by automatically updating the outdated URL.

[jenkins-infra-bot]

dduportal:

• Original comment link
• Raw content of original comment:

  Hi lppedd thanks, we are aware of this issue. As described by Stefan below, your Jenkins controller did carry over a really old configuration. If you set it up to https://updates.jenkins.io, it will work as expected. 
  

Hi lppedd thanks, we are aware of this issue. As described by Stefan below, your Jenkins controller did carry over a really old configuration. If you set it up to https://updates.jenkins.io, it will work as expected.

[jenkins-infra-bot]

lppedd:

• Original comment link
• Raw content of original comment:

  stefan6419846 my URL was set to http://updates.jenkins-ci.org/update-center.json, and resetting it didn't change it. Still see the issue.
  

stefan6419846 my URL was set to http://updates.jenkins-ci.org/update-center.json, and resetting it didn't change it. Still see the issue.