Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul update site generation #365

Merged
merged 169 commits into from
May 23, 2020

Conversation

daniel-beck
Copy link
Contributor

@daniel-beck daniel-beck commented Apr 11, 2020

Large-scale modernization of this tool.

Notable output changes:

  • A few more releases blacklisted that were identified as ambiguous releases
    • aliyun-container-service-deploy has all of its releases suspended now, so it's gone from update sites.
  • Remove https://updates.jenkins.io/temporary-experimental-java11/
  • We only generate correct_* checksums and signatures, not the pre-1.433 variant. This means update center output is no longer compatible with Jenkins 1.433 (late 2011)
  • No longer generate the "fallback" htaccess output.
  • Individual tiered update sites no longer generate latest/ directory, or plugin-documentation-urls.json.
  • Experimental update site no longer generates latest/ directory, plugin-documentation-urls.json, or latestCore.txt.
  • This is now unaffected by the problem causing https://updates.jenkins.io/download/plugins/neoload-jenkins-plugin to not exist (INFRA-2296).

While we remove some redundancy, I checked what files are required for things to continue working, and they are still being created:

Other changes:

  • Update tons of dependencies and parent POM
  • Use fastjson instead of patched json-lib
  • Better docs
  • Blacklist ambiguous releases
  • Filter plugins/versions for malformed version numbers (e.g. spaces, ASCII control chars)
  • Separate tool building/releasing and script execution, move resources out of tool
  • Remove unused fallback creation
  • Remove "Nexus index" mode, completely wipe the dependency to those outdated libs
  • Require authenticated GitHub use, no more unauthenticated fallback
  • add new whitelist mode

It's running experimentally at https://trusted.ci.jenkins.io:1443/job/update-center2 from the origin branch with the same name, and completes a full run (except publishing/rsync) in ~45 seconds.

Copy link
Contributor

@oleg-nenashev oleg-nenashev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see anything wrong in this pull request. I like the new structure , and I agree with all removals. +1 for getting it merged.

@@ -239,8 +248,8 @@
</repositories>

<scm>
<connection>scm:git:git://github.com/jenkinsci/backend-update-center2.git</connection>
<developerConnection>scm:git:ssh://git@github.com/jenkinsci/backend-update-center2.git</developerConnection>
<connection>scm:git:git://github.com/jenkins-infra/update-center2.git</connection>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be 'update-center3' now? :)

README.adoc Outdated Show resolved Hide resolved
@daniel-beck
Copy link
Contributor Author

@daniel-beck daniel-beck force-pushed the daniel-beck:mavenless-plus branch from 4b416e0 to 6e3e332 9 minutes ago

FTR no need to panic:

Subject: EMAILADDRESS=nobody@example.invalid, CN=example.org, OU=LocalDev, O=Jenkins, C=DE
Issuer: EMAILADDRESS=nobody@example.invalid, CN=example.org, OU=LocalDev, O=Jenkins, C=DE

It looks like my testing of fastjson output compatibility was incomplete
and the library generates float numbers different from what Jenkins core
writes out as the 'canonical' form (json-lib), resulting in checksum
errors.

Very small numbers are written in scientific notation in both json-lib
and fastjson, but fastjson uses 'E' while json-lib uses 'e'.

Floats without fractional part are written like ints by json-lib, while
fastjson writes them with '.0' suffix.

This basically means that, without switching out the serialization
library, we're unable to have floats in update center data.
Especially now that the wiki is pretty much gone
@daniel-beck daniel-beck merged commit d053b58 into jenkins-infra:master May 23, 2020
@daniel-beck
Copy link
Contributor Author

Oleg mentioned a preference for this to be squashed, so that's how I merged it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants