Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] First stab at adding PCT ITs for the Docker image #133

Closed
wants to merge 8 commits into from

Conversation

@batmat
Copy link
Member

commented Mar 29, 2019

No description provided.

batmat added 2 commits Mar 29, 2019
:facepalm:
sh 'make docker'
}

stage("Run known successful case(s)") {

This comment has been minimized.

Copy link
@batmat

batmat Mar 29, 2019

Author Member

TODO: prepare already more a parallel data structure, to run many variations in parallel, so it's easy to add more as we go: a must-fail one, etc.

This comment has been minimized.

Copy link
@batmat

batmat Apr 1, 2019

Author Member

TODO 2:

  • download a Jenkins WAR and map it, so the testing stability is not subject to the new available weeklies
batmat added 5 commits Apr 1, 2019
(not used yet in PCT run though)
This reverts commit 7d59f12, reversing
changes made to 661782e.
@batmat

This comment has been minimized.

Copy link
Member Author

commented Apr 3, 2019

AFAICT, now the failure we're seeing in the exact issue I'm hitting locally that I fixed reverting a previous commit.

https://ci.jenkins.io/blue/rest/organizations/jenkins/pipelines/jenkinsci-libraries/pipelines/plugin-compat-tester/branches/PR-133/runs/6/nodes/95/log/?start=0

+ pwd
+ pwd
+ docker run --rm -v /mnt/agent-workspace/workspace/ries_plugin-compat-tester_PR-133/jenkins.war:/pct/jenkins.war:ro -v /mnt/agent-workspace/workspace/ries_plugin-compat-tester_PR-133/out:/pct/out -e JDK_VERSION=11 -e ARTIFACT_ID=buildtriggerbadge -e VERSION=buildtriggerbadge-2.10 jenkins/pct
Running PCT for plugin buildtriggerbadge
+ '[' 0 -eq 1 ']'
+ '[' -n buildtriggerbadge ']'
+ echo 'Running PCT for plugin buildtriggerbadge'
+ CUSTOM_MAVEN_SETTINGS=/pct/m2-settings.xml
+ FAIL_ON_ERROR_ARG=
+ '[' -n '' ']'
+ FAIL_ON_ERROR_ARG=-failOnError
+ '[' -f /pct/m2-settings.xml ']'
+ MVN_SETTINGS_FILE=/pct/default-m2-settings.xml
+ '[' -n '' ']'
+ '[' -z buildtriggerbadge ']'
+ CHECKOUT_SRC=https://github.com/jenkinsci/buildtriggerbadge-plugin.git
+ '[' -f /pct/jenkins.war ']'
+ mkdir -p /pct/tmp
+ cp /pct/jenkins.war /pct/tmp/jenkins.war
+ JENKINS_WAR_PATH=/pct/tmp/jenkins.war
+ WAR_PATH_OPT='-war /pct/tmp/jenkins.war '
++ groovy /pct/scripts/readJenkinsVersion.groovy /pct/tmp/jenkins.war
Using custom Jenkins WAR v. 2.164.1 from /pct/tmp/jenkins.war
+ JENKINS_VERSION=2.164.1
+ echo 'Using custom Jenkins WAR v. 2.164.1 from /pct/tmp/jenkins.war'
+ [[ 2.164.1 =~ .*SNAPSHOT.* ]]
+ SHOULD_CHECKOUT=1
+ '[' -z buildtriggerbadge-2.10 ']'
+ echo 'Will be testing with buildtriggerbadge:buildtriggerbadge-2.10, shouldCheckout=1'
+ extra_java_opts=()
+ [[ -n '' ]]
+ LOCAL_CHECKOUT_ARG=
Will be testing with buildtriggerbadge:buildtriggerbadge-2.10, shouldCheckout=1
+ '[' 1 -eq 1 ']'
+ '[' -z '' ']'
+ mkdir -p /pct/tmp/localCheckoutDir
+ cd /pct/tmp/localCheckoutDir
+ TMP_CHECKOUT_DIR=/pct/tmp/localCheckoutDir/undefined
+ '[' -e /pct/plugin-src/pom.xml ']'
+ echo 'Checking out from https://github.com/jenkinsci/buildtriggerbadge-plugin.git:buildtriggerbadge-2.10'
+ git clone https://github.com/jenkinsci/buildtriggerbadge-plugin.git
Checking out from https://github.com/jenkinsci/buildtriggerbadge-plugin.git:buildtriggerbadge-2.10
Cloning into 'buildtriggerbadge-plugin'...
++ ls .
+ mv buildtriggerbadge-plugin /pct/tmp/localCheckoutDir/undefined
+ cd /pct/tmp/localCheckoutDir/undefined
+ git checkout buildtriggerbadge-2.10
Note: checking out 'buildtriggerbadge-2.10'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at cebb6b7... [maven-release-plugin] prepare release buildtriggerbadge-2.10
+ cd /pct/tmp/localCheckoutDir/undefined
+ '[' -z buildtriggerbadge ']'
+ mv /pct/tmp/localCheckoutDir/undefined /pct/tmp/localCheckoutDir/buildtriggerbadge
+ LOCAL_CHECKOUT_ARG='-localCheckoutDir /pct/tmp/localCheckoutDir/buildtriggerbadge'
+ mkdir -p /pct/tmp/work
+ mkdir -p /pct/out
+ TEST_JDK_HOME=/usr/lib/jvm/java-11-openjdk-amd64
+ TEST_JAVA_ARGS=''\'' -Xmx768M -Djava.awt.headless=true -Djdk.net.URLClassPath.disableClassPathURLCheck=true'\'''
+ pctExitCode=0
+ echo java -jar /pct/pct-cli.jar -reportFile /pct/out/pct-report.xml -workDirectory /pct/tmp/work -war /pct/tmp/jenkins.war -skipTestCache true -failOnError -localCheckoutDir /pct/tmp/localCheckoutDir/buildtriggerbadge -includePlugins buildtriggerbadge -mvn /usr/bin/mvn -m2SettingsFile /pct/default-m2-settings.xml -testJDKHome /usr/lib/jvm/java-11-openjdk-amd64 -testJavaArgs \' -Xmx768M -Djava.awt.headless=true '-Djdk.net.URLClassPath.disableClassPathURLCheck=true'\''' '|| echo $? > /pct/tmp/pct_exit_code'
+ chmod +x /pct/tmp/pct_command
+ cat /pct/tmp/pct_command
java -jar /pct/pct-cli.jar -reportFile /pct/out/pct-report.xml -workDirectory /pct/tmp/work -war /pct/tmp/jenkins.war -skipTestCache true -failOnError -localCheckoutDir /pct/tmp/localCheckoutDir/buildtriggerbadge -includePlugins buildtriggerbadge -mvn /usr/bin/mvn -m2SettingsFile /pct/default-m2-settings.xml -testJDKHome /usr/lib/jvm/java-11-openjdk-amd64 -testJavaArgs ' -Xmx768M -Djava.awt.headless=true -Djdk.net.URLClassPath.disableClassPathURLCheck=true' || echo $? > /pct/tmp/pct_exit_code
+ sh -ex /pct/tmp/pct_command
+ java -jar /pct/pct-cli.jar -reportFile /pct/out/pct-report.xml -workDirectory /pct/tmp/work -war /pct/tmp/jenkins.war -skipTestCache true -failOnError -localCheckoutDir /pct/tmp/localCheckoutDir/buildtriggerbadge -includePlugins buildtriggerbadge -mvn /usr/bin/mvn -m2SettingsFile /pct/default-m2-settings.xml -testJDKHome /usr/lib/jvm/java-11-openjdk-amd64 -testJavaArgs  -Xmx768M -Djava.awt.headless=true -Djdk.net.URLClassPath.disableClassPathURLCheck=true
Checking /pct/tmp/jenkins.war for plugin split metadata…
Checking jdk version as splits may depend on a jdk version
found split cycles: [script-security jaxb, cvs jaxb, script-security windows-slaves, javadoc jaxb, mailer jaxb, credentials matrix-auth, jdk-tool jaxb, bouncycastle-api jaxb, script-security bouncycastle-api, matrix-project jaxb, junit jaxb, subversion jaxb, ant jaxb, ldap jaxb, antisamy-markup-formatter jaxb, script-security matrix-auth, maven-plugin jaxb, external-monitor-job jaxb, script-security matrix-project, pam-auth jaxb, matrix-auth jaxb, script-security command-launcher, credentials windows-slaves, windows-slaves jaxb, command-launcher jaxb, script-security antisamy-markup-formatter]
Not adding jaxb 2.163 2.3.0 11 as split because jdk specified 11 is newer than running jdk 11
found splits: [maven-plugin 1.296 1.296, subversion 1.310 1.0, cvs 1.340 0.1, ant 1.430 1.0, javadoc 1.430 1.0, external-monitor-job 1.467 1.0, ldap 1.467 1.0, pam-auth 1.467 1.0, mailer 1.493 1.2, matrix-auth 1.535 1.0.2, windows-slaves 1.547 1.0, antisamy-markup-formatter 1.553 1.0, matrix-project 1.561 1.0, junit 1.577 1.0, bouncycastle-api 2.16 2.16.0, command-launcher 2.86 1.0, jdk-tool 2.112 1.0]
17:04:48.404 [main] DEBUG org.reflections.Reflections - going to scan these urls:
jar:file:/pct/pct-cli.jar!/
17:04:48.524 [main] INFO  org.reflections.Reflections - Reflections took 116 ms to scan 1 urls, producing 40 keys and 78 values 
Hook: org.jenkins.tools.test.hook.BlueOceanHook
Hook: org.jenkins.tools.test.hook.DeclarativePipelineHook
Hook: org.jenkins.tools.test.hook.StructsHook
Hook: org.jenkins.tools.test.hook.SkipUIHelperPlugins
17:04:48.536 [main] DEBUG org.reflections.Reflections - going to scan these urls:
jar:file:/pct/pct-cli.jar!/
17:04:48.608 [main] INFO  org.reflections.Reflections - Reflections took 71 ms to scan 1 urls, producing 40 keys and 78 values 
Hook: org.jenkins.tools.test.hook.TransformPom
Loaded TransformPom
17:04:48.610 [main] DEBUG org.reflections.Reflections - going to scan these urls:
jar:file:/pct/pct-cli.jar!/
17:04:48.665 [main] INFO  org.reflections.Reflections - Reflections took 55 ms to scan 1 urls, producing 40 keys and 78 values 
Hook: org.jenkins.tools.test.hook.MultiParentCompileHook
Loaded multi-parent compile hook
Scanned contents of /pct/tmp/jenkins.war: {"id":"default","core":{"name":"core","version":"2.164.1","url":"https://foobar"},"plugins":{}}
Scanned contents of /pct/tmp/jenkins.war: {"id":"default","core":{"name":"core","version":"2.164.1","url":"https://foobar"},"plugins":{"subversion":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/subversion.hpi","name":"subversion","version":"1.54","dependencies":[]},"pam-auth":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/pam-auth.hpi","name":"pam-auth","version":"1.1","dependencies":[]},"external-monitor-job":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/external-monitor-job.hpi","name":"external-monitor-job","version":"1.4","dependencies":[]},"translation":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/translation.hpi","name":"translation","version":"1.10","dependencies":[]},"command-launcher":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/command-launcher.hpi","name":"command-launcher","version":"1.0","dependencies":[{"name":"script-security","version":"1.18.1","optional":"false"}]},"ldap":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/ldap.hpi","name":"ldap","version":"1.11","dependencies":[{"name":"mailer","version":"1.8","optional":"false"}]},"junit":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/junit.hpi","name":"junit","version":"1.6","dependencies":[]},"credentials":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/credentials.hpi","name":"credentials","version":"2.1.2","dependencies":[]},"windows-slaves":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/windows-slaves.hpi","name":"windows-slaves","version":"1.0","dependencies":[]},"matrix-auth":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/matrix-auth.hpi","name":"matrix-auth","version":"1.1","dependencies":[]},"ssh-slaves":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/ssh-slaves.hpi","name":"ssh-slaves","version":"1.15","dependencies":[{"name":"credentials","version":"2.1.2","optional":"false"},{"name":"ssh-credentials","version":"1.6.1","optional":"false"}]},"ssh-credentials":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/ssh-credentials.hpi","name":"ssh-credentials","version":"1.10","dependencies":[{"name":"credentials","version":"1.16.1","optional":"false"}]},"mailer":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/mailer.hpi","name":"mailer","version":"1.20","dependencies":[{"name":"display-url-api","version":"1.0","optional":"false"}]},"ant":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/ant.hpi","name":"ant","version":"1.2","dependencies":[]},"display-url-api":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/display-url-api.hpi","name":"display-url-api","version":"2.0","dependencies":[]},"matrix-project":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/matrix-project.hpi","name":"matrix-project","version":"1.4.1","dependencies":[{"name":"script-security","version":"1.13","optional":"false"}]},"jdk-tool":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/jdk-tool.hpi","name":"jdk-tool","version":"1.0","dependencies":[]},"bouncycastle-api":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/bouncycastle-api.hpi","name":"bouncycastle-api","version":"2.16.0","dependencies":[]},"javadoc":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/javadoc.hpi","name":"javadoc","version":"1.1","dependencies":[]},"cvs":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/cvs.hpi","name":"cvs","version":"2.11","dependencies":[]},"script-security":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/script-security.hpi","name":"script-security","version":"1.18.1","dependencies":[]},"maven-plugin":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/maven-plugin.hpi","name":"maven-plugin","version":"2.14","dependencies":[{"name":"javadoc","version":"1.0","optional":"false"},{"name":"junit","version":"1.6","optional":"false"},{"name":"mailer","version":"1.7","optional":"false"},{"name":"token-macro","version":"1.1","optional":"true"}]},"antisamy-markup-formatter":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/antisamy-markup-formatter.hpi","name":"antisamy-markup-formatter","version":"1.1","dependencies":[]},"jaxb":{"url":"jar:file:/pct/tmp/jenkins.war!/WEB-INF/detached-plugins/jaxb.hpi","name":"jaxb","version":"2.3.0","dependencies":[]}}}
Using custom Test JDK home: /usr/lib/jvm/java-11-openjdk-amd64
Starting plugin tests on core coordinates : MavenCoordinates[groupId=org.jenkins-ci.plugins, artifactId=plugin, version=2.164.1]
Plugin ant not in included plugins => test skipped !
Plugin antisamy-markup-formatter not in included plugins => test skipped !
Plugin bouncycastle-api not in included plugins => test skipped !
Plugin command-launcher not in included plugins => test skipped !
Plugin credentials not in included plugins => test skipped !
Plugin cvs not in included plugins => test skipped !
Plugin display-url-api not in included plugins => test skipped !
Plugin external-monitor-job not in included plugins => test skipped !
Plugin javadoc not in included plugins => test skipped !
Plugin jaxb not in included plugins => test skipped !
Plugin jdk-tool not in included plugins => test skipped !
Plugin junit not in included plugins => test skipped !
Plugin ldap not in included plugins => test skipped !
Plugin mailer not in included plugins => test skipped !
Plugin matrix-auth not in included plugins => test skipped !
Plugin matrix-project not in included plugins => test skipped !
Plugin maven-plugin not in included plugins => test skipped !
Plugin pam-auth not in included plugins => test skipped !
Plugin script-security not in included plugins => test skipped !
Plugin ssh-credentials not in included plugins => test skipped !
Plugin ssh-slaves not in included plugins => test skipped !
Plugin subversion not in included plugins => test skipped !
Plugin translation not in included plugins => test skipped !
Plugin windows-slaves not in included plugins => test skipped !
No HTML report is generated, because it has been disabled or no tests have been executed
+ '[' -f /pct/tmp/pct_exit_code ']'
+ [[ 0 != \0 ]]
Archiving artifacts
+ cat out/pct-report.html
+ grep Tests : Success
cat: out/pct-report.html: No such file or directory
script returned exit code 1

I.e. no test at all is run.

cc @alecharp who I think saw the same as I did.

@batmat batmat closed this Apr 6, 2019
@batmat batmat reopened this Apr 6, 2019
@batmat

This comment has been minimized.

Copy link
Member Author

commented Apr 6, 2019

Last failure was a test failure, and seems like the newly added IT was actually successful.
I strongly suspect this is a similar failure to the one recently fixed on jenkinsci/jenkins#3962

image

@batmat batmat closed this Apr 6, 2019
@batmat batmat reopened this Apr 6, 2019
@batmat

This comment has been minimized.

Copy link
Member Author

commented Apr 7, 2019

Successful after rebuild, like I was hoping for.

@batmat

This comment has been minimized.

Copy link
Member Author

commented Apr 19, 2019

Superseded, cf. #140

@batmat batmat closed this Apr 19, 2019
@batmat batmat deleted the batmat:add-ITs branch Apr 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.