From 5db52412024919f032922e4b133b7bdc9535cc32 Mon Sep 17 00:00:00 2001 From: Philipp Pixel Date: Wed, 11 Sep 2019 14:57:38 +0200 Subject: [PATCH 1/5] #4 Support automatic Maven-settings.xml, support truststore # withMavenSettings The function `withMavenSettings() { -> settingsXml }` provides an easy- to-use functionality which writes a settings file and cleans it up afterwards. # truststore Sharing a truststore is a fickle thing because it contains crucial certificates. Once a build job finishes it is unnecessarily hard to remove truststore files. For this reason `copy()` and `use()` exist so that the files are removed on different Jenkins nodes within the Job build workspace. --- README.md | 44 ++++++++++++++++++++ vars/helper.groovy | 13 ++++++ vars/truststore.groovy | 48 ++++++++++++++++++++++ vars/withMavenSettings.groovy | 75 +++++++++++++++++++++++++++++++++++ 4 files changed, 180 insertions(+) create mode 100644 vars/helper.groovy create mode 100644 vars/truststore.groovy create mode 100644 vars/withMavenSettings.groovy diff --git a/README.md b/README.md index 0c10132..daa39c1 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,50 @@ It is possible (although not necessary) to explicitly work with docker networks. ``` +## Working with Truststores + +Often comes a Truststore into play while working with Jenkins and Java. Jenkins can accommodate necessary certificates in its truststore so Java applications like Maven (and others too!) can successfully interact with other parties, like download artifacts from artifact repositories or transport data over the network. Even so, it may be necessary to provide these Java applications with the right certificates when otherwise encrypted communication would fail without doing so. + +## Simple Truststore pipeline + +For such circumstances this library provides a small snippet. + +``` +Library ('zalenium-build-lib') import com.cloudogu.Truststore + +def truststore = new com.cloudogu.Truststore(this) + +node('master') { + truststore.copy() // by default Jenkin's truststore is copied to the workspace +} +node('anotherOne') { + //use the truststore + //javaAppRun -Djavax.net.ssl.trustStore=./truststore.jks -Djavax.net.ssl.trustStorePassword=changeit + + truststore.remove() // remove the truststore for added security +} +``` + +## Alternative Ways of Configuration + +It is possible to supply a different truststore than the Jenkins one. Also it is possible to provide a different name in order to avoid filename collision: + + ``` + Library ('zalenium-build-lib') import com.cloudogu.Truststore + + def truststore = new com.cloudogu.Truststore(this, '/path/to/alternative/truststore.jks', 'mytruststore-alpha.jks') + + node('master') { + truststore.copy() + } + node('anotherOne') { + //use the truststore + //javaAppRun -Djavax.net.ssl.trustStore=./mytruststore-alpha.jks -Djavax.net.ssl.trustStorePassword=thatOtherPassword + + truststore.remove() + } + ``` + ## Locking Right now, only one Job can run Zalenium Tests at a time. diff --git a/vars/helper.groovy b/vars/helper.groovy new file mode 100644 index 0000000..cdb7dbd --- /dev/null +++ b/vars/helper.groovy @@ -0,0 +1,13 @@ +String generateZaleniumJobName() { + return "${JOB_BASE_NAME}_${BUILD_NUMBER}" +} + +String findHostName() { + String regexMatchesHostName = 'https?://([^:/]*)' + + // Storing matcher in a variable might lead to java.io.NotSerializableException: java.util.regex.Matcher + if (!(env.JENKINS_URL =~ regexMatchesHostName)) { + script.error 'Unable to determine hostname from env.JENKINS_URL. Expecting http(s)://server:port/jenkins' + } + return (env.JENKINS_URL =~ regexMatchesHostName)[0][1] +} \ No newline at end of file diff --git a/vars/truststore.groovy b/vars/truststore.groovy new file mode 100644 index 0000000..b0370c4 --- /dev/null +++ b/vars/truststore.groovy @@ -0,0 +1,48 @@ +/** + * Provides a simple way to use a Java trust store over different Jenkins nodes. + * + * @param pathToTruststore provides the path to a Java truststore. It defaults to the truststore on the Jenkins master. + */ +def copy(String pathToTruststore = '/var/lib/jenkins/truststore.jks') { + String truststoreFile = 'truststore.jks' + String stashName = 'truststore' + + sh "cp ${pathToTruststore} ./${truststoreFile}" + stash includes: truststoreFile, name: stashName + sh "rm -f ./${truststoreFile}" +} + +/** + * Unstashes the previously stashed truststore to the current workspace. The truststore will be deleted after the closure + * body finished. + * + * truststore.copy() MUST be called beforehand, otherwise there would be no truststore to be unstashed. + * + * 
+ *     node 'master' {
+ *      truststore.copy()
+ *     }
+ *     node 'docker' {
+ *         truststore.use { 
+ *             javaOrMvn "-Djavax.net.ssl.trustStore=truststore.jks -Djavax.net.ssl.trustStorePassword=changeit"
+ *         }
+ *     }
+ *
+ * @param closure this closure is executed after the truststore was successfully unstashed. + */ +def use(Closure inner) { + String truststoreFile = 'truststore.jks' + String stashName = 'truststore' + + try { + unstash name: stashName + + inner.call(truststoreFile) + + } catch (Exception ex) { + echo "withTruststore failed because an exception occurred: ${ex}" + throw ex + } finally { + sh "rm -f ./${truststoreFile}" + } +} \ No newline at end of file diff --git a/vars/withMavenSettings.groovy b/vars/withMavenSettings.groovy new file mode 100644 index 0000000..935352a --- /dev/null +++ b/vars/withMavenSettings.groovy @@ -0,0 +1,75 @@ +/** + * Provides a simple custom maven settings.xml file to current working directory with Maven Central mirror in the + * current CES instance. + * + * Example call: + * + * 
+ * nexusCreds = usernamePassword(credentialsId: 'jenkinsNexusServiceUser', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME')
+ * withMavenSettings(nexusCreds, 'cesinstance.stage.host.tld', '/usr/share/maven') { settingsXml ->
+ *     // do your maven call here
+ *     mvn "-s ${settingsXml} test"
+ * } // settings.xml will be removed automatically
+ *
+ * + * @param nexusCredentials Jenkins credentials which provide USERNAME and PASSWORD to an account which enables Nexus interaction + * @param cesFQDN the full qualified domain name of the current CES instance, f. i. cesinstance.stage.host.tld + * @param pathToLocalMavenRepository without the .m2 directory part, f. i. /usr/share/maven. The suffix /.m2/repository will be added automatically. + */ +def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenRepository, Closure closure) { + echo "write settings.xml to ${pathToLocalMavenRepository}" + String settingsXml = "settings.xml" + withCredentials([nexusCredentials]) { + writeFile file: settingsXml, text: """ + + ${pathToLocalMavenRepository}/.m2/repository + + + ${cesFQDN} + ${USERNAME} + ${PASSWORD} + + + + + ${cesFQDN} + ${cesFQDN} Central Mirror + https://${cesFQDN}/nexus/repository/itzbundshared/ + central + + + """ + } + + try { + closure.call(settingsXml) + } finally { + sh "rm -f ${settingsXml}" + } +} + +def mvnWithSettings(def nexusCredentials, String cesFQDN, String mvnCallArgs) { + def currentHome = env.HOME + settings(nexusCredentials, cesFQDN, currentHome) { settingsXml -> + mvnPure settingsXml, mvnCallArgs + } +} + +/** + * This method extracts the Maven 3 installation from Jenkins and calls Maven with the given settings.xml and Maven arguments + */ +def mvn(String settingsXml, String mvnCallArgs) { + def mvnHome = tool 'M3' + + mvnWithHome(mvnHome, settingsXml, mvnCallArgs) +} + +def customMvnWithSettings(def nexusCredentials, String cesFQDN, String mvnHome, String pathToLocalMavenRepository, String mvnCallArgs) { + settings(nexusCredentials, cesFQDN, pathToLocalMavenRepository) { settingsXml -> + mvnWithHome(mvnHome, settingsXml, mvnCallArgs) + } +} + +def mvnWithHome(String mvnHome, String settingsXml, String mvnCallArgs) { + sh "${mvnHome}/bin/mvn -s ${settingsXml} --batch-mode -V -U -e -Dsurefire.useFile=false ${mvnCallArgs}" +} \ No newline at end of file From 57ebf55787f93bc9b0c86361a7ba2e121c1eff31 Mon Sep 17 00:00:00 2001 From: Philipp Pixel Date: Mon, 16 Sep 2019 15:30:51 +0200 Subject: [PATCH 2/5] #4 add documentation, remove duplicate documentation --- README.md | 51 +++++++++++++++++++------------------- vars/helper.txt | 5 ++++ vars/truststore.groovy | 10 -------- vars/truststore.txt | 25 +++++++++++++++++++ vars/withMavenSettings.txt | 47 +++++++++++++++++++++++++++++++++++ 5 files changed, 102 insertions(+), 36 deletions(-) create mode 100644 vars/helper.txt create mode 100644 vars/truststore.txt create mode 100644 vars/withMavenSettings.txt diff --git a/README.md b/README.md index daa39c1..6256c45 100644 --- a/README.md +++ b/README.md @@ -69,21 +69,25 @@ Often comes a Truststore into play while working with Jenkins and Java. Jenkins ## Simple Truststore pipeline -For such circumstances this library provides a small snippet. +For such circumstances this library provides a small snippet. The global `truststore` variable ensures that any truststore files which are copied in the process are also removed at the end of both `copy` and `use` actions. -``` -Library ('zalenium-build-lib') import com.cloudogu.Truststore +In order to successfully provide a truststore to any Java process this sequence must be in order: -def truststore = new com.cloudogu.Truststore(this) +1. copy the truststore with `truststore.copy()` +1. use the copied truststore with `truststore.use { truststoreFile -> }` -node('master') { - truststore.copy() // by default Jenkin's truststore is copied to the workspace +Here is a more elaborate example: + +``` +Library ('zalenium-build-lib') _ + +node 'master' { + truststore.copy() } -node('anotherOne') { - //use the truststore - //javaAppRun -Djavax.net.ssl.trustStore=./truststore.jks -Djavax.net.ssl.trustStorePassword=changeit - - truststore.remove() // remove the truststore for added security +node 'docker' { + truststore.use { truststoreFile -> + javaOrMvn "-Djavax.net.ssl.trustStore=${truststoreFile} -Djavax.net.ssl.trustStorePassword=changeit" + } } ``` @@ -91,21 +95,16 @@ node('anotherOne') { It is possible to supply a different truststore than the Jenkins one. Also it is possible to provide a different name in order to avoid filename collision: - ``` - Library ('zalenium-build-lib') import com.cloudogu.Truststore - - def truststore = new com.cloudogu.Truststore(this, '/path/to/alternative/truststore.jks', 'mytruststore-alpha.jks') - - node('master') { - truststore.copy() - } - node('anotherOne') { - //use the truststore - //javaAppRun -Djavax.net.ssl.trustStore=./mytruststore-alpha.jks -Djavax.net.ssl.trustStorePassword=thatOtherPassword - - truststore.remove() - } - ``` +``` +Library ('zalenium-build-lib') _ + +node('master') { + truststore.copy('/path/to/alternative/truststore/file.jks') +} +node('anotherOne') { + //truststore.use ... as usual +} +``` ## Locking diff --git a/vars/helper.txt b/vars/helper.txt new file mode 100644 index 0000000..743dcd8 --- /dev/null +++ b/vars/helper.txt @@ -0,0 +1,5 @@ +- generateZaleniumJobName() - creates a jobname unique to the current Jenkins job. For the 99th Jenkins build of the job + "ACME" the job name would be "ACME_99". + +- findHostName() - returns the host name of the current Jenkins node. This is usually needed for getting the FQDN of the + Cloudogu EcoSystem instance which runs the Jenkins master node. \ No newline at end of file diff --git a/vars/truststore.groovy b/vars/truststore.groovy index b0370c4..8ae7d10 100644 --- a/vars/truststore.groovy +++ b/vars/truststore.groovy @@ -18,16 +18,6 @@ def copy(String pathToTruststore = '/var/lib/jenkins/truststore.jks') { * * truststore.copy() MUST be called beforehand, otherwise there would be no truststore to be unstashed. * - * 
- *     node 'master' {
- *      truststore.copy()
- *     }
- *     node 'docker' {
- *         truststore.use { 
- *             javaOrMvn "-Djavax.net.ssl.trustStore=truststore.jks -Djavax.net.ssl.trustStorePassword=changeit"
- *         }
- *     }
- *
* @param closure this closure is executed after the truststore was successfully unstashed. */ def use(Closure inner) { diff --git a/vars/truststore.txt b/vars/truststore.txt new file mode 100644 index 0000000..5e22e1f --- /dev/null +++ b/vars/truststore.txt @@ -0,0 +1,25 @@ +Starts copies and provides a truststore files for safe use over different Jenkins nodes. When the build job finishes there +will be no truststore file left in the Jenkins workspace. + +By default Jenkins's truststore will be used: `/var/lib/jenkins/truststore.jks` + +(optional) Parameters: + +- pathToTruststore - This path pointing to the truststore file will be used instead of the default path. + +Method summary: + +- copy() - copies a truststore file and stashed it for later use. +- use { truststoreFile -> } - enables the usage of the previously copied truststore file. The truststore file which + was extracted from the stash will be deleted once the body finishes. + +Exemplary calls: + +node 'master' { + truststore.copy() +} +node 'docker' { + truststore.use { truststoreFile -> + javaOrMvn "-Djavax.net.ssl.trustStore=${truststoreFile} -Djavax.net.ssl.trustStorePassword=changeit" + } +} \ No newline at end of file diff --git a/vars/withMavenSettings.txt b/vars/withMavenSettings.txt new file mode 100644 index 0000000..808c38b --- /dev/null +++ b/vars/withMavenSettings.txt @@ -0,0 +1,47 @@ +provides more sophisticated Maven support for custom `settings.xml` files which refer central maven mirror to in the +context of a Cloudogu EcoSystem. This global variable writes custom `settings.xml` files and can additionally invoke a +provided or custom Maven call. + +Any `settings.xml` being writting will be removed once the settings body is left. + +Once written, the `settings.xml`'s structure will look like this: + +| ++->settings +| +-> localRepository +| `-> servers +| `-> server +| +-> id +| +-> username +| `-> password +| +`-> mirrors + `-> mirror + +-> id (reference to the server id) + +-> name + +-> url + +-> mirrorOf (fixed value of central maven mirror) + +Example call: + +nexusCreds = usernamePassword(credentialsId: 'jenkinsNexusServiceUser', passwordVariable: 'PASSWORD', usernameVariable: 'USERNAME') +withMavenSettings.mvnWithSettings(nexusCreds, 'cesinstance.stage.host.tld', '/usr/share/maven') { settingsXml -> + // do something with the settings.xml here +} // settings.xml will be removed automatically + +Example call with included Maven call: + +withMavenSettings.mvnWithSettings(nexusCreds, 'cesinstance.stage.host.tld', 'yourMavenGoal') + +- settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenRepository, Closure closure) - Writes a + `settings.xml` file with the given Nexus credentials and executes the closure body. This method provides a parameter + `settingsXml` which contains the relative file path to the 'settings.xml' file. + +- mvn(String settingsXml, String mvnCallArgs) - call the Jenkins Tool Maven with a provided `settings.xml` + +- mvnWithSettings(def nexusCredentials, String cesFQDN, String mvnCallArgs) - call Jenkins Tool Maven wrapped within a + settings file. This is basically a convenience mix of `settings` and `mvn` + +- customMvnWithSettings(def nexusCredentials, String cesFQDN, String mvnHome, String pathToLocalMavenRepository, String mvnCallArgs) - + similar to `mvnWithSettings` but calls a custom Maven installation instead. It is necessary to know about the location of + the Maven installation. Calling this method can make sense if the Maven installation resides in a Docker container. \ No newline at end of file From 1890580c50a143230e6be9acc99c57d1fb5e4b45 Mon Sep 17 00:00:00 2001 From: Philipp Pixel Date: Tue, 17 Sep 2019 15:53:36 +0200 Subject: [PATCH 3/5] #4 Rename weird Jenkins node name in example --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 6256c45..2b0fa84 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,7 @@ Library ('zalenium-build-lib') _ node('master') { truststore.copy('/path/to/alternative/truststore/file.jks') } -node('anotherOne') { +node('anotherNode') { //truststore.use ... as usual } ``` From 18ec717bab56836e6d6146e77c124aa8dd3a2713 Mon Sep 17 00:00:00 2001 From: Philipp Pixel Date: Wed, 18 Sep 2019 08:28:23 +0200 Subject: [PATCH 4/5] #4 Review changes: Add whitespace and clarifications Signed-off-by: Philipp Pixel --- vars/helper.groovy | 2 +- vars/truststore.groovy | 2 +- vars/truststore.txt | 2 +- vars/withMavenSettings.groovy | 2 +- vars/withMavenSettings.txt | 12 ++++++------ 5 files changed, 10 insertions(+), 10 deletions(-) diff --git a/vars/helper.groovy b/vars/helper.groovy index cdb7dbd..fb13a08 100644 --- a/vars/helper.groovy +++ b/vars/helper.groovy @@ -10,4 +10,4 @@ String findHostName() { script.error 'Unable to determine hostname from env.JENKINS_URL. Expecting http(s)://server:port/jenkins' } return (env.JENKINS_URL =~ regexMatchesHostName)[0][1] -} \ No newline at end of file +} diff --git a/vars/truststore.groovy b/vars/truststore.groovy index 8ae7d10..c099b0f 100644 --- a/vars/truststore.groovy +++ b/vars/truststore.groovy @@ -35,4 +35,4 @@ def use(Closure inner) { } finally { sh "rm -f ./${truststoreFile}" } -} \ No newline at end of file +} diff --git a/vars/truststore.txt b/vars/truststore.txt index 5e22e1f..2cd4b40 100644 --- a/vars/truststore.txt +++ b/vars/truststore.txt @@ -1,4 +1,4 @@ -Starts copies and provides a truststore files for safe use over different Jenkins nodes. When the build job finishes there +Copies and provides a truststore file for safe use over different Jenkins nodes. When the build job finishes there will be no truststore file left in the Jenkins workspace. By default Jenkins's truststore will be used: `/var/lib/jenkins/truststore.jks` diff --git a/vars/withMavenSettings.groovy b/vars/withMavenSettings.groovy index 935352a..bd11179 100644 --- a/vars/withMavenSettings.groovy +++ b/vars/withMavenSettings.groovy @@ -72,4 +72,4 @@ def customMvnWithSettings(def nexusCredentials, String cesFQDN, String mvnHome, def mvnWithHome(String mvnHome, String settingsXml, String mvnCallArgs) { sh "${mvnHome}/bin/mvn -s ${settingsXml} --batch-mode -V -U -e -Dsurefire.useFile=false ${mvnCallArgs}" -} \ No newline at end of file +} diff --git a/vars/withMavenSettings.txt b/vars/withMavenSettings.txt index 808c38b..2918a6e 100644 --- a/vars/withMavenSettings.txt +++ b/vars/withMavenSettings.txt @@ -1,4 +1,4 @@ -provides more sophisticated Maven support for custom `settings.xml` files which refer central maven mirror to in the +Provides sophisticated Maven support for custom `settings.xml` files which refer to a central maven mirror in the context of a Cloudogu EcoSystem. This global variable writes custom `settings.xml` files and can additionally invoke a provided or custom Maven call. @@ -11,15 +11,15 @@ Once written, the `settings.xml`'s structure will look like this: | +-> localRepository | `-> servers | `-> server -| +-> id -| +-> username -| `-> password +| +-> id (generated with FQDN) +| +-> username (taken from credentials) +| `-> password (taken from credentials) | `-> mirrors `-> mirror +-> id (reference to the server id) - +-> name - +-> url + +-> name (generated from FQDN) + +-> url (generated from FQDN) +-> mirrorOf (fixed value of central maven mirror) Example call: From 906d2e4d0996e0f0e4df1468b13a03796d3ba347 Mon Sep 17 00:00:00 2001 From: Philipp Pixel Date: Wed, 18 Sep 2019 09:41:04 +0200 Subject: [PATCH 5/5] #4 Review changes: Correct Maven call This was a refactoring artifact. This fix was successfully tested. Signed-off-by: Philipp Pixel --- vars/withMavenSettings.groovy | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vars/withMavenSettings.groovy b/vars/withMavenSettings.groovy index bd11179..8f29f0e 100644 --- a/vars/withMavenSettings.groovy +++ b/vars/withMavenSettings.groovy @@ -51,7 +51,7 @@ def settings(def nexusCredentials, String cesFQDN, String pathToLocalMavenReposi def mvnWithSettings(def nexusCredentials, String cesFQDN, String mvnCallArgs) { def currentHome = env.HOME settings(nexusCredentials, cesFQDN, currentHome) { settingsXml -> - mvnPure settingsXml, mvnCallArgs + mvn settingsXml, mvnCallArgs } }