Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

[JENKINS-20884] Variable expansion in maven goals #14

Closed
wants to merge 3 commits into from

8 participants

@zygm0nt

I had a Jenkins job, where I needed to calculate goals to be executed in a shell script. So I two pre-build steps:

  • shell script
  • inject environment variables (EnvInject)

Then I wanted to put that variable into maven goals, like this:

-o clean install -pl ${some_variable}

This did not work. This change fixes that.

Would be happy to know if this is ok, or should be fixed in some other way.

Regards
Marcin

@cloudbees-pull-request-builder

plugins » maven-plugin #46 UNSTABLE
Looks like there's a problem with this pull request

@jenkinsadmin
Owner

Thank you for a pull request! Please check this document for how the Jenkins project handles pull requests

@imod
Collaborator

can you please add a test case? e.g. along the same way as it was made here jenkinsci/jenkins#799

Marcin Cylke added some commits
@zygm0nt

I've created a test. But it is not working yet.

I have a problem with prebuilders, because they don't populate actions on getRootBuild() object, which I use in my changed code, in MavenModuleSetBuild.

Perhaps you could look into the unit test and advise something?

@cloudbees-pull-request-builder

plugins » maven-plugin #47 FAILURE
Looks like there's a problem with this pull request

@imod
Collaborator

please have a go with the PR I have just placed ( #16 ) - this might fix your issue with the testcase too

@bcreasy bcreasy referenced this pull request from a commit in bcreasy/maven-plugin
@bcreasy bcreasy applied basic fix from jenkinsci#14 b02916c
@hicolour

:+1: this would be very very... useful

@KostyaSha
Collaborator

Any updates?

@jglick jglick commented on the diff
src/main/java/hudson/maven/MavenModuleSetBuild.java
@@ -668,6 +668,15 @@ protected Result doRun(final BuildListener listener) throws Exception {
return r;
}
+ logger.println("prebuilders: " + Joiner.on(", ").join(project.getPrebuilders()));
+ logger.println("root actions: " + Joiner.on(", ").join(project.getLastBuild().getActions()));
+
+ for (Action action : getRootBuild().getActions()) {
+ if(action instanceof EnvironmentContributingAction){
+ ((EnvironmentContributingAction) action).buildEnvVars(MavenModuleSetBuild.this, envVars);
@jglick Owner
jglick added a note

Normally this should not have to be called from plugins, because AbstractBuild.getEnvironment(TaskListener) calls it already.

@KostyaSha Collaborator

AbstractBuild.getEnvironment(TaskListener) is called before prebuilders, and groovy or other prebuilder may modify in EnvironmentContributingAction some values. So envVars need to be refreshed

@KostyaSha Collaborator

Calling again envVars.overrideExpandingAll(getEnvironment(listener)) is potentially wrong. Is there any way to keep EnvVars transient so prebuilds had ability to access and modify it directly instead of polling classes?

@KostyaSha Collaborator

Also prebuilders like Groovy script calls .getEnvironment(listener) from perform internals and uses EnvVars that maybe not the same that were called in MavenModuleSetBuild. If there is no critical overrides in variables before, then additional call to .getEnvironment(TaskListener) maybe a solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@kohsuke kohsuke referenced this pull request from a commit
@kohsuke kohsuke [JENKINS-20884] Mutating 'envVars' returned from getEnvironment() cre…
…ates a bad precedent.

It is better to just recompute the envVars, which will reflect all the added environments.

See: #14

Relevant conversation:

(11:31:53 AM) KostyaSha: kohsuke, while windows is eating your IO, could you advice with #14 ?
(11:34:27 AM) kohsuke: KostyaSha: the bug looks legit, but I'm not sure about the fix. Basically I agree with Jesse's comment
(11:35:35 AM) KostyaSha: kohsuke, this broken by design i think, it should be a good idea to share envvars for builders. This should also reduce .getEnvironments() calls
(11:36:33 AM) KostyaSha: kohsuke, is there any place where they maybe safely shared?
(11:37:03 AM) kohsuke: I'm afraid I don't understand the notion of "sharing envvars"
(11:37:35 AM) kohsuke: It gets computed from various things, and I thought EnvironmentContributingAction is a part of it
(11:38:44 AM) KostyaSha: kohsuke, yes, and they are stored in envvars variable, then job calls prebuilders that modifies EnvironmentContributingAction and then job calls maven build with not updated envvars content
(11:39:05 AM) kohsuke: then it should just call EnvVars envVars = getEnvironment(listener); again
(11:39:50 AM) KostyaSha: kohsuke, i not sure that there is no any specific changes with envvars after first getEnvironment(listener) call and before prebuilders
(11:40:30 AM) KostyaSha: i compared on my local instance and this should work, but i not sure... potentially some changes to envvars maybe lost...
(11:40:31 AM) kohsuke: Basically, one should never modify what Run.getEnvironment() returned
(11:40:51 AM) kohsuke: If the map returned is missing some desirable entries, then it should be fixing by having the getEnvironment() implementation add them
(11:41:11 AM) kohsuke: It looks to me that this change violates that idea
(11:42:02 AM) KostyaSha: i like idea of refreshing with simple call to getEnvironment(listener)
(11:42:03 AM) kohsuke: if environment-contributing subset of rootBuild.actions should be a part of the env vars, according to the above principle it should be done in the getEnvironment() method
(11:42:52 AM) kohsuke: (Also, let's not print out random stuff into "logger.println" that most users would not care
(11:43:02 AM) kohsuke: Those should be j.u.logging statements)
(11:43:28 AM) KostyaSha: kohsuke, yeah this logger not needed of course
(11:44:55 AM) KostyaSha: kohsuke,  https://github.com/zygm0nt/maven-plugin/blob/master/src/main/java/hudson/maven/MavenModuleSetBuild.java#L648 what this part do?
(11:45:44 AM) kohsuke: That looks wrong to me, too, for the same reason
(11:45:49 AM) kohsuke: All right, you convinced me to open this in IDE
(11:46:03 AM) kohsuke: Screw the preparation for the meeting
(11:47:29 AM) kohsuke: Wow, that line has jglick fixing HUDSON-3502
(11:47:31 AM) jenkins-admin: JENKINS-3502:Xvnc does not set the DISPLAY environment (Closed) https://issues.jenkins-ci.org/browse/JENKINS-3502
(11:48:06 AM) KostyaSha: 0_o
(11:48:17 AM) kohsuke: From 2009
(11:49:34 AM) KostyaSha: kohsuke, and the next block is also added for resolving variables
(11:49:42 AM) kohsuke: Yep
(11:50:28 AM) KostyaSha: so after every step we need recalculate changes... so easy to allow do direct modifications of envvars i think
cab3151
@kohsuke
Owner

I agree with Jesse, in that it looks wrong to have EnvVars mutated after it gets returned from getEnvironment. Whatever goes into the environment variables should all happen in getEnvironment() implementation.

My fix from cab3151 to 14f28ed addresses this accordingly, and it passes the test @zygm0nt wrote.

@kohsuke kohsuke closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 5, 2013
  1. Variable expansion in maven goals

    Marcin Cylke authored
Commits on Dec 9, 2013
  1. not working integration test

    Marcin Cylke authored
  2. not working integration test

    Marcin Cylke authored
This page is out of date. Refresh to see the latest.
View
11 src/main/java/hudson/maven/MavenModuleSetBuild.java
@@ -668,6 +668,15 @@ protected Result doRun(final BuildListener listener) throws Exception {
return r;
}
+ logger.println("prebuilders: " + Joiner.on(", ").join(project.getPrebuilders()));
+ logger.println("root actions: " + Joiner.on(", ").join(project.getLastBuild().getActions()));
+
+ for (Action action : getRootBuild().getActions()) {
+ if(action instanceof EnvironmentContributingAction){
+ ((EnvironmentContributingAction) action).buildEnvVars(MavenModuleSetBuild.this, envVars);
@jglick Owner
jglick added a note

Normally this should not have to be called from plugins, because AbstractBuild.getEnvironment(TaskListener) calls it already.

@KostyaSha Collaborator

AbstractBuild.getEnvironment(TaskListener) is called before prebuilders, and groovy or other prebuilder may modify in EnvironmentContributingAction some values. So envVars need to be refreshed

@KostyaSha Collaborator

Calling again envVars.overrideExpandingAll(getEnvironment(listener)) is potentially wrong. Is there any way to keep EnvVars transient so prebuilds had ability to access and modify it directly instead of polling classes?

@KostyaSha Collaborator

Also prebuilders like Groovy script calls .getEnvironment(listener) from perform internals and uses EnvVars that maybe not the same that were called in MavenModuleSetBuild. If there is no critical overrides in variables before, then additional call to .getEnvironment(TaskListener) maybe a solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ }
+ }
+
parsePoms(listener, logger, envVars, mvn, mavenVersion, mavenBuildInformation); // #5428 : do pre-build *before* parsing pom
SplittableBuildListener slistener = new SplittableBuildListener(listener);
proxies = new HashMap<ModuleName, ProxyImpl2>();
@@ -786,7 +795,7 @@ protected Result doRun(final BuildListener listener) throws Exception {
final List<MavenArgumentInterceptorAction> argInterceptors = this.getBuild().getActions(MavenArgumentInterceptorAction.class);
-
+
// find the correct maven goals and options, there might by an action overruling the defaults
String goals = project.getGoals(); // default
for (MavenArgumentInterceptorAction mavenArgInterceptor : argInterceptors) {
View
141 src/test/java/hudson/maven/MavenEnvironmentContributingActionFromBuilderTest.java
@@ -0,0 +1,141 @@
+package hudson.maven;
+
+import java.io.IOException;
+import java.util.Collection;
+import java.util.Collections;
+
+import org.junit.Assert;
+import org.junit.Rule;
+import org.junit.Test;
+import org.jvnet.hudson.test.Bug;
+import org.jvnet.hudson.test.ExtractResourceSCM;
+import org.jvnet.hudson.test.JenkinsRule;
+import org.kohsuke.stapler.StaplerRequest;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import hudson.EnvVars;
+import hudson.Extension;
+import hudson.Launcher;
+import hudson.model.AbstractBuild;
+import hudson.model.AbstractProject;
+import hudson.model.Action;
+import hudson.model.BuildListener;
+import hudson.model.Cause;
+import hudson.model.EnvironmentContributingAction;
+import hudson.model.InvisibleAction;
+import hudson.model.Result;
+import hudson.tasks.BuildWrapper;
+import hudson.tasks.BuildWrapperDescriptor;
+import hudson.tasks.Builder;
+import hudson.util.ArgumentListBuilder;
+import net.sf.json.JSONObject;
+
+/**
+ * This test case verifies that a maven build also takes EnvironmentContributingAction into account to resolve variables on the command line
+ *
+ * @see hudson.model.EnvironmentContributingAction
+ * @author Marcin Cylke (mcl)
+ */
+public class MavenEnvironmentContributingActionFromBuilderTest {
+
+ @Rule
+ public JenkinsRule j = new JenkinsRule();
+
+ @Test
+ @Bug(20844)
+ public void builderInjectedVariableFromEnvironmentContributingActionMustBeAvailableInMavenModuleSetBuild() throws Exception {
+ j.jenkins.getInjector().injectMembers(this);
+
+ final MavenModuleSet p = j.createMavenProject("mvn");
+
+ p.setMaven(j.configureMaven3().getName());
+ p.setScm(new ExtractResourceSCM(getClass().getResource("maven3-project.zip")));
+ p.setGoals("initialize -Dval=${KEY}");
+
+ String keyValue = "MY_VALUE";
+
+ p.getPrebuilders().add(new TestMvnBuilder(keyValue));
+ p.getBuildWrappersList().add(new AssertingBuildWrapper("-Dval=" + keyValue));
+
+ j.assertBuildStatus(Result.SUCCESS, p.scheduleBuild2(0, new Cause.UserIdCause()).get());
+ }
+
+ /**
+ * This action contributes env variables
+ */
+ private static final class TestAction extends InvisibleAction implements EnvironmentContributingAction {
+ private final String key, value;
+
+ public TestAction(String key, String value) {
+ this.key = key;
+ this.value = value;
+ }
+
+ @Override
+ public void buildEnvVars(AbstractBuild<?, ?> arg0, EnvVars vars) {
+ vars.put(key, value);
+ }
+
+ }
+
+ /**
+ * This action verifies that the variable in the maven arguments got replaced
+ */
+ private static class MvnCmdLineVerifier extends InvisibleAction implements MavenArgumentInterceptorAction {
+ private String containsString;
+
+ public MvnCmdLineVerifier(String containsString) {
+ this.containsString = containsString;
+ }
+
+ @Override
+ public ArgumentListBuilder intercept(ArgumentListBuilder cli, MavenModuleSetBuild arg1) {
+ String all = cli.toString();
+ Assert.assertTrue(containsString + " was not found in the goals arguments(" + all + ")",
+ all.contains(containsString));
+ return cli;
+ }
+
+ @Override
+ public String getGoalsAndOptions(MavenModuleSetBuild arg0) {
+ return null;
+ }
+ }
+
+ /**
+ * This wrapper adds MvnCmdLineVerifier to the build to test whether the variable really got replaced
+ */
+ public static class AssertingBuildWrapper extends BuildWrapper {
+ private String containsString;
+
+ public AssertingBuildWrapper(String expectedString) {
+ this.containsString = expectedString;
+ }
+
+ @Override
+ public Environment setUp(AbstractBuild build, Launcher launcher, BuildListener listener) throws IOException, InterruptedException {
+
+ build.addAction(new MvnCmdLineVerifier(containsString));
+
+ return new Environment() { };
+ }
+ }
+
+ /**
+ * This builder returns an action with variable replacement to build process
+ */
+ public static class TestMvnBuilder extends Builder {
+
+ private final String envVariableValue;
+
+ public TestMvnBuilder(String envVariableValue) {
+ this.envVariableValue = envVariableValue;
+ }
+
+ @Override
+ public Collection<? extends Action> getProjectActions(AbstractProject<?, ?> project) {
+ return Collections.singletonList(new TestAction("KEY", envVariableValue));
+ }
+ }
+}
Something went wrong with that request. Please try again.