[JENKINS-31801] Add Pipeline throttle(category) step #46

abayer · 2017-03-08T22:34:15Z

POC
Getting to finalized implementation
Snippet generator support
Tests!

Note that yes, this depends on workflow-api - I need that for FlowExecution and friends. And I had to remove the existing guava:18.0 test dependency due to that violently conflicting with StepContext. I haven't actually run tests yet, let alone written new ones, but I wanted to throw this up for initial feedback ASAP.

cc @reviewbybees esp @jglick for Pipeline sanity review and @oleg-nenashev for Throttle sanity review

jglick · 2017-03-08T22:39:24Z

pom.xml

@@ -44,10 +44,10 @@ THE SOFTWARE.
  </licenses>

  <properties>
-    <jenkins.version>1.609.3</jenkins.version>
+    <jenkins.version>1.642.3</jenkins.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>


Yup yup - at one point, I was experimenting with stuff that did need newer versions of other dependencies and bumped things, but I'll roll this back now.

jglick · 2017-03-08T22:39:28Z

pom.xml

-    <compileSource>1.6</compileSource>
-    <compileTarget>1.6</compileTarget>
+    <compileSource>1.7</compileSource>
+    <compileTarget>1.7</compileTarget>


java.level should be used instead

jglick · 2017-03-08T22:39:33Z

pom.xml

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-    <compileSource>1.6</compileSource>
-    <compileTarget>1.6</compileTarget>
+    <compileSource>1.7</compileSource>


Brought these back now that we're depending on 1.642.3 again.

Use java.level, Luke

jglick · 2017-03-08T22:40:14Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java

@@ -229,29 +243,82 @@ public boolean isThrottleMatrixConfigurations() {
        assert category != null && !category.equals("");
        List<Queue.Task> categoryTasks = new ArrayList<Queue.Task>();
        Collection<ThrottleJobProperty> properties;
-        DescriptorImpl descriptor = Jenkins.getActiveInstance().getDescriptorByType(DescriptorImpl.class);    
+        DescriptorImpl descriptor = Jenkins.getActiveInstance().getDescriptorByType(DescriptorImpl.class);


Please avoid gratuitous whitespace changes. Configure your editor to not make them.

jglick · 2017-03-08T22:42:27Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleJobProperty.java

+                // No run found, so remove the throttle.
+                descriptor.removeThrottledPipelineForCategory(currentPipeline.getKey(), category, null);
+            } else if (!(flowNodeRun instanceof FlowExecutionOwner.Executable)) {
+                // If for some reason we've somehow ended up with a non-pipeline job, remove the throttle.


Why not just index by FlowExecutionOwner to begin with?

Is there an easy way to go from some easily serializable identifier for FlowExecutionOwner to an actual instance of FlowExecutionOwner, like with Run.fromExternalizableId?

No, it is just Serializable.

Ah. Still not sure I want to actually save that large a chunk of data when I can just do the round-tripping...

jglick · 2017-03-08T22:55:35Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+        return new ThrottleStepExecution(this, context);
+    }
+
+}


No descriptor?

jglick · 2017-03-08T22:56:07Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+        if (parentInfo != null) {
+            // Make sure we record the node used for the parent throttle step if it exists.
+            Computer computer = getContext().get(Computer.class);
+            if (computer != null && computer.getNode() != null) {


Return Node from getRequiredContext and skip both the null check and the getNode conversion.

Aaaaah, for some reason I didn't think that was possible. Win!

jglick · 2017-03-08T22:57:32Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+
+    @Override
+    public void stop(Throwable cause) throws Exception {
+        if (body != null)


This method can be left blank—will not be called for a block-scoped step anyway, unless it is doing something between returning from start and the BodyInvoker.start.

jglick · 2017-03-08T22:57:59Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+
+        body = getContext().newBodyInvoker()
+                .withContext(info)
+                .withCallback(BodyExecutionCallback.wrap(getContext()))


I think you meant to use TailCall and call removeThrottledPipelineForCategory.

Aaaah, you're right. Thanks.

jglick · 2017-03-08T22:58:33Z

src/main/resources/hudson/plugins/throttleconcurrents/Messages.properties

+ThrottleMatrixProjectOptions.DisplayName=Additional options for Matrix projects
+
+ThrottleJobProperty.DescriptorImpl.NoSuchCategory=Requested category "{0}" does not exist, so cannot throttle.


trailing newline

abayer · 2017-03-10T22:22:19Z

@jglick Did a fairly drastic rewrite here - I think I got out of the blocking-CPS-VM issues, and am tracking FlowNodes of the start nodes for the step, but I'm not sure if I'm actually on the right path. =)

jetersen · 2017-03-13T08:43:56Z

I would be happy to test this! 👍

jetersen · 2017-03-13T09:47:03Z

Would this be a correct usage, see test stage?
I might rewrite it to scripted, to get parallel with failfast!

Sorry for sick indentation but such is life 🍎

pipeline {
    agent none
    options {
        buildDiscarder(logRotator(numToKeepStr:'20'))
    }
    stages {
        stage('Build') {
            agent { label 'ios' }
            steps {
                withCredentials([string(credentialsId: 'keychain', variable: 'PASSWORD')]) {
                    sh "security list-keychains -s ~/Library/Keychains/login.keychain"
                    sh "security unlock-keychain -p ${PASSWORD} ~/Library/Keychains/login.keychain"
                }
                wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm', 'defaultFg': 1, 'defaultBg': 2]) {
                    withCredentials([
                        string(credentialsId: 'fastlaneMatch', variable: 'MATCH_PASSWORD'),
                        string(credentialsId: 'fastlanePassword', variable: 'FASTLANE_PASSWORD')])
                    {
                        sh 'fastlane build'
                    }
                }
            }
        }
        stage('Test') {
            agent none
            steps {
                throttle('ios-simulator') {
                    node('ios') {
                        lock("ios-simulator-${env.NODE_NAME}") { // Do I even need lock?
                            withCredentials([string(credentialsId: 'keychain', variable: 'PASSWORD')]) {
                                sh "security list-keychains -s ~/Library/Keychains/login.keychain"
                                sh "security unlock-keychain -p ${PASSWORD} ~/Library/Keychains/login.keychain"
                            }
                            retry(5) {
                                wrap([$class: 'AnsiColorBuildWrapper', 'colorMapName': 'XTerm', 'defaultFg': 1, 'defaultBg': 2]) {
                                    withCredentials([
                                        string(credentialsId: 'fastlaneMatch', variable: 'MATCH_PASSWORD'),
                                        string(credentialsId: 'fastlanePassword', variable: 'FASTLANE_PASSWORD')])
                                    {
                                        sh 'fastlane test'
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Needed to bump to newer dependency versions, most notably to get PlaceholderTask.getNode(). Still a work in progress, mind you.

abayer · 2017-03-13T19:44:04Z

pom.xml

@@ -26,7 +26,7 @@ THE SOFTWARE.
  <parent>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>plugin</artifactId>
-    <version>2.6</version>
+    <version>2.22</version>


2.23 led to errors due to a TestExtension usage in workflow-support's tests, so I bumped back down to 2.22 for now at least.

Harmless, just stack traces. workflow-support itself should be bumped up.

abayer · 2017-03-13T19:44:34Z

pom.xml

@@ -44,10 +44,10 @@ THE SOFTWARE.
  </licenses>

  <properties>
-    <jenkins.version>1.609.3</jenkins.version>
+    <jenkins.version>1.642.3</jenkins.version>


As mentioned in the commit message, we needed to bump dependency versions to get the latest durable-task-step for PlaceholderTask.getNode().

abayer · 2017-03-13T19:46:25Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java

        // Handle multi-configuration filters
-        if (!shouldBeThrottled(task, tjp)) {
+        if (!shouldBeThrottled(task, tjp) && pipelineCategories.isEmpty()) {


So re: this and later similar ugliness - I plan to rewrite ThrottleQueueTaskDispatcher pretty much completely before I'm done here, or at least rationalize it a lot better. This was me wedging things together until they worked, with clean up planned for afterwards.

abayer · 2017-03-13T19:47:40Z

src/test/java/hudson/plugins/throttleconcurrents/ThrottleStepTest.java

+                        "  node('first-agent') {\n" +
+                        "    semaphore 'wait-first-job'\n" +
+                        "  }\n" +
+                        "}\n", false));


Soooo - as I mentioned a few lines up, I'm getting script security exceptions on this when the sandbox is enabled and I can't figure out why, so for the moment, disabling script security until someone (probably @jglick!) tells me what I'm doing wrong to cause those errors to show up. =)

Also discovered that Run<?,?> is a very bad Map key.

abayer · 2017-03-13T21:31:20Z

src/test/java/hudson/plugins/throttleconcurrents/ThrottleStepTest.java

+                "  node('" + label + "') {\n" +
+                "    semaphore 'wait-" + jobName + "-job'\n" +
+                "  }\n" +
+                "}\n", false);


Soooo - as I mentioned a few lines up, I'm getting script security exceptions on this when the sandbox is enabled and I can't figure out why, so for the moment, disabling script security until someone (probably @jglick!) tells me what I'm doing wrong to cause those errors to show up. =)

ghost · 2017-03-13T21:33:47Z

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

jetersen · 2017-03-14T12:03:55Z

This supersedes #34 ?

jetersen · 2017-03-14T12:52:01Z

Just checking if agent { label } + throttle with work same as agent none + throttle + node(label)

Are these two pipeline equal since throttle might run before reserving the executor?

pipeline {
    agent none
    stages {
        stage('test') {
            agent { label 'ios' }
            steps {
                throttle('ios-simulator') {
                    sh 'do stuff'
                }
            }
        }
    }
}

pipeline {
    agent none
    stages {
        stage('test') {
            agent none
            steps {
                throttle('ios-simulator') {
                    node('ios') {
                        sh 'do stuff'
                    }
                }
            }
        }
    }
}

jetersen · 2017-03-17T09:26:55Z

Would it be possible to have agent label and throttle work so that we can have post steps that have file path context.

I would perhaps suggest this syntax but I know it requires work on the pipeline model definition instead.
This properly requires a JIRA issue, but just wondering if its at all possible :)

pipeline {
    agent none
    stages {
        stage('test') {
            agent {
                label 'ios'
                throttle 'ios-simulator'
            }
            steps {
                sh 'do stuff'
            }
        }
        post {
            always {
                junit '**/*.junit'
            }
        }
    }
}

This one below does not work:

pipeline {
    agent none
    stages {
        stage('test') {
            agent { label 'ios' }
            steps {
                throttle('ios-simulator') {
                    sh 'do stuff'
                }
            }
        }
        post {
            always {
                junit '**/*.junit'
            }
        }
    }
}

Neither does this:

pipeline {
    agent none
    stages {
        stage('test') {
            agent none
            steps {
                throttle('ios-simulator') {
                    node('ios') {
                        sh 'do stuff'
                    }
                }
            }
        }
        post {
            always {
                junit '**/*.junit'
            }
        }
    }
}

oleg-nenashev

Looks good so far, but I do not feel this is a final implementation

oleg-nenashev · 2017-03-09T08:11:06Z

pom.xml

-    <compileSource>1.6</compileSource>
-    <compileTarget>1.6</compileTarget>
+    <compileSource>1.7</compileSource>
+    <compileTarget>1.7</compileTarget>


java.level should be used instead

oleg-nenashev · 2017-03-17T09:34:10Z

pom.xml

  </parent>

  <artifactId>throttle-concurrents</artifactId>
  <packaging>hpi</packaging>
  <name>Jenkins Throttle Concurrent Builds Plug-in</name>
-  <version>1.9.1-SNAPSHOT</version>
+  <version>2.0-SNAPSHOT</version>


I would advice going through the beta- or -rc for this change. The potential impact is too high.

oleg-nenashev · 2017-03-17T09:34:49Z

pom.xml

    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-    <compileSource>1.6</compileSource>
-    <compileTarget>1.6</compileTarget>
+    <compileSource>1.7</compileSource>


Use java.level, Luke

oleg-nenashev · 2017-03-17T09:37:10Z

pom.xml

+        <dependency>
+            <groupId>org.jenkins-ci.plugins.workflow</groupId>
+            <artifactId>workflow-durable-task-step</artifactId>
+            <version>2.8</version>


Is there a need to have it as required dependency? It pulls a significant number of Pipeline plugins into (like Pipeline Support & Co)

Yes - we need ExecutorStepExecution.class for checking against, etc.

You do not use this class from what I see

Ah, that may have gone away. Fixing.

Nope, it's still there - specifically ExecutorStepExecution.PlaceholderTask in ThrottleQueueTaskDispatcher.

oleg-nenashev · 2017-03-17T09:38:30Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+        this.step = step;
+    }
+
+    public String getCategory() {


Nonnull I'd guess

oleg-nenashev · 2017-03-17T09:39:07Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+    }
+
+    @Nonnull
+    public String getCategory() {


Only one category? I would expect an opportunity to define multiple categories like I can do for Job Properties

I'd rather keep it simple and wait to see what the demand is like for more complex.

First of all thanks for this Pullrequest I was waiting for it for quite some time :)
I would really like to be able to throttle multiple categories as this is our usual scenario in our freestyle jobs.

oleg-nenashev · 2017-03-17T09:40:24Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+
+    @Override
+    public void stop(Throwable cause) throws Exception {
+


Should it also call removeThrottledPipelineForCategory ?

Apparently, from Jesse's earlier comments, no - stop doesn't have relevance here.

oleg-nenashev · 2017-03-17T09:40:53Z

src/main/resources/hudson/plugins/throttleconcurrents/ThrottleStep/config.jelly

+<?jelly escape-by-default='true'?>
+<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define"
+         xmlns:l="/lib/layout" xmlns:t="/lib/hudson" xmlns:f="/lib/form">
+    <f:entry title="${%Category}" field="category">


I would expect multiple categories

oleg-nenashev · 2017-03-17T09:43:36Z

src/test/java/hudson/plugins/throttleconcurrents/ThrottleConcurrentTest.java

@@ -25,6 +26,7 @@

 import static org.assertj.core.api.Assertions.assertThat;

+@Ignore("Depends on a newer version of Guava than can be used with Pipeline")


It is a pretty important test suite. I do not feel that it is really required to have such declarative syntax here, so maybe it worth just rewriting it

I agree. I'll tackle that either here or in a follow-up PR.

Actually, on second thought, this is basically covered by ThrottleStepTest, isn't it?

I take that back - did some experimenting and there is something weird once you've got enough Pipeline tasks in queue. Working on it.

Aaaah, queue.getPendingItems(). Fixed.

oleg-nenashev · 2017-03-17T09:44:55Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStepExecution.java

+    }
+
+    @Override
+    public boolean start() throws Exception {


Maybe makes sense to contribute some information to the build log using the BuildListener from the context

Good point.

abayer · 2017-04-07T15:48:17Z

Check that, think I hit the deadlock in testing. =)

jglick

Only have a fuzzy idea what this is doing, so pobably not going to be a good reviewer.

jglick · 2017-04-13T22:20:21Z

pom.xml

@@ -26,7 +26,7 @@ THE SOFTWARE.
  <parent>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>plugin</artifactId>
-    <version>2.6</version>
+    <version>2.22</version>


Harmless, just stack traces. workflow-support itself should be bumped up.

jglick · 2017-04-13T22:23:01Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java

+        if (origTask instanceof PlaceholderTask) {
+            PlaceholderTask task = (PlaceholderTask)origTask;
+            try {
+                FlowNode firstThrottle = firstThrottleStartNode(task.getNode());


This really deserves a proper API so you can get rid of the workflow-durable-task-step dependency. It was never intended as an API plugin.

jglick · 2017-04-13T22:24:01Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java

+                    // Check if this is a *different* throttling node.
+                    StepDescriptor desc = ((StepNode) enclosing).getDescriptor();
+                    if (desc != null && desc.getClass().equals(ThrottleStep.DescriptorImpl.class)) {
+                        return enclosing;


Oof. @svanoort?

jglick · 2017-04-13T22:24:27Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+import java.util.Set;
+import java.util.StringTokenizer;
+
+public class ThrottleStep extends Step implements Serializable {


If you just have the one field, probably better to inline that into the execution state.

abayer · 2017-04-20T18:52:24Z

@oleg-nenashev So are we good to go here? I defer to you for merging and releasing.

oleg-nenashev

Just minor comments so far. Will test it manually this week

oleg-nenashev · 2017-04-25T15:29:19Z

pom.xml

+        <dependency>
+            <groupId>org.jenkins-ci.plugins.workflow</groupId>
+            <artifactId>workflow-durable-task-step</artifactId>
+            <version>2.8</version>


You do not use this class from what I see

oleg-nenashev · 2017-04-25T15:31:46Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+
+        StringTokenizer tokenizer = new StringTokenizer(categories, ",");
+        while (tokenizer.hasMoreTokens()) {
+            catList.add(tokenizer.nextToken().trim());


If I understand correctly, it is going to inject empty strings. Which may become a real problem if somebody uses something like cat_a,${myvar},cat_c expression

Not sure what you mean?

I mean strings like throttle("cat_a,,cat_b"), which may come to this method

Aaaaaah. Changing to discard empty.

oleg-nenashev · 2017-04-25T15:33:18Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+            return Collections.singleton(TaskListener.class);
+        }
+
+        public FormValidation doCheckCategoryName(@QueryParameter String value) {


Restricted ? not sure if RequirePost is mandatory

Good question. This doesn't actually expose any meaningful information - all you can do is tell whether there's a category with a given name, not anything about the category, etc. I think it's harmless, but defer to others.

oleg-nenashev · 2017-04-25T15:35:43Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

+
+        @Override
+        public String getDisplayName() {
+            return "Throttle execution of node blocks within this body";


Nice2have: make it localizable

Reasonable.

oleg-nenashev · 2017-04-25T15:44:06Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java

+                            firstThrottle.getId());
+                }
+            } catch (IOException | InterruptedException e) {
+                return new ArrayList<>();


No logging? 🐜

oleg-nenashev · 2017-04-25T15:44:39Z

src/main/java/hudson/plugins/throttleconcurrents/ThrottleQueueTaskDispatcher.java

+                return new ArrayList<>();
+            }
+        }
+        return new ArrayList<>();


Nice2have: assert : false

? We're calling this method for all tasks, so an assert would cause problems.

oleg-nenashev

The success path behavior looks good. Though 🐛 from me according to the UX testing

Documentation:

The feature is not documented. I am working on Wiki => GitHub migration in a parallel PR to make it possible

Configuration:

No validation of the categories field in the snippet generator, no context help about available categories
No built-n Documentation for the "Categories" field in the Snipper generator. 🐛 according to the criteria we have IIRC
Multiple categories are being passed as string, not as a list/array. It complicates management of categories in the code

Behavior:

If the declared category does not exist, it is just ignored. It is not safe if a user plans to throttle a special resource (like my favorite FPGAs) and makes a typo in Pipeline. I would expect the script to fail if there is a non-existent category. Smells like a 🐛 to me
Behavior for duplicated definitions is not clear. One may expect throttle('A,A,A') to acquire 3 items from the category pool A to run. I would add an explicit disclaimer that it is not how it is supposed to work
"Throttle Concurrent Builds" Job Property appears in the UI, but does nothing for both modes from what I see. It may confuse plugin users, hence 🐛. The JobProperty should be hidden IMHO, it is not a part of MVP to have its support for Pipeline

abayer · 2017-04-28T13:43:46Z

@oleg-nenashev Thanks for testing it out!

Snippet generator stuff definitely needs to be updated - I'll get that done.

Multiple categories are being passed as string, not as a list/array. It complicates management of categories in the code

So this is an interesting question. I've found surprisingly few examples of steps taking lists, so I went conservative. I'll play around with this to see if I can get a good list or array approach.

If the declared category does not exist, it is just ignored.

I wasn't sure what the best behavior would be in this case. I opted for a "first, do no harm" approach, but am fine with switching to fail.

Behavior for duplicated definitions is not clear.

Huh. I literally never thought about that. So you're thinking logging something when there are duplicate categories?

"Throttle Concurrent Builds" Job Property appears in the UI, but does nothing for both modes from what I see.

This isn't a change introduced by this PR - I think I did that a while back. It would, I think, apply to the full Pipeline execution, but yeah, it's pretty useless. I just worry that someone out there may be using it and would get screwed if we dropped it.

oleg-nenashev · 2017-04-28T13:51:33Z

I wasn't sure what the best behavior would be in this case. I opted for a "first, do no harm" approach, but am fine with switching to fail.

Yeah, this is my approach as well. I just expect less harm if the job fails than if it gets executed incorrectly. Better fail than ~~undefined behavior~~ sorry :)

So you're thinking logging something when there are duplicate categories?

Or I can probably just add note to README in #47.

This isn't a change introduced by this PR - I think I did that a while back. It would, I think, apply to the full Pipeline execution, but yeah, it's pretty useless. I just worry that someone out there may be using it and would get screwed if we dropped it.

I will check if it actually works. If yes, likely we need to polish UI a bit to prevent confusion of users. Not sure how exactly

…step

abayer · 2017-05-18T14:49:53Z

@oleg-nenashev Ok, I think I've responded to everything but the job property thing (which is a separate matter entirely).

oleg-nenashev · 2017-05-18T16:46:55Z

@abayer After some consideration, I still feel bad about the TCB JobProperty UX. Would it be possible to add a disclaimer to its built-in help at least?

abayer · 2017-05-18T16:50:28Z

@oleg-nenashev I...guess? I'd almost rather just make it not show up and say "screw it" to anyone who's for some reason somehow using the job property with a Pipeline currently.

abayer · 2017-05-18T16:51:58Z

Also, I'm not the one who added that support in the first place, as far as I can tell. =)

oleg-nenashev · 2017-05-18T16:54:11Z

@abayer

I'd almost rather just make it not show up and say "screw it" to anyone who's for some reason somehow using the job property with a Pipeline currently.

It is 2.0, so probably it is a valid option. The possible confusion is too high since the behavior is not documented at all. And the plugin explicitly says Pipeline is not supported. Just add incompatibleSince to pom.xml in such case

@Casz @jenkinsci/code-reviewers If you use Global config for TCB, please let us know

oleg-nenashev · 2017-05-18T16:57:07Z

The Pipeline support has been added by @dsavenko in #32 . And yeah, both me and @abayer approved it. So I would rather vote for leaving as is and documenting

abayer · 2017-05-18T17:06:31Z

@oleg-nenashev Then frankly, I think documenting it is a separate issue, isn't it?

oleg-nenashev · 2017-05-18T17:09:03Z

@abayer

Then frankly, I think documenting it is a separate issue, isn't it?

think some basic built-in docs should be included in this PR, because the new functionality leads to confusion with the existing one. A couple of sentences in JobProperty is enough. Regarding the full documentation, I will add it in #47 . Does it sound like a deal?

abayer · 2017-05-18T17:12:35Z

Deal.

…Pipeline

oleg-nenashev

Needs some manual testing of UI, but the code looks good to me. 🐝

oleg-nenashev · 2017-05-18T17:25:34Z

src/main/java/hudson/plugins/throttleconcurrents/pipeline/ThrottleStep.java

@@ -80,6 +63,14 @@ public boolean takesImplicitBlockArgument() {
        public FormValidation doCheckCategoryName(@QueryParameter String value) {
            return ThrottleJobProperty.fetchDescriptor().doCheckCategoryName(value);
        }
+
+        public List<ThrottleJobProperty.ThrottleCategory> getCategories() {
+            return ThrottleJobProperty.fetchDescriptor().getCategories();


Wrap to the unmodifyableList?

oleg-nenashev · 2017-05-18T17:31:37Z

@reviewbybees done
I will test it tomorrow & merge it if everything is fine

oleg-nenashev · 2017-05-19T13:42:12Z

All my comments have been addressed. I have created https://issues.jenkins-ci.org/browse/JENKINS-44373 for better diagnosability, but it is no a blocker for the merge. @abayer should I squash it?

[JENKINS-31801] Initial work on throttle(category) step - needs tests

af877cd

abayer added the work-in-progress label Mar 8, 2017

abayer requested review from jglick and oleg-nenashev March 8, 2017 22:34

jglick reviewed Mar 8, 2017

View reviewed changes

abayer added 2 commits March 10, 2017 14:18

Reworked to no longer rely on StepExecutions.

c000c30

Add a trailing newline to messages.

91cdb4b

abayer added 2 commits March 10, 2017 15:02

Make findbugs happy.

d011b40

Whoops, this needs to take a block

666ff55

Initial test, actually working

42f4da1

Needed to bump to newer dependency versions, most notably to get PlaceholderTask.getNode(). Still a work in progress, mind you.

abayer commented Mar 13, 2017

View reviewed changes

abayer added 5 commits March 13, 2017 13:15

Cleanup, commenting, javadoc

8378722

Test across all nodes

492c500

Also discovered that Run<?,?> is a very bad Map key.

Add interop with freestyle test

89f7bce

Add snippet generator support.

33b8e0f

Add snippetizer support and test

f09aea8

abayer commented Mar 13, 2017

View reviewed changes

abayer removed the work-in-progress label Mar 13, 2017

jglick self-assigned this Mar 13, 2017

oleg-nenashev reviewed Mar 17, 2017

View reviewed changes

Allow multiple comma-separated categories

13cafdf

jglick reviewed Apr 13, 2017

View reviewed changes

oleg-nenashev reviewed Apr 25, 2017

View reviewed changes

abayer added 2 commits April 25, 2017 12:14

Minor review responses, moving step UI to correct dir

fdd9102

Go away, empty category names!

4ec2057

oleg-nenashev requested changes Apr 28, 2017

View reviewed changes

oleg-nenashev mentioned this pull request Apr 28, 2017

[JENKINS-31801] - Documentation Revamp + Pipeline support docs #47

Merged

5 tasks

abayer added 2 commits May 18, 2017 10:24

Fixed up snippetizer, switched to a list of strings for the throttle …

968d77c

…step

Check for and respond to duplicate or non-existent category names

0b222d3

Adding help for ThrottleJobProperty pointing out it doesn't work for …

7d83b60

…Pipeline

oleg-nenashev approved these changes May 18, 2017

View reviewed changes

unmodifiableList

ee1e959

oleg-nenashev self-assigned this May 18, 2017

oleg-nenashev merged commit 0bbd079 into master May 19, 2017

iaroslav42 mentioned this pull request May 31, 2017

Throttling flyweight tasks #34

Open

		ThrottleMatrixProjectOptions.DisplayName=Additional options for Matrix projects

		ThrottleJobProperty.DescriptorImpl.NoSuchCategory=Requested category "{0}" does not exist, so cannot throttle.


		@Override
		public void stop(Throwable cause) throws Exception {

		@@ -25,6 +26,7 @@

		import static org.assertj.core.api.Assertions.assertThat;

		@Ignore("Depends on a newer version of Guava than can be used with Pipeline")

[JENKINS-31801] Add Pipeline throttle(category) step #46

[JENKINS-31801] Add Pipeline throttle(category) step #46

Conversation

abayer commented Mar 8, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Mar 10, 2017

jetersen commented Mar 13, 2017

jetersen commented Mar 13, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Mar 13, 2017

jetersen commented Mar 14, 2017 • edited

jetersen commented Mar 14, 2017

jetersen commented Mar 17, 2017 • edited

oleg-nenashev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ilker-demir Apr 7, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Apr 7, 2017

jglick left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Apr 20, 2017

oleg-nenashev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abayer commented Mar 8, 2017 •

edited

jetersen commented Mar 13, 2017 •

edited

jetersen commented Mar 14, 2017 •

edited

jetersen commented Mar 17, 2017 •

edited

ilker-demir Apr 7, 2017 •

edited