Adding basic retention #91

Merged
merged 8 commits into from Nov 30, 2016

Projects

None yet

3 participants

@jeffersongirao
jeffersongirao commented Nov 14, 2016 edited
  • The idea is to re-use slaves after successful builds, the slave is terminated after being idle for the time defined in idleMinutes. That is useful for builds that rely heavily on cached data for "speed".
/**
* Kubernetes cloud provider.
- *
+ *
@carlossg
carlossg Nov 20, 2016

it helps reviewing PRs if there are no spurious changes in imports or whitespace

@jeffersongirao
jeffersongirao Nov 21, 2016

My bad, I reverted such changes.

@@ -44,9 +38,11 @@
private transient final KubernetesCloud cloud;
- @DataBoundConstructor
+ @Deprecated
@carlossg
carlossg Nov 20, 2016

I think there is no need to deprecate it
It should call the other constructor this(...) to avoid duplication

@@ -27,6 +27,10 @@
<f:textbox/>
</f:entry>
+ <f:entry field="retainedAfterTask" title="${%Retain slave after task}">
@carlossg
carlossg Nov 20, 2016

instead of having a boolean, I think it's better to have idleMinutes here, like the docker plugin, so it can be set per template for how long it should be kept

@jeffersongirao

Thanks for the feedback, I've made the suggested changes. Cheers!

+ RetentionStrategy rs)
+ throws Descriptor.FormException, IOException {
+
+ this(template, nodeDescription, cloud, labelStr);
@carlossg
carlossg Nov 21, 2016

I think you'd better do it the other way around, call this(template, nodeDescription, cloud, labelStr, rs); in the first constructor, call super in the second one

@@ -49,6 +49,8 @@
private int instanceCap = Integer.MAX_VALUE;
+ private int idleMinutes = Integer.MIN_VALUE;
@carlossg
carlossg Nov 21, 2016

why not just use Integer and set it to null?

@jeffersongirao
jeffersongirao Nov 21, 2016

Just afraid of NullPointerException in general. I will change it.

+
+ @DataBoundSetter
+ public void setIdleMinutesStr(String idleMinutes) {
+ if ("".equals(idleMinutes)) {
@carlossg
carlossg Nov 21, 2016

you'd want to use StringUtils.isBlank() not just the empty string

@jeffersongirao
jeffersongirao Nov 21, 2016

Is it ok to introduce a dependency here? I was just trying to be consistent with what is implemented for setInstanceCapStr.

+ if (t.getIdleMinutes() == Integer.MIN_VALUE) {
+ retentionStrategy = new OnceRetentionStrategy(cloud.getRetentionTimeout());
+ } else {
+ retentionStrategy = new CloudRetentionStrategy(cloud.getRetentionTimeout());
@carlossg
carlossg Nov 21, 2016

shouldn't be t.getIdleMinutes() to use the idle time for the template ?

@jeffersongirao
jeffersongirao Nov 21, 2016

You are right, fixing it.

@jeffersongirao
jeffersongirao commented Nov 21, 2016 edited

I did changes, please let me know if you prefer me to squash the commits in a single one. Cheers!

@carlossg

Just one last comment, and then it needs rebase

+
+ @Deprecated
+ public KubernetesSlave(PodTemplate template, String nodeDescription, KubernetesCloud cloud, Label label)
+ throws Descriptor.FormException, IOException {
@carlossg
carlossg Nov 29, 2016

can you just call

this(template, nodeDescription, cloud, label, new OnceRetentionStrategy(cloud.getRetentionTimeout()))

to avoid duplicated code

@jeffersongirao

Fixed and rebased. Thanks!

@carlossg carlossg merged commit 78b1c25 into jenkinsci:master Nov 30, 2016

1 check passed

Jenkins This pull request looks good
Details
@carlossg

thanks!

@JeanMertz

I'm not sure if this is the PR that caused our issues, but it's closely related, so I'll put this here:

We use different docker images for each of our projects running on Jenkins using the kubernetes-plugin.

I've noticed recently that a lot of times a job of project A might end up on a pod/container of project B.

Because each pod/container setup is highly coupled to the different projects, a lot of tools are missing for project A to run inside a pod of project B, and thus the run fails.

We haven't set the idleMinutes value, so it should default to 0, and thus OnceRetentionStrategy, but that doesn't seem to be the case.

Something else that I noticed: since this PR(?) any custom pod template that we define in a Jenkinsfile gets automatically added to the list of global list of "Kubernetes Pod Template" configs in the global configuration page of Jenkins.

This means that after a while, we end up with thousands of templates on that page, grinding that page to a halt. Possibly, this is the cause of the shared pod templates we are seeing?

@JeanMertz

If I do a "reload from disk", the extra templates disappear from the global configuration (leaving only those that we actually defined in the global scope)

@jeffersongirao

Hello @JeanMertz, I tried to reproduce your issue but I could not. Could you please, give more details about the version of Jenkins you are trying it out and example of the Jenkinsfile? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment