Implement scale endpoint for jobs

[soltysh]

Currently Deployments, RC and RS have scale endpoint which allows directly scaling those instead of mutating the object's spec when scaling up/down. kubectl scale job modifies .spec.parallelism directly. I'm proposing jobs should have /scale endpoint to leverage scale functionality.

@kubernetes/sig-apps ideas/objections?

[0xmichalis]

SGTM

[adohe-zz]

SGTM.

[wanghaoran1988]

/cc

[wanghaoran1988]

@soltysh @Kargakis I am try to add the scale endpoint for the Job, but I have a problem now, I am using the scale struct in the api extentions pkg，I am not sure whether it's correct, or should I create a new one in batch pkg ?
Also I found the parallelism filed doesn't exits in the JobStatus struct, should we update to add this to the struct to show how many replicas are actually running?

[0xmichalis]

Also I found the parallelism filed doesn't exits in the JobStatus struct, should we update to add this to the struct to show how many replicas are actually running?

Yeah, I have thought about this before and I think it deserves its own issue. Can you please open one and discuss that there?

[wanghaoran1988]

After I go through the logic, I find that actual parallelism (number of pods running at any instant) may be more or less than requested parallelism,docs. So I think to align with the doc, we need have it's own Scale struct, here is the spec in my mind, the Completions field is immutable, but the real parallelism pods depends on this, so I put that in the ScaleStatus spec.
@Kargakis @soltysh Could you please take a look, if it's ok, I will start working on this.

type ScaleSpec struct {
	// Specifies the maximum desired number of pods the job should
	// run at any given time. The actual number of pods running in steady state will
	// be less than this number when ((.spec.completions - .status.successful) < .spec.parallelism),
	// i.e. when the work left to do is less than max parallelism.
	// +optional
	Parallelism *int32
}

// represents the current status of a scale subresource.
type ScaleStatus struct {
	// Specifies the desired number of successfully finished pods the
	// job should be run with.  Setting to nil means that the success of any
	// pod signals the success of all pods, and allows parallelism to have any positive
	// value.  Setting to 1 means that parallelism is limited to 1 and the success of that
	// pod signals the success of the job.
	// +optional
	Completions *int32

	// The number of actively running pods.
	// +optional
	Active int32

	// label query over pods that should match the replicas count.
	// More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors
	// +optional
	Selector *metav1.LabelSelector
}

// represents a scaling request for a resource.
type Scale struct {
	metav1.TypeMeta
	// Standard object metadata; More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata.
	// +optional
	metav1.ObjectMeta

	// defines the behavior of the scale. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status.
	// +optional
	Spec ScaleSpec

	// current status of the scale. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status. Read-only.
	// +optional
	Status ScaleStatus
}

[0xmichalis]

@soltysh can you ptal on @wanghaoran1988's proposal?

[soltysh]

@wanghaoran1988 here's the thing. I've talked some time ago with @deads2k and we both agreed that having Scale structure inside every API group isn't the right solution. The idea was to have one Scale structure that would work with every resource out there. To do that we'd need some dynamic scale client, see #32523 (comment). If you could work on it I'd be more than happy reviewing it :)

[deads2k]

I have a pull adding the required information to discovery here: #49971

[soltysh]

There's also #29698 touching this topic.

[wanghaoran1988]

Seems @DirectXMan12 is working on this from the comments here:#49971 (comment)

[wanghaoran1988]

@DirectXMan12 Are you working on this?

[DirectXMan12]

Once #49971 lands, I'll revive my polymorphic scale client PR in some form.

[sttts]

@DirectXMan12 #49971 landed. Any plans for the polymorphic scale?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[nikhita]

/remove-lifecycle stale

[DirectXMan12]

Polymorphic scale landed in 1.9, FWIW (see the referenced commits in the activity stream above), so this is no longer blocked by that

[soltysh]

Based on #58468 (comment) this is not going to happen. Additionally, we're starting to deprecate kubectl scale job functionality in 1.10.

[cometta]

Will there be a new scale endpoint feature after deprecated scale job command?

[soltysh]

Nope, scaling job was considered a mistake, since they are different from all other resources that allow scaling. You can still modify it's spec field, though.

[shangmu]

Then what's the expected behavior (or fundamental difference) of modifying the spec field? If there's no fundamental difference in the two (between kubectl scale and editing the spec field), why not just keep the scale job command and explicitly state the limitations?