Autoscaler: rash decisions after crash or start

[mgencur]

When the autoscaler first starts or crashes and restarts it makes rash decisions about scaling.
For example, when it crashes and restarts, and there are no requests for 2 seconds it will immediately scale to down significantly (scale to 1). See my test below which simulates the situation when there are no statistics (after crash) except for those from last two seconds. The test returns 0 but there's actually a guard against scaling to 0, so in reality it will scale to 1.
Another example is going to a panic mode, it's enough to have a high load for two seconds (instead of the default 6-second window) and the autoscaler goes into the panic mode. However, in this case it's probably not such a big concern.

Expected Behavior

Make sure we have statistics from a stable window (by default 60 seconds) before scaling down significantly.

Actual Behavior

Scaling down very quickly when there are not enough statistics.

Steps to Reproduce the Problem

func TestAutoscaler_ShortlyAfterStartOrCrash_ScaleToZero(t *testing.T) {
	a := newTestAutoscaler(10.0)
	now := a.recordLinearSeries(
		t,
		time.Now(),
		linearSeries{
			startConcurrency: 0,
			endConcurrency:   0,
			durationSeconds:  2,
			podCount:         10,
		})

	a.expectScale(t, now, 10, true)
	// ^^^fails with "Expected 10. Got 0."
}

[hohaichi]

@mgencur PR#3771 should fix this issue. The repro test above is not applicable though, because it tests how the autoscaler calculates the desired scale from (concurrency) stats instead of testing where the scaling action is decided from the desired scale. There are tests in the PR that assert the scale-to-minScale and bounce-from-0 when there is no metric (looks for scaleTo: -1).