Autoscaler multi-tenancy and strategy support

[glyn]

To support event-driven revisions, the autoscaler needs to support multi-tenancy. That is, the autoscaler needs to handle scaling for all revisions rather than just one.

Each revision needs its own scaling strategy so that scaling of request-response and event-driven revisions can use distinct data structures, algorithms, and metrics types (e.g. replica versus topic metrics).

For background, see the Autoscaling elafros/riff convergence design document.

[glyn]

I’ve been thinking about how to structure the autoscaler to accommodate both distinct metrics types and distinct data structures (based on replica alone for request-response replicas and based on replica and topic for event-driven replicas). We could attempt to share state in a "base" autoscaler of the form:

map[ReplicaId]Scaler

with a minimal Scaler interface:

type Scaler interface {
    Propose() int32
}

There could be distinct implementations of Scaler indexed in strategy-specific ways but sharing state with the instance stored in the base autoscaler.

Alternatively, it might be cleaner from a concurrency perspective if there were two distinct autoscalers (and no stateful "base") both implementing a multi-tenant-oriented Autoscaler interface:

typer Autoscaler interface {
    Propose() map[ReplicaId]int32
}

These autoscalers would share no state and so could have distinct designs optimised separately.

[glyn]

The Elafros autoscaling contract proposal separates request-response and event-driven autoscaling into independent components.

[josephburnett]

I think that @grantr has been working with @glyn to define a multi-tenant autoscaler strategy.

[glyn]

See #1067 for a rebased, improved version of #963.

[glyn]

/assign @glyn

[evankanderson]

/kind spec

[glyn]

Note to self: metrics for the Prometheus exporter are gathered by stats_reporter.go.

[glyn]

Broke out #1658 for adding the missing dynamic reconfiguration support. The strategy support is less urgent and is deferred now that eventing is reusing the serving autoscaler support.

[glyn]

Broke out #1677 to address an outstanding FIXME.