Support schedule configuration like CronJob

[tjun]

What you want to add

Add schedule node count schedule configuration like this

spec:
  scaleTargetRef:
    projectId: your-gcp-project-id
    instanceId: your-spanner-instance-id
  minNodes: 1
  maxNodes: 4
  cron:
    schedule: "*/1 * * * *"
    period: 1H
    minNodes: 4

Why this is needed

We sometimes want to increase node num before starting a batch job.

[apstndb]

Possible requirements:

• Supports multiple CRON schedules per SpannerAutoscaler resource.
• Supports on-demand constraints for on-demand tasks beside CRON schedules.
  • It should be configured without modifying SpannerAutoscaler resource.

[apstndb]

Precedents:

• Google Cloud Game Server Service
  • https://cloud.google.com/game-servers/docs/how-to/configuring-cluster-scaling-behavior?hl=en#creating_multiple_scaling_schedules
  • It is not Kubernetes CRD but it is enhancement of Agones controller and based on Kubernetes resource model.
  • It supports multiple default/fixed(startTime-endTime)/recurring(cron based) settings.
• https://github.com/d-kuro/scheduled-pod-autoscaler
  • It supports multiple settings via separated custom resource(ScheduledPodAutoscaler/Schedule)
  • Article: https://zenn.dev/d_kuro/articles/bb75adeba67d60

[apstndb]

Currently, SpannerAutoscaler CRD has two responsibility.

• Scale
  • scale compute capacity of Cloud Spanner instance from concrete value(PU/node count)
  • analogous to ReplicaSet/Deployment
• Autoscaler
  • compute and modify current desired scale within the scale config
  • analogous to HorizontalPodAutoscaler

We want to add a new functionality.

• Schedule
  • individual schedule
    • time spec
      • fixed(start time/end time or duration)
      • recurring(cron with duration)
      • (We want TZ support)
    • scale constraints within the period
      • If their periods are overlapped, we can use most strong one for the period(e.g. use max resource of them)
  • analogous to Schedule of d-kuro/scheduled-pod-autoscaler

Considerations

• Should each Schedule be separated from Autoscaler?
  • If Schedule is separated CRD, non IoC-style on-demand schedules may be possible.
    • Should we do garbage collection?
    • end time or duration deterministic is not deterministic so on-demand "done" is needed.
      • delete the resource?
      • patch some field?
        • end time or some flag
• How to link the Autoscaler and the Schedule?
  • Use another CRD
    • analogous to ScheduledPodAutoscaler of d-kuro/scheduled-pod-autoscaler
      • It also have default scale config
  • Doesn't use another CRD(Schedule directly references the Autoscaler)
    • Current scale config is not represented as concrete resource
      • Use status?
    • Autoscaler controller will be complicated

(For reference) Relationships in d-kuro/scheduled-pod-autoscaler

• Schedule
  • .spec.scaleTargetRef references to ScheduledPodAutoscaler
• ScheduledPodAutoscaler
  • .spec.horizontalPodAutoscalerSpec.scaleTargetRef references to HorizontalPodAutoscaler
• HorizontalPodAutoscaler
  • It is generated and updated by the autoscaler
• Deployment/ReplicaSet
  • It is not touched by the autoscaler

[apstndb]

Draft schema

Written in CUE to express disjunctions.

import "time"

#ScheduledSpannerAutoscaler: {
	apiVersion: "spanner.mercari.com/v1alpha1"
	kind:       "ScheduledSpannerAutoscaler"
	metadata: {
		name:      string
		namespace: string
	}
	spec: {
		// SpannerAutoscaler without {min,max}{Nodes,ProcessingUnits}
		spannerAutoscalerTemplate: {
			scaleTargetRef: {
				projectId:  string
				instanceId: string
			}
			{
				serviceAccountSecretRef: {
					namespace: string
					name:      string
					key:       string
				}
			} | {
				impersonateConfig: targetServiceAccount: string
			}
			maxScaleDownNodes: int
			targetCPUUtilization: {
				highPriority: int
			}
		}
		defaultScaleConfig: #ScaleConfig
	}
}

#Schedule: {
	apiVersion: "spanner.mercari.com/v1alpha1"
	kind:       "Schedule"
	metadata: {
		name:      string
		namespace: string
	}
	spec: {
		// References in same namespace
		ScheduledSpannerAutoscalerRef: {
			name: string
		}
		scaleConfig: #ScaleConfig
		{
			manual: {
				// RFC3339
				startTime: time.Time
				// RFC3339
				endTime?: time.Time
			}
		} | {
			cron: {
				// in github.com/robfig/cron compatible format
				spec:      string
				duration:  time.Duration
				timeZone?: string | *"UTC"
			}
		}
	}
}

#ScaleConfig: {
	// Only supports processing units
	min: int
	max: int
}

[apstndb]

Missing use case

Schedule last day of month

1. Some CRON implementation have L(last) notation. robfig/cron doesn't yet support it.
   Run Cron last Monday of month robfig/cron#319
2. First day of month with advanced offset can be last day of month.
   Example) 2022-01-01T00:30:00+10:00 = 2021-12-31T23:30:00+09:00

IMHO, 2 is realistic approach.