Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Backup Annotations #116

Merged
merged 6 commits into from
Aug 18, 2020
Merged

Backup Annotations #116

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
df07593
Added support to pass annotations down to the backup subcontroller an…
PrismaPhonic Aug 10, 2020
4c842f8
Re-ran make generate.
PrismaPhonic Aug 10, 2020
788a6a2
Update language per review suggestion.
PrismaPhonic Aug 15, 2020
c5b8834
Ensure we also update annotations in UpdateInPlace.
PrismaPhonic Aug 15, 2020
b0e99a1
Also update backup location annotations for vttablet pods.
PrismaPhonic Aug 15, 2020
713c4d8
Ensure backup pod annotations are filled in from pool annotations.
PrismaPhonic Aug 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions deploy/crds/planetscale.com_vitessbackupstorages.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ spec:
properties:
location:
properties:
annotations:
additionalProperties:
type: string
type: object
azblob:
properties:
account:
Expand Down
4 changes: 4 additions & 0 deletions deploy/crds/planetscale.com_vitessclusters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ spec:
locations:
items:
properties:
annotations:
additionalProperties:
type: string
type: object
azblob:
properties:
account:
Expand Down
4 changes: 4 additions & 0 deletions deploy/crds/planetscale.com_vitesskeyspaces.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ spec:
backupLocations:
items:
properties:
annotations:
additionalProperties:
type: string
type: object
azblob:
properties:
account:
Expand Down
4 changes: 4 additions & 0 deletions deploy/crds/planetscale.com_vitessshards.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ spec:
backupLocations:
items:
properties:
annotations:
additionalProperties:
type: string
type: object
azblob:
properties:
account:
Expand Down
12 changes: 12 additions & 0 deletions docs/api/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1968,6 +1968,18 @@ <h3 id="planetscale.com/v2.VitessBackupLocation">VitessBackupLocation
a shared host path for local testing.</p>
</td>
</tr>
<tr>
<td>
<code>annotations</code></br>
<em>
map[string]string
</em>
</td>
<td>
<p>Annotations can optionally be used to attach custom annotations to Pods
created for this component.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="planetscale.com/v2.VitessBackupSpec">VitessBackupSpec
Expand Down
4 changes: 0 additions & 4 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,11 @@ module planetscale.dev/vitess-operator
go 1.13

require (
github.com/Azure/go-autorest v14.0.0+incompatible // indirect
github.com/ahmetb/gen-crd-api-reference-docs v0.1.5-0.20190629210212-52e137b8d003
github.com/hashicorp/consul/api v1.5.0 // indirect
github.com/hashicorp/go.net v0.0.1 // indirect
github.com/klauspost/compress v1.7.5 // indirect
github.com/klauspost/cpuid v1.2.1 // indirect
github.com/operator-framework/operator-sdk v0.16.0
github.com/prometheus/client_golang v1.4.1
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da // indirect
github.com/sirupsen/logrus v1.4.2
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.4.0
Expand Down
254 changes: 8 additions & 246 deletions go.sum
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions pkg/apis/planetscale/v2/vitessbackupstorage_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ type VitessBackupLocation struct {
// a shared host path for local testing.
// +kubebuilder:validation:EmbeddedResource
Volume *corev1.VolumeSource `json:"volume,omitempty"`
// Annotations can optionally be used to attach custom annotations to Pods
// that need access to this backup storage location.
Annotations map[string]string `json:"annotations,omitempty"`
}

// GCSBackupLocation specifies a backup location in Google Cloud Storage.
Expand Down
7 changes: 7 additions & 0 deletions pkg/apis/planetscale/v2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion pkg/controller/vitessbackupstorage/reconcile_subcontroller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,18 +77,21 @@ func (r *ReconcileVitessBackupStorage) reconcileSubcontroller(ctx context.Contex
Kind: &corev1.Pod{},

New: func(key client.ObjectKey) runtime.Object {
return &corev1.Pod{
pod := &corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Namespace: key.Namespace,
Name: key.Name,
Labels: labels,
},
Spec: *spec,
}
update.Annotations(&pod.Annotations, vbs.Spec.Location.Annotations)
PrismaPhonic marked this conversation as resolved.
Show resolved Hide resolved
return pod
},
UpdateInPlace: func(key client.ObjectKey, newObj runtime.Object) {
pod := newObj.(*corev1.Pod)
update.Labels(&pod.Labels, labels)
update.Annotations(&pod.Annotations, vbs.Spec.Location.Annotations)
},
UpdateRecreate: func(key client.ObjectKey, newObj runtime.Object) {
pod := newObj.(*corev1.Pod)
Expand Down
43 changes: 24 additions & 19 deletions pkg/controller/vitessshard/reconcile_backup_job.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
planetscalev2 "planetscale.dev/vitess-operator/pkg/apis/planetscale/v2"
"planetscale.dev/vitess-operator/pkg/operator/reconciler"
"planetscale.dev/vitess-operator/pkg/operator/results"
"planetscale.dev/vitess-operator/pkg/operator/update"
"planetscale.dev/vitess-operator/pkg/operator/vitessbackup"
"planetscale.dev/vitess-operator/pkg/operator/vttablet"
)
Expand Down Expand Up @@ -219,31 +220,35 @@ func vtbackupSpec(key client.ObjectKey, vts *planetscalev2.VitessShard, parentLa
minRetentionTime := time.Duration(0)
minRetentionCount := 1

// Fill in the parts of a vttablet spec that make sense for vtbackup.
tabletSpec := &vttablet.Spec{
GlobalLockserver: vts.Spec.GlobalLockserver,
Labels: labels,
Images: vts.Spec.Images,
KeyRange: vts.Spec.KeyRange,
Vttablet: &pool.Vttablet,
Mysqld: pool.Mysqld,
DataVolumePVCName: key.Name,
DataVolumePVCSpec: pool.DataVolumeClaimTemplate,
KeyspaceName: keyspaceName,
DatabaseName: vts.Spec.DatabaseName,
DatabaseInitScriptSecret: vts.Spec.DatabaseInitScriptSecret,
BackupLocation: backupLocation,
BackupEngine: vts.Spec.BackupEngine,
InitContainers: pool.InitContainers,
SidecarContainers: pool.SidecarContainers,
ExtraEnv: pool.ExtraEnv,
Annotations: pool.Annotations,
}
update.Annotations(&tabletSpec.Annotations, backupLocation.Annotations)
PrismaPhonic marked this conversation as resolved.
Show resolved Hide resolved

return &vttablet.BackupSpec{
InitialBackup: backupType == vitessbackup.TypeInit,
MinBackupInterval: minBackupInterval,
MinRetentionTime: minRetentionTime,
MinRetentionCount: minRetentionCount,

// Fill in the parts of a vttablet spec that make sense for vtbackup.
TabletSpec: &vttablet.Spec{
GlobalLockserver: vts.Spec.GlobalLockserver,
Labels: labels,
Images: vts.Spec.Images,
KeyRange: vts.Spec.KeyRange,
Vttablet: &pool.Vttablet,
Mysqld: pool.Mysqld,
DataVolumePVCName: key.Name,
DataVolumePVCSpec: pool.DataVolumeClaimTemplate,
KeyspaceName: keyspaceName,
DatabaseName: vts.Spec.DatabaseName,
DatabaseInitScriptSecret: vts.Spec.DatabaseInitScriptSecret,
BackupLocation: backupLocation,
BackupEngine: vts.Spec.BackupEngine,
InitContainers: pool.InitContainers,
SidecarContainers: pool.SidecarContainers,
ExtraEnv: pool.ExtraEnv,
},
TabletSpec: tabletSpec,
}
}

Expand Down
2 changes: 1 addition & 1 deletion pkg/controller/vitessshard/reconcile_tablets.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ func vttabletSpecs(vts *planetscalev2.VitessShard, parentLabels map[string]strin
drain.SupportedAnnotation: "ensure that the tablet is not a master",
}
update.Annotations(&annotations, pool.Annotations)

update.Annotations(&annotations, backupLocation.Annotations)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to watch out for the case when backupLocation is nil. Unlike vtbackup, we still reconcile vttablets when backups are not configured.

tablets = append(tablets, &vttablet.Spec{
GlobalLockserver: vts.Spec.GlobalLockserver,
Labels: labels,
Expand Down