Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Velero restores should work regardless of the length of the Restore name #1021

Closed
rosskukulinski opened this issue Oct 30, 2018 · 7 comments

Comments

Projects
None yet
3 participants
@rosskukulinski
Copy link
Member

commented Oct 30, 2018

Create a restore from a backup with a long name defaults to a Restore name that's even longer:

Backup: k8s-dev-auth-namespace-backup-schedule-20181026050059
Restore: k8s-dev-auth-namespace-backup-schedule-20181026050059-20181026153607

ark restore describe k8s-dev-auth-namespace-backup-schedule-20181026050059-20181026153607

...
Phase: Completed
...
Errors:
...
  Namespaces:
    auth:  error restoring /tmp/046287764/resources/secrets/namespaces/auth/default-token-kg2bm.json: Secret "default-token-kg2bm" is invalid: metadata.labels: Invalid value: "k8s-dev-auth-namespace-backup-schedule-20181026050059-20181026153607": must be no more than 63 characters

Ark restore applies labels to restored objects which includes the name of the restore the object is created from. Kubernetes label values must be less than 63 characters, and will be rejected if they are longer.

We ran into similar issues in Gimbal for replicating Services -- That solution is documented here: https://github.com/heptio/gimbal/blob/master/docs/discovery-naming-conventions.md#handling-discovered-names-that-are-longer-than-63-characters which might be of interest.

@nzoueidi

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2019

I am trying to fix this. I suspect that ctx.restore.Name and ctx.restore.Spec.backupName are the two componenets that you @rosskukulinski are talking about above, right?

@skriss

This comment has been minimized.

Copy link
Contributor

commented Feb 15, 2019

@nzoueidi to summarize this issue - Kubernetes object names have a maximum length of 253 characters. When Velero executes a restore, it adds a label, velero.io/restore-name, to each restored object, whose value is the name of the restore. However, label values have a maximum length of 63 characters.

We need to come up with a way to ensure that the label values have at most 63 characters, while still being able to represent objects with names of length up to 253 characters.

A couple ways we could do this:

  • When a restore is created, add a ShortName label to it, and use some kind of hash to populate this label with a unique name that's <= 63 characters. Use this value when labeling resources.
  • label restored objects with the restore's UID rather than its name
  • don't allow restores to be created with names > 63 characters (i.e. have them fail validation)

Also note that there are likely additional places where this same issue is possible - we use object names as label value in multiple places. It would be best to address all of these, and in a consistent way.

A few references:

https://kubernetes.io/docs/concepts/overview/working-with-objects/names/
https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/

@nzoueidi

This comment has been minimized.

Copy link
Contributor

commented Feb 19, 2019

I hope I can push my PR by the end of this week.

@skriss

This comment has been minimized.

Copy link
Contributor

commented Feb 19, 2019

I'd recommend proposing an approach in a comment here before doing all of the coding!

@nzoueidi

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2019

Yep, sorry for the delay. This is my approach, please correct me if I am mistaken.

  • Whenever the restore is created, I use its UID automatically rather then its name.
  • If somehow, the UID combined by the timestap of the restore is >= 31, then we could use a ShortName (the same first point that you mentioned above).
  • However, the name is stored and we can use it if the criteria was/is met (the name of the restore given by Velero + timestamp is <= 63) otherwise, we apply the same two points above.

@skriss wdyt?

@skriss

This comment has been minimized.

Copy link
Contributor

commented Mar 11, 2019

Reading through this again, I think an approach similar to the one used in Gimbal (https://github.com/heptio/gimbal/blob/master/docs/discovery-naming-conventions.md#handling-discovered-names-that-are-longer-than-63-characters) makes sense. This would look something like:

  • if the length of the backup/restore name is <= 63 characters, use it as the value of the label
  • if it's > 63 characters, take the SHA256 hash of the name. the value of the label will be the first 57 characters of the backup/restore name plus the first six characters of the SHA256 hash.

@nzoueidi - Let's start by doing that for all labels that are applied in this function: https://github.com/heptio/velero/blob/master/pkg/restore/restore.go#L993

@skriss skriss self-assigned this Apr 19, 2019

@skriss

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

Anshul is actively working on this issue.

canshul08 added a commit to canshul08/velero that referenced this issue Apr 24, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue Apr 24, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue Apr 24, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

@skriss skriss changed the title Ark restores should work regardless of the length of the Restore name Velero restores should work regardless of the length of the Restore name Apr 24, 2019

canshul08 added a commit to canshul08/velero that referenced this issue Apr 24, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue Apr 25, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue Apr 30, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue Apr 30, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

canshul08 added a commit to canshul08/velero that referenced this issue May 1, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>

@nrb nrb closed this in #1392 May 1, 2019

jessestuart added a commit to jessestuart/velero that referenced this issue May 28, 2019

Validate restore name label length
Velero should handle cases when the label length exceeds 63 characters.

- if the length of the backup/restore name is <= 63 characters, use it as the value of the label
- if it's > 63 characters, take the SHA256 hash of the name. the value of
  the label will be the first 57 characters of the backup/restore name
  plus the first six characters of the SHA256 hash.

Fixes heptio#1021

Signed-off-by: Anshul Chandra <anshulc@vmware.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.