Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore: Replace short UID generation with more standard UUIDs #62731

Merged
merged 22 commits into from Feb 7, 2023

Conversation

ryantxu
Copy link
Member

@ryantxu ryantxu commented Feb 1, 2023

We currently generate a UID using github.com/teris-io/shortid -- and then loop 3x in the database to make sure it is actually unique. This PR replaces that logic with a more standard UUID that we can depend on being unique.

This also makes sure the generated UUID starts with a string -- this is to ensure that if we want to use it as a k8s name, it can be https://kubernetes.io/docs/concepts/overview/working-with-objects/names/

The new UIDs look like:

nba00926-5ca8-4016-818c-db9ca0f0adb8
p98fd744-db83-42a4-bf42-1f244fa83e89
caaf3b03-f2f9-4322-9404-a288bacc3c4f
bfcf9ff9-e6b0-4b9d-82cb-fba91609a44e
l838be3d-52e0-4b11-8a9a-2281556dbd76
c7cf09cf-2e58-4304-84e8-22ca20c42178
l2fb84a0-fcbb-4123-aa85-81986c5e58b0
e1f3a7c9-25ec-4408-8e51-6e0acdd88cdc
bd649328-90fd-4cb5-a437-141d0cc558bd
d024e671-6f59-450f-abd1-987fb7cd831b
yd2363ef-4acb-4c98-9f45-df4ab0262566
pbe71b2a-2529-440c-875a-21938243dbde
sf9ea917-bf12-4a37-b69c-1e0840cf6879
ke728d0e-e559-44b7-a48b-28d97a1c0bbb

before... the generated UIDs look like:

Ei74RD9mz
3GsCac3kz
-YaGDGIMk
fuFWehBmk
GlAqcPgmz
xMsQdBfWz
2WaUpZmWk
M94gg_RWz
Hmf8FDkmz
WVpf2jp7z
j6T00KRZz
7p7JkqWVz
XMuLlpZ4k
9Z7AhQiVz
U_bZIMRMk

These may look a little bit better in URLs ¯\(ツ)/¯ -- but given that they:

  1. contain capitals and "_"
  2. sometimes start with numbers or punctuation
    they can not be used as k8s names

NOTE: this change is not required, but it does seem worthwhile to stop creating invalid k8s names. This will help us have fewer things to migrate if we do want to map our current "uid" concept to a k8s name.

ryantxu and others added 4 commits February 2, 2023 12:16
* Alerting: Fix template validation in provisioning api

Fix issue where provisioning API accepts a malformed template having extra
text outside of definition block and template name matching definition name.
…62033)

* Start work on allowing certain resources to pass through Cache-Control headers.
---------

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
@ryantxu ryantxu requested a review from DanCech February 3, 2023 00:35
@ryantxu ryantxu changed the title WIP/UIDs: Generate UIDs that are compatible with k8s names UIDs: Replace short UID generation with UUIDs Feb 3, 2023
@ryantxu ryantxu changed the title UIDs: Replace short UID generation with UUIDs UIDs: Replace short UID generation with more standard UUIDs Feb 3, 2023
@ryantxu ryantxu changed the title UIDs: Replace short UID generation with more standard UUIDs Chore: Replace short UID generation with more standard UUIDs Feb 3, 2023
Copy link
Collaborator

@DanCech DanCech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, assuming we haven't broken anything else that used short uids. The one that seems like it might be an issue would be the short url service as they won't be so short any more.

One thought may be to instead add GenerateUID() and convert things over to that where it makes sense while leaving GenerateShortUID as-is, but obviously that will result in a somewhat larger change.

@ying-jeanne
Copy link
Contributor

The change looks safe to me, the length of UID is limited to 40 bytes, which is backward compatible for db + unique index limitation. Since the generation is more strict and validation stays the same way, modify dashboard is not impacted. Just need to pay attention, the save dashboard service is used by both folder and dashboard create api, that means the folder service is gonna having the same uid pattern. We probably want to update some docs if this is going used outside, for example we allow user to modify fold/dashboard uid manually right now still.

@ryantxu ryantxu added this to the 9.5.0 milestone Feb 6, 2023
@ryantxu ryantxu added no-backport Skip backport of PR add to changelog labels Feb 6, 2023
@JacobsonMT
Copy link
Member

Coincidentally, I was in the middle of making a similar change but isolated to just the ngalert package. Not because of k8s naming standards but because the current short uid generator consistently creates case-insensitive conflicts when ran in a large enough tight loop. Normally this is fine, but if someone is using MySQL this can and does create key errors.

Ex: #60494, https://github.com/grafana/support-escalations/issues/4887

And a couple other places where it's looming but rare.

Copy link
Member

@JacobsonMT JacobsonMT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from the alerting side. There are probably a good amount of db reads checking for duplicates we can get rid of now, but those are best changed separately. 🚀

@ryantxu ryantxu marked this pull request as ready for review February 7, 2023 01:15
@ryantxu ryantxu requested review from a team as code owners February 7, 2023 01:15
@ryantxu ryantxu requested review from zserge, mildwonkey and idafurjes and removed request for a team February 7, 2023 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants