Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playlist: Add create+update timestamps to the database #76295

Merged
merged 47 commits into from
Oct 10, 2023

Conversation

ryantxu
Copy link
Member

@ryantxu ryantxu commented Oct 10, 2023

What is this feature?

This adds created+updated epoc columns to the playlist database

Why do we need this feature?

The k8s api service needs something to track when a resource changes

Who is this feature for?

This is to help move k8s development forward.

After running the playlist table will look like:
image


Extracted from #75709

@@ -82,7 +82,6 @@ func (*OSSMigrations) AddMigration(mg *Migrator) {
accesscontrol.AddManagedPermissionsMigration(mg, accesscontrol.ManagedPermissionsMigrationID)
accesscontrol.AddManagedFolderAlertActionsMigration(mg)
accesscontrol.AddActionNameMigrator(mg)
addPlaylistUIDMigration(mg)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: this does not remove the migration -- the same migrations are now added in addPlaylistMigrations

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this doesn't remove the migration, but we must not change the migrations list! It is additive only - that's set out in our contribution guidelines and changing past migrations can cause unexpected behavior in existing instances: https://github.com/grafana/grafana/blob/main/contribute/backend/database.md#migrations

@ryantxu ryantxu marked this pull request as ready for review October 10, 2023 18:45
@ryantxu ryantxu requested review from tolzhabayev and a team as code owners October 10, 2023 18:45
@ryantxu ryantxu requested review from suntala, yangkb09 and mildwonkey and removed request for a team October 10, 2023 18:45

func addPlaylistUIDMigration(mg *Migrator) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^^ this was split as a different function before, but now all migrations added from the same entry point

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this also changes the order of the migrations, doesn't it? What's the benefit of merging it into one function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the cleanup -- the only reason i did it was to reuse the same table structure created in playlistV2() but does not really matter, we can create a new one for every migration

Copy link
Contributor

@mildwonkey mildwonkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can change existing migrations the way this PR does, but I'll get a second opinion on that. Otherwise, this looks (and works) ok to me!

🤔 Not directly related, but has anyone suggested we add a provisioning source field to all of our kinds, to differentiate between records saved via k8s, the UI, terraform, etc?

@@ -82,7 +82,6 @@ func (*OSSMigrations) AddMigration(mg *Migrator) {
accesscontrol.AddManagedPermissionsMigration(mg, accesscontrol.ManagedPermissionsMigrationID)
accesscontrol.AddManagedFolderAlertActionsMigration(mg)
accesscontrol.AddActionNameMigrator(mg)
addPlaylistUIDMigration(mg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this doesn't remove the migration, but we must not change the migrations list! It is additive only - that's set out in our contribution guidelines and changing past migrations can cause unexpected behavior in existing instances: https://github.com/grafana/grafana/blob/main/contribute/backend/database.md#migrations

}
// Add columns used for kubernetes dual write synchronization
mg.AddMigration("Add playlist column created_at", NewAddColumnMigration(playlistV2, &Column{
Name: "created_at", Type: DB_BigInt, Nullable: false, Default: "0",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be DATETIME SQL type (same about updated_at)?

Copy link
Member Author

@ryantxu ryantxu Oct 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with DATETIME, you can't apply a default to an existing structure :( so it can not be a required field, and that leads to needing pointers 🤮

see also https://github.com/grafana/grafana/blob/main/contribute/backend/recommended-practices.md#106---date-comparison

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not saying this as an argument in either direction, but I poked through the code and we're more likely to use an int64 than DATETIME in other tables, too 🤷🏻

@ryantxu
Copy link
Member Author

ryantxu commented Oct 10, 2023

removed the migration cleanup -- does not really matter, so no issue with it

@ryantxu
Copy link
Member Author

ryantxu commented Oct 10, 2023

but has anyone suggested we add a provisioning source field to all of our kinds, to differentiate between records saved via k8s, the UI, terraform, etc?

yes -- entity api has an origin field to help with this. In k8s it will be stored as an annotation on any resource. I don't think it is worth adding to the existing tables though.

Copy link
Contributor

@mildwonkey mildwonkey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻 LGTM, thanks for addressing the migration concerns!

@ryantxu ryantxu merged commit c26e3d8 into main Oct 10, 2023
14 checks passed
@ryantxu ryantxu deleted the playlist-add-timestamps2 branch October 10, 2023 19:46
@zerok zerok modified the milestones: 10.2.x, 10.2.0 Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants