Incremental migrations #891
Conversation
github-actions
bot
added
area/CLI
jakedt
force-pushed
the
incremental-migrations
branch
3 times, most recently
from
c4e28fa
to
1588cdc
Compare
There was a problem hiding this comment.
If I understand the migration instructions correctly, we'd need to run the migrations manually rather than automatically? Would that mean disabling the k8s post-upgrade hook temporarily?
Suggestion: instead of starting SpiceDB with different flags, should that be different spicedb versions? ie something like:
- Upgrade to v1.14.0
- migrations ran by post-upgrade hook (migration 9 and 10 but without the backfilling in 10)
- 1.14.0 always does "write both read old"
- Manually run
spicedb run-job backfill-xidrun-jobis something I made up. Effectively a migration to be ran manually.
- Upgrade to v1.15.0
- Migration 11 ran by post-upgrade hook
- Upgrade to v1.16.0
- 1.16.0 always does "write both read new"
- Check that the v1.15.0 migrations have been ran, refuse to start otherwise
- Upgrade to v1.17.0
- Run migrations 12/13
- 1.16.0 always does "write new read new" ("old" doesn't exist anymore)
Maybe it can be consolidated to 3-phase rather than 4, I haven't overly thought about it
| "github.com/rs/zerolog/log" | ||
| ) | ||
|
|
||
| const batchSize = 1000 |
There was a problem hiding this comment.
That sounds very low :) Need to trade-off between execution speed and large table locks though, there's no right answer I don't think
There was a problem hiding this comment.
I'm just going to make it a command line parameter to the migrate command, defaulting to 1000.
| SET xid = id::text::xid8, snapshot = CONCAT(id, ':', id, ':')::pg_snapshot | ||
| WHERE id IN ( | ||
| SELECT id FROM relation_tuple_transaction | ||
| WHERE snapshot IS NULL |
There was a problem hiding this comment.
This WHERE clause (and the 2 other ones) might get very slow and resource-intensive as there's no index on snapshot.
You could create an index on snapshot/created_xid for the purpose of the migration, then clean it up at the end of the migration (or in a follow-up migration)?
|
|
||
| var addXIDIndices = []string{ | ||
| // Replace the indices that are inherent from having a primary key constraint | ||
| "CREATE UNIQUE INDEX CONCURRENTLY ix_rttx_oldpk ON relation_tuple_transaction (id)", |
There was a problem hiding this comment.
If I'm understanding correctly this migration isn't ran in a transaction. There's going to be issues if this migration is interrupted while creating indices: it won't be possible to re-run it as PG will complain about indices already existing.
Could just add a IF NOT EXISTS here
jakedt
force-pushed
the
incremental-migrations
branch
from
beddbac
to
9aff3ba
Compare
|
@williamdclt Updated, PTAL |
jakedt
force-pushed
the
incremental-migrations
branch
from
9aff3ba
to
ca8c50a
Compare
|
|
||
| // MigrationVariable contains constants that can be used as context keys that might | ||
| // be relevant in a number of different migration scenarios. | ||
| type MigrationVariable int |
There was a problem hiding this comment.
do we want to make these ints vs strings? I have concerns there might be accidental overlap with something we don't inject
There was a problem hiding this comment.
Not possible, the equality operator the context uses considers the type:
https://go.dev/play/p/16BfmcC0d9X
Migration instructions:
spicedb migrate add-xid-columns--datastore-migration-phase write-both-read-oldspicedb migrate add-xid-constraints- performs a backfill and may take a long time--datastore-migration-phase write-both-read-newspicedb migrate drop-id-constraints--datastore-migration-phaseflagspicedb migrate head