Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add database fixtures for testing migrations #4858

Merged
merged 14 commits into from
Nov 8, 2022

Conversation

mafredri
Copy link
Member

@mafredri mafredri commented Nov 2, 2022

This PR adds two types of fixtures for testing database migrations

  1. Incremental fixtures in coderd/database/migrations/testdata/fixtures
  2. Full deployment fixtures in coderd/database/migrations/testdata/full_dumps/vX.X.X

For 1), the purpose of the fixtures folder is for us to incrementally add data we feel should be tested during a migration, it's based on a complete dump, but the idea is to evolve it via manual additions (see 000024 example).

For 2), the purpose is for us to easily take a specific version of Coder, create a deployment, dump the database, and ensure that this deployment upgrades without issue for all eternity.

TODO

  • Documentation
    • Describe the logic behind 000022 in file names and how to add fixtures
    • Describe how to create full dumps
  • (In this or future PR) Checks to ensure new tables are populated with data?

Questions

Thoughts on where to place the documentation?

@mafredri mafredri self-assigned this Nov 2, 2022
Copy link
Member

@kylecarbs kylecarbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I enjoy the automated nature of this - it'll just tell us when we broke prior versions!

How do you imagine we'll promote devs to add a new dump or fixture?

@mafredri

This comment was marked as outdated.

@mafredri
Copy link
Member Author

mafredri commented Nov 3, 2022

@kylecarbs with 7d1c3e8 it looks like this:

❯ go test ./coderd/database/migrations -run=TestMigrateUpWithFixtures
--- FAIL: TestMigrateUpWithFixtures (0.00s)
    migrate_test.go:257: The following tables have zero rows, consider adding fixtures for them or create a full database dump:
    migrate_test.go:258: tables have zero rows: [audit_logs git_auth_links group_members licenses replicas]
    migrate_test.go:260: See https://github.com/coder/coder/blob/main/docs/CONTRIBUTING.md#database-fixtures-for-testing-migrations for more information
FAIL
FAIL	github.com/coder/coder/coderd/database/migrations	7.608s
FAIL

@mafredri
Copy link
Member Author

mafredri commented Nov 4, 2022

This PR is now pretty much complete:

  • We have docs for adding fixtures
  • We detect if new tables are added without fixtures and return a descriptive error

The only remaining issue would be to improve how fast a user might run into the migration/fixture error, but I think we can improve that when/if it becomes annoying.

@bpmct would love your input on this (esp. the docs part).

@mafredri mafredri requested a review from bpmct November 4, 2022 18:10
@mafredri mafredri merged commit e906d0d into main Nov 8, 2022
@mafredri mafredri deleted the mafredri/database-fixtures branch November 8, 2022 17:59
@github-actions github-actions bot locked and limited conversation to collaborators Nov 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants