flux-account: add new update-db command
#244
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a new command to flux-account: update-db, which will look at tables and their existing columns in an already existing flux-accounting DB and add any new tables or columns to existing tables. It does so by creating a temporary database to compare schema; it will then compare tables and then iterate through each table and compare columns. After the iteration is done, the temporary database is removed. For testing purposes, there is an optional argument for the update-db command to specify a path for the temporary database; if not specified (which should be the norm and is noted in the description of the command), the temporary database is created and removed from the current working directory.
Add two helper scripts for running the sharness tests in t1017-update-db.t: modify_accounting_db.py is a helper script that adds two columns to the association_table and a new table to the flux-accounting DB for testing the functionality of the flux account-update-db command. check_db_info.py is a helper script that checks the names of the tables in a flux-accounting DB or the names of the columns within a table in a flux-accounting DB.
cmoussa1
changed the title
update-db commandupdate-db command
|
Thanks @ryanday36! Looking forward to seeing this tool in action. Setting MWP shortly |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Noted in #215, there is a need for supporting the ability to update the flux-accounting database when a new version of the database is released so that the existing DB does not have to be completely re-created and have all values re-inserted. This will be especially important when new columns are added to an already existing table or a new table is added. There currently exists both
export-dbandpop-dbcommands to export and import database information into.csvfiles, but these commands also involve creation of a new DB to migrate this information. It would be nice to have support to just update the database that currently resides on a system with any new potential tables or columns after a new release of flux-accounting.This is a [WIP] PR that adds a new command to flux-accounting:
update-db, which will look at the DB schema of the database currently residing on a system and update it to add any missing tables or add any missing columns within each of the tables. It does so by creating a temporary database to compare schema; it will then compare tables and then iterate through each table and compare columns. After the iteration is done, the temporary database is removed. For testing purposes, there is an optional argument for theupdate-dbcommand to specify a path for the temporary database; if not specified (which should be the norm and is noted in the description of the command), the temporary database is created and removed from the current working directory.Two helper scripts are added to the
t/directory to help with testing:modify_accounting_db.py is a helper script that adds two columns to the association_table and a new table to the flux-accounting DB for testing the functionality of the flux account-update-db command.
check_db_info.py is a helper script that checks the names of the tables in a flux-accounting DB or the names of the columns within a table in a flux-accounting DB.
The sharness test file
t1017-export-db.tuses these two helper scripts to modify a flux-accounting DB to simulate a newer DB being released with an additional table and two additional columns added to one of the existing tables. The test runs theupdate-dbcommand and makes sure the missing table and columns are added to the flux-accounting DB successfully.Fixes #215