Provides 2 ways for a table to be dropped (along w/ associated sql manifest rows) when updating the database. This allows for the addition/removal of columns while updating a single table, instead of only being able to do so by rebuilding the whole database.
Also includes a few try/except blocks for the zone_facts table. Add logging.error when a field could not be found/calculated due to missing data - this means the rebuilding of the database will still work even if a necessary table (building_permits, census, etc. ) is missing. Allows for faster testing of things that only apply to certain sections of the database b/c you can add 'skips' to any tables you're not currently working on.
…e unique_data_id results in an empty table, delete the table before loading the data. This allows for the addition/removal of columns in a single table, as the code will recreate the table after it is deleted, using the current version of meta.json Note this will not work for tables that have more than one unique_data_id in them, because the table will never be empty Also handles errors due to missing or not found data in the zone_facts table - if a data field is not found, we want to log the error but don't want to terminate the code, for when this is run on a server.
…pdating the database. This handles the situation where there are multiple unique_data_ids in a database table but we want to update the table structure without rebuilding the entire database.
Before running the command, check the 'crime' table and sql manifest table - there should be 3 crime data sets: 2015, 2016 and 2017.
From the /python/scripts folder, run this command:
What should happen:
To restore your database to original state by adding in the missing data set: