Database Re-Structuring

[rtamalin]

Update primary key definitions

Now that we have identified the target primary key definitions we can
set about implementing them as a series of Alembic migrations.

As part of handling this we have to make the following types of changes
to a number of tables:

• add a new sequence based column to act as the primary key
• update nullable settings for some fields, either because
  the existing settings are wrong or because they should be
  nullable, but previously were being used in primary key
  definitions and as such couldn't be configured as nullable.
• change the primary key definition, potentially to an added
  sequence field.

This patch adds a new migration step which implements primary key
changes.

Custom Data Validation

This patch adds custom validation of the servers tables name field,
ensuring that the name field cannot be a null/empty value for an
update server.

XML Data Import Updates

Fixed incremental update to work with tables whose primary key has
been converted to an integer sequence; the search for existing rows
will look for an exact match for all data fields rather that just
the primary key fields.

Misc Changes

Removed defunct orm_exporttopostgres script.

Updated run_standalone.sh to support overriding env settings so as
to avoid the need to edit a committed file to test.

Added git ignorance of the top-level .vscode directory.

Version bumped to 2.0.4

Closes: #74

[jesusbv]

LGTM

[bear454]

Just a nitpick - this isn't relevant to the PR.

[rtamalin]

Not really, and I can move to a separate PR if you want ;-)

[bear454]

Are the (oddball) default values necessary here?

[rtamalin]

Those are the values that @guangyee setup initially and we have continued to maintain them.

The only reason I change these lines was to allow the script to pick up the values already configured/exported in the dev-env rather than having to edit the script if it was necessary to change them.

[bear454]

Just a comment on style - the @validates("name") specificies that we're validating the name attribute; the actual validation performed is that it's not blank on update servers.

Consider def validate_is_not_blank ... and using the key in the error handler.

[bear454]

We could re-use that validation for deprecatedon and deletedon on Images, as well.

[rtamalin]

With regard to deprecatedon and deletedon, I'm assuming that you want to relate the blankness check to the image?

As for the validate_name() validator, since the definition of validator methods is table specific, and the internal implementation is table specific, in that we are checking the self.type field to see if it has a specific value, I'm not sure that it would be worth the effort to finagle things to share an implementation with the equivalent for the images table, as it may actually make the code, which is pretty minimal/simple right now, overly complex.

As regards the choice of validate_name as the name for this validator method, well it is accurate, apt and succint given that we are validating the name field, and then internally the implemented check is checking for specific conditions that apply to the name field. I'm not sure that the same applies for validate_is_not_blank given that the name field can be blank for region servers, which would suggest a name of validate_name_is_not_blank_for_update_servers which is getting somewhat unwieldy.

I suppose we could implement a helper function that raises ValueError if the value is blank, that validate_name could call... But for that to be generic enough to be shareable between valdators it would need to either take the table, key, value and an exception message fragment as arguments so that it could construct the required exception message, or take the value and exception message itself as arguments... Which would greatly increase the complexity of what is currently relatively simple code, for not a whole lot of benefit maintainability wise.

[rtamalin]

Good point on using the key argument in the exception message - I'll fix that now.