ForeignKey: allow skipIfNone to deal with not required fields

[vlcinsky]

Checking foreignKey on fields helps a lot.

However, if the field to check is not required, the foreignKey check fails with empty values.

Proposal

If foreignKeys[].fields contains a field, which is not required, skip the foreign key check for all the records containing None value on those not required fields.

This way the foreignKeys would be able detecting broken references (using real values, which are not present in the foreign table), but would not fail in case, we do not want to refer anywhere.

[vlcinsky]

One possible solution would be to add to foreignKey another property called skipIfNone which would define rules for skipping checking foreignKey constraint for records having None value:

• True: if any of referring fields (in fields property) is None, skip checking this constraint
• False: (default) always check presence of foreign key even if any of fields is None
• "FieldName": single field name. If field with this value is None, skip checking the constraint. The name must be present in fields property.
• [fieldA, fieldB, fieldC]: array of field names. If any of values for given fields is None, skip checking the constraint. All field names here must be present in fields property.

sample snippet from yaml:

foreignKeys:
  # default "check foreignKey regardless of any field value being None"
  - fields: [CID, TABCD, LCD]
    reference:
      resource: locationcodes
      fields: [CID, TABCD, LCD]
  # explicitly ask for checking foreignKey in all cases
  - fields: [CLASS, TCD, STCD]
    skipIfNone: False
    reference:
      resource: subtypes
      fields: [CLASS, TCD, STCD]
  # To skip foreignKey if any of fields (CID or NID) are None
  - fields: [CID, NID]
    skipIfNone: True
    reference:
      resource: names
      fields: [CID, NID]
   # To skip foreignKey check for records having POL_LCD None
  - fields: [CID, TABCD, POL_LCD]
    skipIfNone: POL_LCD
    reference:
      resource: administrativearea
      fields: [CID, TABCD, LCD]
   # To skip foreignKey check for records having in TABCD or ROADID value None
  - fields: [CID, TABCD, ROADID]
    skipIfNone: [TABCD, ROADID]
    reference:
      resource: roads
      fields: [CID, TABCD, ROADID]