Create ExtendedFields

[TaiWilkin]

Overview

Users can submit facility list items individually through the API or
via a list. During parsing, we search for known extended fields and create
ExtendedFields as applicable. Later, we assign a facility id to the
ExtendedFields if one has been assigned to the list item (for MATCHED
and NEW_FACILITY statuses).

For POTENTIAL_MATCH statuses, the item has not yet been assigned to a
facility. When the user confirms or rejects the match, the list item and
ExtendedFields are both linked to a facility.

Extended facility details can also be submitted via facility claim. In
this case, the facility has already been created, so the ExtendedFields
are linked to a facility immediately. A user can later update the
details supplied with their claim; in this case, changed fields are
updated, and removed fields are deleted.

Connects #1493

Notes

In the issue, we list promote as a location where a ExtendedField might need to be updated. However, while the facility is updated to point to a different FacilityListItem, the FacilityListItem (which the ExtendedField is linked to) still points to the same facility, so it doesn't seem that any changes are needed to the ExtendedField in response.

Testing Instructions

• Run ./scripts/manage migrate
• Upload a list, then process it with ./scripts/manage batch_process --list-id 16 --action parse
• On your lists page, get the id of a successfully parsed list item, then look in the ExtendedFields in the admin. There should be five ExtendedFields containing appropriate data. They should not have a facility assigned to them yet.
• Geocode and match the list. A new facility should be created.

./scripts/manage batch_process --list-id 16 --action geocode
./scripts/manage batch_process --list-id 16 --action match

• Get the OAR ID of the new facility, then view the ExtendedFields. They should now have the facility’s OAR ID assigned to them. (If instead of a new facility, a potential match is created, the fields will not be linked yet. Confirm the match to link the fields.)
• Create a facility using the API http://localhost:8081/api/docs/#!/facilities/facilities_create. ExtendedFields should be created and linked to the new facility (If instead of a new facility, a potential match is created, the fields will not be linked yet. Confirm the match to link the fields.)

{
    "country": "China",
    "name": "Nantong Jackbeanie Headwear & Garment Co. Ltd.",
    "address": "No.808,the third industry park,Guoyuan Town,Nantong 226500.",
"number_of_workers": "200 - 1,000",
"native_language_name": "Jackbeanie Headwear"
}

• Claim a facility, then update the claim with facility details including the number of workers and the native language name. ExtendedFields should be created from the claim details.
• Update the claim to have no number of workers listed and updating the native language name. The ExtendedField for the native language name should have been updated, and theExtendedField for the number of workers should have been deleted.

Checklist

• fixup! commits have been squashed
• CI passes after rebase
• CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines

[jwalgran]

Everything is working as described. I like the design of butting a set of helper function in an extended_fields module. I do want to discuss the decision to make ExtendedField for list item core fields.

From the issue:

When parsing a FacilityListItem, look for well known extended field keys in the submitted list item and create ExtendedField rows.

The design in my head when I wrote this was that we would only create number_of_workers and native_language_name extended fields for submitted list items, so I was suprised to see country, name, and address. Name and address do make sense for claims. Did you opt to also create them for list items for consistency?

If we do go ahead with creating extended field rows for all core fields, we will need to add a data migration to create the extended field records for every list item created before this feature. That could be created in a follow up issue/PR if appropriate.

[jwalgran]

We want to allow a contributor to specify a range. Consider adding some regex processing to allow flexibly extracting the min and max

>>> value = '123'
>>> [int(x) for x in re.findall(r'([0-9]+)', value.replace(',', ''))]
[123]
>>> value = '1,00 to 1,500'
>>> [int(x) for x in re.findall(r'([0-9]+)', value.replace(',', ''))]
[100, 1500]

[jwalgran]

See the earlier suggestion about using a regex to extract min and max from the value

[TaiWilkin]

I've updated the number_of_workers to use the suggested regex processing and format the min and max found values.
I was adding the address / etc for consistency, but I agree that it makes more sense to just add those for claims, so I have removed them from the list item processing.

[jwalgran]

Changes work well. 👍

[TaiWilkin]

Thank you for reviewing!