Update Schema Generator for handling special cases where ancestor classes are needed

[jordanpadams]

pds:Internal_Reference/pds:lid_reference and pds:Internal_Reference/pds:reference_type are completely useless without the context of the ancestor class. We need to build in a mechanism to enable these "special cases" where we need additional ancestor classes to be included.

Observing System component is another special cases that is pretty useless without some massaging of the data.

Acceptance criteria:
When a pds4 label is loaded in the registry
Then the different references are stored in the document in attributes like "ref_lid_", for example ref_lid_intruments, ref_lib_investigation...

[tdddblog]

There are much more use cases like this. Multiple nesting example (a shapefile label).
The data doesn't make any sense when it is flattened out:

<Table_Character>
<local_identifier>VERTICES_FACETS_TABLE</local_identifier>
<Record_Character>
<Field_Character>
<name>NUMBER_OF_VERTICES</name>
...
<Table_Character>
<local_identifier>VERTEX_COORDINATE_TABLE>/local_identifier>
<Record_Character>
<Field_Character>
<name>X</name>
...

[jordanpadams]

You are able to accomplish this through Harvest config

[rchenatjpl]

If I'm not testing this correctly, please say so. From the attached, I harvest then registry-manager load-data. http://localhost:9200/registry/_search?q=* shows the attached .png. The fact that the ref_lid_* are unique tests this issue?

OK, then I added *.bad, which adds two more lid_references (is_telescope and is_facility), both not in registry.json. 1) registry-manager load-data annoyingly throws 1 ERROR at a time instead of both. 2) Is the user responsible for creating an alternative to registry.json? If so, please point me to the right documentation. https://nasa-pds.github.io/pds-registry-app/operate/reg-custom.html didn't do it for me.
Archive.zip

[tloubrieu-jpl]

@jordanpadams @tdddblog should this ticket be tagged as wontfix ? Thanks

Or maybe it is what led to the creation of the following type of fields in elasticsearch:
"ref_lid_instrument": [
"urn:nasa:pds:context:instrument:grs.mess",
"urn:nasa:pds:context:instrument:mascs.mess",
"urn:nasa:pds:context:instrument:ns.mess"
],
"ref_lid_instrument_host": "urn:nasa:pds:context:instrument_host:spacecraft.mess",
"ref_lid_investigation": "urn:nasa:pds:context:investigation:mission.messenger",
"ref_lid_target": "urn:nasa:pds:context:target:planet.mercury",

I am trying to find out what acceptance criteria should provided for the I&T.

[jordanpadams]

@tloubrieu-jpl yes, those fields are what was implemented here.

[jordanpadams]

@tloubrieu-jpl per above, it looks like @rchenatjpl already tested this last I&T?

[tloubrieu-jpl]

@jordanpadams yes I've seen that at least he attempted to test ! Do you think I should ignore this ticket for this build ? It came up I guess because it was re-opened after 2020-10-26T00:00:00Z time of the previous build...

[jordanpadams]

@tloubrieu-jpl yea let's ignore this ticket. either tagging it as "untestable" or maybe we have some other label like "ignore" to make sure it doesn't show up in the report? i will leave it to you.