TG2-VALIDATION_COUNTRYSTATEPROVINCE_CONSISTENT

[Tasilee]

TestField	Value
GUID	e654f562-44f8-43fd-983b-2aaba4c6dda9
Label	VALIDATION_COUNTRYSTATEPROVINCE_CONSISTENT
Description	Are the combination of the values of dwc:country, dwc:stateProvince consistent with the values in the bdq:sourceAuthority?
TestType	Validation
Darwin Core Class	Location
Information Elements ActedUpon	dwc:country
	dwc:stateProvince
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country or dwc:stateProvince are EMPTY; COMPLIANT if the value of dwc:stateProvince occurs as an administrative entity that is a child to the entity matching the value of dwc:country in the bdq:sourceAuthority, and the match to dwc:country is an ISO country-like entity in the bdq:sourceAuthority; otherwise NOT_COMPLIANT
Data Quality Dimension	Consistency
Term-Actions	COUNTRYSTATEPROVINCE_CONSISTENT
Parameter(s)	bdq:sourceAuthority
Source Authority	bdq:sourceAuthority default = "The Getty Thesaurus of Geographic Names (TGN)" {[https://www.getty.edu/research/tools/vocabularies/tgn/index.html]}
Specification Last Updated	2023-09-18
Examples	[dwc:country="Australia", dwc:stateProvince="WA": Response.status=RUN_HAS_RESULT, Response.result=COMPLIANT, Response.comment="dwc:country is consistent with dwc:stateProvince. dwc:stateProvince matches Western Australia in dwc:country Australia"]
	[dwc:country="Australia", dwc:stateProvince="Florida": Response.status=RUN_HAS_RESULT, Response.result=NOT_COMPLIANT, Response.comment="dwc:country is not consistent with dwc:stateProvince"]
Source	VertNet, Kurator
References	Chapman, AD and Wieczorek, JR (2020). Georeferencing Best Practices. Copenhagen: GBIF Secretariat (https://doi.org/10.15468/doc-gg7h-s853) VertNet (2019). DwCVocabs (https://github.com/VertNet/DwCVocabs/blob/master/vocabs/Geography.csv) Getty Research Institute (2017). Getty Thesaurus of Geographic Names Online (https://www.getty.edu/research/tools/vocabularies/tgn/index.html)
Example Implementations (Mechanisms)	Kurator
Link to Specification Source Code	https://github.com/kurator-org/kurator-validation/blob/master/packages/kurator_dwca/workflows/dwca_geography_assessor.yaml
Notes	See table #95 (comment). A fail condition may arise from the content being internally inconsistent (not all of the information can be true at the same time), or from the vocabulary being incapable of resolving the combination of term values. This test should match despite leading or trailing whitespace or there are leading or trailing non-printing characters.

[ArthurChapman]

Suggest description be
Are the combination of the values of the terms dwc:country, dwc:stateProvince consistent with values in the bdq:sourceAuthority?

[ArthurChapman]

Suggested Expected Response be altered to (changed in italics)

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if *either of *the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the combination of values of dwc:country and dwc:stateProvince are consistent with the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[Tasilee]

Thanks @ArthurChapman - agreed and changed.

[chicoreus]

Suggest we conform more closely to the >1 logic in @tucotuco's table in #95

Perhaps change from:
EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if either of the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the combination of values of dwc:country and dwc:stateProvince are consistent with the bdq:sourceAuthority; otherwise NOT_COMPLIANT

(1) change "either of" to "both", to allow for a checking for just one value (e.g. WA, row 10 in #95 (comment))

(2) either in the specification or in the notes indicate that "consistent with" means matched at least once.

To:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if both of the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the combination of values of dwc:country and dwc:stateProvince are found in combination at least once the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[chicoreus]

@Tasilee 's comment on #201 #201 (comment) applies here.

Amending the specification to parallel #201, when only a single term is present the test can run, this is a case that is likely to lead to #200 returning COMPLIANT and #201 returning NOT_COMPLIANT:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the combination of values of dwc:country and dwc:stateProvince are consistent with the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[tucotuco]

I like the solution @chicoreus has proposed above. Maybe the expected response should parallel what I proposed for STATEPROVINCE_FOUND. Something like this:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the value of dwc:stateProvince occurs as an administrative entity that is a child to the entity matching the value of dwc:country in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[chicoreus]

Paralell with #199 is good, we can probably tighten up (e.g. prevent compliant when county is mapped onto state province and state province onto country):

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the value of dwc:stateProvince occurs as an administrative entity that is a child to the entity matching the value of dwc:country in the bdq:sourceAuthority, and the match to dwc:country is an ISO country-like entity in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[Tasilee]

Updated Expected Response.

[Tasilee]

Added to Notes: "This test will fail if there are leading or trailing white space or non-printing characters."

[Tasilee]

Changed Expected response from "and" to "or"

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country and dwc:stateProvince are EMPTY; COMPLIANT if the value of dwc:stateProvince occurs as an administrative entity that is a child to the entity matching the value of dwc:country in the bdq:sourceAuthority, and the match to dwc:country is an ISO country-like entity in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the terms dwc:country or dwc:stateProvince are EMPTY; COMPLIANT if the value of dwc:stateProvince occurs as an administrative entity that is a child to the entity matching the value of dwc:country in the bdq:sourceAuthority, and the match to dwc:country is an ISO country-like entity in the bdq:sourceAuthority; otherwise NOT_COMPLIANT

[ArthurChapman]

In the Notes the Reference to "See table #95 (comment)" (i.e. "See table #95 (comment))" will need to be updated - but not sure how we can reference the comment

#95 can be changed to "VALIDATION_GEOGRAPHY_CONSISTENT (78640f09-8353-411a-800e-9b6d498fb1c9)" but the comment and table won't appear there without us putting it somewhere we can reference it.

[Tasilee]

Post Zoom 11/7/2023, I have aligned the Source Authority with the suggested syntax:

bdq:sourceAuthority default = "The Getty Thesaurus of Geographic Names (TGN)" [https://www.getty.edu/research/tools/vocabularies/tgn/index.html]

to

bdq:sourceAuthority default = "The Getty Thesaurus of Geographic Names (TGN)" {[https://www.getty.edu/research/tools/vocabularies/tgn/index.html]}

[Tasilee]

This test should have Data Quality Dimension "Consistency" rather than "Conformance". Edited.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted".

Also changed "Field" to "TestField", "Output Type" to "TestType" and updated "Specification Last Updated"

[chicoreus]

Edited "fail" text to indicate that this complex comparison should not be affected by leading or trailing whitespace.