TG2-ISSUE_IDENTIFICATIONQUALIFIER_DETECTED

[iDigBioBot]

TestField	Value
GUID	2eb62922-a8c6-4be7-aeef-56e1ca64c135
Label	ISSUE_IDENTIFICATIONQUALIFIER_DETECTED
Description	An identification qualifier was detected among relevant taxon terms
TestType	ISSUE
Darwin Core Class	Taxon
Information Elements ActedUpon	dwc:scientificName
	dwc:identificationQualifier
	dwc:genericName
	dwc:specificEpithet
	dwc:infaspecificEpithet
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if the taxon name fields are bdq:Empty; ISSUE if any of the fields dwc:identificationQualifier, dwc:genericName, dwc:scientificName, dwc:specificEpithet or dwc:infraspecificEpithet contain an identification qualifier as in the bdq:sourceAuthority; otherwise NOT_ISSUE
Data Quality Dimension	Conformance
Term-Actions	IDENTIFICATIONQUALIFIER_DETECTED
Parameter(s)	bdq:sourceAuthority
Specification Last Updated	2024-08-18
Source Authority	bdq:sourceAuthority default = "Darwin Core dwc:identificationQualifier" {[https://dwc.tdwg.org/terms/#dwc:identificationQualifier]} {Unspecified vocabulary dwc:identificationQualifier}
Examples	[dwc:scientificName="Thomomys cf. bottae", dwc:identificationQualifier="", dwc:genericName="", dwc:specificEpithet="", dwc:infraspecificEpithet="", Response.status=RUN_HAS_RESULT, Response.result=POTENTIAL_ISSUE, Response.comment="dwc:identificationQualifier is bdq:NotEmpty"]
	[dwc:scientificName="Eucalyptus camaldulensis", dwc:identificationQualifier="", dwc:genericName="River Gum", dwc:specificEpithet="", dwc:infraspecificEpithet="", Response.status=RUN_HAS_RESULT, Response.result=NOT_ISSUE, Response.comment="An identification qualifier was not detected"]
Source	VertNet
References	Chapman AD (2005). Principles and Methods of Data Cleaning Primary Species Occurrence Data. Copenhagen: GBIF Secretariat. (https://www.gbif.org/document/80528)
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes	In this test "identification qualifier" has the meaning defined in the definition of dwc:identificationQualifier. dwc:genus is not included, as if a question mark is present in only genus but not scientific name, by the Darwin Core definition of genus, this expresses uncertainty about placement in the classification rather than identification. Can use a vocabulary to detect identificationQualifier as a token, but the resulting identificationQualifier need not necessarily follow a controlled vocabulary.

[iDigBioBot]

Comment by Lee Belbin (@Tasilee) migrated from spreadsheet:
Needs vocab lookup then?

[ArthurChapman]

@chicoreus - that may need to be a different test - which is a NOTIFICATION and this one should be a VALIDATION (i.e. dwc:identificationQualifier is not NULL). Need to check if we have that one.

[ianengelbrecht]

Just a note on the note here, the current Darwin Core definition of genus doesn't include anything about the interpretation of question marks. Does this need to be added?

[ArthurChapman]

@ianengelbrecht - the note to the test mentions why we haven't included "?" in genus here - mainly because it could have two interpretations - 1) uncertainty in identification, 2) uncertainty in placement in heirarchy. One could argue that it was worth including and a flag, but at this stage we have agreed not to include.

[ArthurChapman]

If this test is kep then dwc:identificationQualifier needs to be included in the test.

[ArthurChapman]

As I note under #106 , I vote that we move that test and this one to Supplemental. Alternatively, I see some value in retaining this test to flag any record where the identification is suspect - identifying any of the terms listed in the @pzermoglio (final) list as a source Authority (except for "L." which is also used as the abbreviation for Linnaeus as an author) in any if the fields: dwc:scientificName, dwc:specificEpithet or dwc:infraspecificEpithet, or have a value in dwc:identificationQualifier.

[Tasilee]

Thanks @ArthurChapman. Post discussions today, the key issue is how many false negatives that would result from this VALIDATION; returning COMPLIANT when the record contained some form of identification qualifier.

@pzermoglio 's research suggests > 1000 variants of characters that may indicate an identification qualifier. If it was a long tail distribution where say 10 character combinations detected ~90%, then maybe this would be a useful VALIDATION.

I also don't understand why this VALIDATION is not using dwc:identifictionQualifier. (This may have been raised in our tele today and was indeed mentioned by @ArthurChapman above).

Pondering. Votes by COB May 26 please.

[tucotuco]

I vote to deprecate the test.

…

On Thu, May 21, 2020 at 8:07 PM Lee Belbin ***@***.***> wrote: Thanks @ArthurChapman <https://github.com/ArthurChapman>. Post discussions today, the key issue is how many false negatives that would result from this VALIDATION; returning COMPLIANT when the record contained some form of identification qualifier. @pzermoglio <https://github.com/pzermoglio> 's research suggests > 1000 variants of characters that may indicate an identification qualifier. If it was a long tail distribution where say 10 character combinations detected ~90%, then maybe this would be a useful VALIDATION. I also don't understand why this VALIDATION is not using dwc:identifictionQualifier. (This may have been raised in our tele today and was indeed mentioned by @ArthurChapman <https://github.com/ArthurChapman> above). Pondering. Votes by COB May 26 please. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#97 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQ7226V4UOR34ZLQUJJK3RSWXZ3ANCNFSM4EKSQVAQ> .

[chicoreus]

I concur with moving this test from core to supplemental. Implementation of effective detection of an identification qualifier, when the identificationQualifier term is empty, is non-trivial, and it is unclear what data quality needs are addressed that would not be addressed by the consumer of the data asking a simple question such as is identificationQualifier empty and specificEpithet contains exactly one word.

[chicoreus]

Rephrasing as issue, updating markdown table to more closely conform with current usage. Marking as immature given absence of controlled vocabulary and discussion of immaturity above.

[ArthurChapman]

Example needs updating to conform with current usage.

[Tasilee]

Added examples to conform to current template

[Tasilee]

Changed Test to TestField and added Description

[Tasilee]

Standardized reference to "EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available" in Expected Response and tried to standardize bdq:sourceAuthority