TG2-AMENDMENT_COORDINATES_FROM_VERBATIM

[iDigBioBot]

TestField	Value
GUID	3c2590c7-af8a-4eb4-af57-5f73ba9d1f8e
Label	AMENDMENT_COORDINATES_FROM_VERBATIM
Description	Proposes an amendment to the values of dwc:decimalLatitude, dwc:decimalLongitude, and dwc:geodeticDatum from geographic coordinate information in the verbatim coordinates terms.
TestType	Amendment
Darwin Core Class	dcterms:Location
Information Elements ActedUpon	dwc:decimalLatitude
	dwc:decimalLongitude
	dwc:geodeticDatum
Information Elements Consulted	dwc:verbatimCoordinates
	dwc:verbatimLatitude
	dwc:verbatimLongitude
	dwc:verbatimCoordinateSystem
	dwc:verbatimSRS
Expected Response	INTERNAL_PREREQUISITES_NOT_MET if 1) either dwc:decimalLatitude or dwc:decimalLongitude are bdq:NotEmpty, or 2) dwc:verbatimCoordinates and one of dwc:verbatimLatitude and dwc:verbatimLongitude are bdq:Empty; FILLED_IN the values of dwc:decimalLatitude, dwc:decimalLongitude and dwc:geodeticDatum (provided that the dwc:verbatimCoordinates can be unambiguously interpreted as geographic coordinates) from 1) dwc:verbatimLatitude, dwc:verbatimLongitude and dwc:verbatimSRS or 2) dwc:verbatimCoordinates and dwc:verbatimSRS; otherwise NOT_AMENDED.
Data Quality Dimension	Completeness
Term-Actions	COORDINATES_FROM_VERBATIM
Parameter(s)
Source Authority
Specification Last Updated	2024-08-20
Examples	[dwc:verbatimLatitude="-23.712", dwc:verbatimLongitude="139.92", dwc:verbatimCoordinates="", dwc:verbatimSRS="EPSG:4326", dwc:verbatimCoordinateSystem="decimal degrees", dwc:decimalLatitude="", dwc:decimalLongitude="": Response.status=FILLED_IN, Response.result=dwc:decimalLatitude="-23.712", dwc:decimalLongitude="139.92", dwc:geodeticDatum="EPSG:4326", Response.comment="Input fields contain interpretable values"]
	[dwc:verbatimLatitude="", dwc:verbatimLongitude="", dwc:verbatimCoordinates="54K 0390210 7377243", dwc:verbatimSRS="EPSG:32754", dwc:verbatimCoordinateSystem="decimal degrees", dwc:decimalLatitude="", dwc:decimalLongitude="":: Response.status=NOT_AMENDED, Response.result="", Response.comment="In the wrong coordinate system"]
Source	ALA
References	Chapman AD and Wieczorek, JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat. https://doi.org/10.15468/doc-gg7h-s853
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes	Transformations between coordinate reference systems should not be made as a part of this test. Though coordinate precision of the verbatim coordinates could also be interpreted during the process of amending decimal coordinates from verbatim coordinates, that amendment is recommended to be an independent test. Note that dwc:verbatimLatitude, dwc:verbatimLongitude and dwc:verbatimCoordinates might all be populated, and they may or not be perfectly consistent with each other. An ideal implementation should check for the consistency of these three fields and not amend them if they are inconsistent.

[iDigBioBot]

Comment by Anonymous migrated from spreadsheet:
None

[ArthurChapman]

Do we need to make a prerequisite that either or both dwc:decimalLatitude and dwc:decimalLongitude are EMPTY?

[tucotuco]

Agreed at TDWG 2018 DQIG meeting that the name TG2-AMENDMENT_COORDINATES_FROM_VERBATIM is satisfactory.

[Tasilee]

In preparing test data, it would seem that the Expected Response should be

INTERNAL_PREREQUISITES_NOT_MET if Verbatim coordinates (either dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates) were not interpretable into coordinates as decimal degrees or Verbatim coordinates were EMPTY or either dwc:decimalLatitude or dwc:decimalLongitude was not EMPTY; AMENDED if dwc:decimalLatitude and dwc:decimalLongitude were populated from information in verbatim coordinate information (dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS); otherwise NOT_CHANGED

?

[ArthurChapman]

Makes sense to me

[tucotuco]

Similar to the countryCode amendment situation. Could we not make an interpretation for the decimal coordinates if they are NOT EMPTY, but also not valid?

…

On Mon, Nov 1, 2021 at 12:48 AM Arthur Chapman ***@***.***> wrote: Makes sense to me — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQ72YECLZR7LKMPLULDCTUJYE2BANCNFSM4EKSLOSA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Tasilee]

Thanks @tucotuco. One wonders where we draw the line. In the case of #73, I figure it is 'relatively easy' to lookup dwc:countryCode but there must be infinite 'shades of grey' comparing verbatim coordinates (which could be anything) to existing values for dwc:decimalLatitude and dwc:decimalLongitude? Or, like I've suggested in #73, IF we have valid interpretable verbatim coordinates, we overwrite dwc:decimalLatitude and dwc:decimalLongitude? I feel uncomfortable with that strategy.

I'd say VALID dwc:decimalLatitude and dwc:decimalLongitude take precedence over verbatim coordinates, so any differences would tend to deprecate the verbatim's potential to amend.

IF there was additional spatial information as in dwc:country or dwc:countryCode, and the error in dwc:decimalLatitude or dwc:decimalLongitude was 'gross' (e.g., sign swap or obvious degree differences), a fix could be applied.

We are talking AI :)

[ArthurChapman]

If you have VALID dwc:decimalLatitude and dwc:decimalLongitude then we don't run this test do we?

I think we need a meeting to sort some of this out.

[ArthurChapman]

I am not sure that the AMENDMENTS are entirely stand alone or we wouldn't need Validations. In many (all?) cases we only run an amendment if a VALIDATION fails don't we?

[Tasilee]

@ArthurChapman : Yes, I am presuming an AMENDMENT will only be run if the VALIDATION reports "NOT_COMPLIANT".

I will continue with the test data for the last 5 AMENDMENTs for SPACE, and then get file out to the group. I've completed the data for AMENDMENTS for NAME, TIME and OTHER. The test data certainly helps to clarify the Expect Responses.

[tucotuco]

I am not sure that the AMENDMENTS are entirely stand alone or we wouldn't need Validations. In many (all?) cases we only run an amendment if a VALIDATION fails don't we?

I disagree. Again, I think we are confounding the tests themselves with what we expect to be sensible ways to string them together. We shouldn't. The process uses the tests, the tests don't depend on the process. They are separate levels of product.

Agreeing with @chicoreus, the tests must be be able to stand independently.

[ArthurChapman]

I think this needs discussing in a forum - we seem to have gone in a circle here in the relation between VALIDATIONS and AMENDMENTS. It has been so long between discussions, I seem to be getting confused as to where we have been and where we are going. We spent a lot of time making sure that AMENDMENTS had corresponding VALIDATIONS. If AMENDMENTS are entirely stand alone - then why do we need many of the VALIDATIONS? We could include the failure state within the AMENDMENT - i.e. reporting more fully why INTERNAL_PREREQUESITES failed - extending that to explain why, could remove the need for the VALIDATION. This would mean a total revisit to the way we do the tests and I don't recommend we go there. I can see lots of places running the VALIDATIONS and not the AMENDMENTS.

I think this needs a lot more discussion - including what you mean by standalone in this case.

[tucotuco]

I'm fine with a live discussion. By stand-alone, I mean that any given test has no dependence on the existence or execution of any other test. AMENDMENTS don't have to be alone, but they must be able to be. The VALIDATIONS need to be in place to inform. It doesn't really matter in implementation if some code is shared between them. I think the two need to remain as independent tests.

…

On Tue, Nov 2, 2021 at 5:39 PM Arthur Chapman ***@***.***> wrote: I think this needs discussing in a forum - we seem to have gone in a circle here in the relation between VALIDATIONS and AMENDMENTS. It has been so long between discussions, I seem to be getting confused as to where we have been and where we are going. We spent a lot of time making sure that AMENDMENTS had corresponding VALIDATIONS. If AMENDMENTS are entirely stand alone - then why do we need many of the VALIDATIONS? We could include the failure state within the AMENDMENT - i.e. reporting more fully why INTERNAL_PREREQUESITES failed - extending that to explain why, could remove the need for the VALIDATION. This would mean a total revisit to the way we do the tests and I don't recommend we go there. I can see lots of places running the VALIDATIONS and not the AMENDMENTS. I think this needs a lot more discussion - including what you mean by standalone in this case. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQ727WNKQTI5F5E5OUQCDUKBD7BANCNFSM4EKSLOSA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Tasilee]

I like the philosophy of stand-alone, as the dependencies are truly legion, as we started to discuss in Gainesville. We do however have at least one 'process': Run VALIDATIONs, run AMENDMENTs, re-run VALIDATIONs.

[tucotuco]

Yes, there is the level of suggested workflows using the tests that we have not fully explored. It may be beyond our scope to do so.

…

On Wed, Nov 3, 2021 at 6:47 PM Lee Belbin ***@***.***> wrote: I like the philosophy of stand-alone, as the dependencies are truly legion, as we started to discuss in Gainesville. We do however have at least one 'process': Run VALIDATIONs, run AMENDMENTs, re-run VALIDATIONs. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQ7245MKROLOC6DLPQ2Q3UKGUXRANCNFSM4EKSLOSA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[ArthurChapman]

In the Expected Response we have "information" twice in the same sentence - is this OK or need changing

"were populated from information in verbatim coordinate information"

[Tasilee]

I would skip the "Verbatim coordinates" and just cut to the chase (dwc:....) in all cases, nd skip the second information.

[ArthurChapman]

Not quite that simple, because "verbatim coordinates" cover a multitude of sins (dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS)

[Tasilee]

My point is if we are angling to being specific, I would much prefer to use "dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates" rather than "verbatim coordinates (dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates)"

[chicoreus]

I'd agree with @Tasilee, in the expected response (specification), it is clearer to reference the specific information elements than a general information element that we may have defined elsewhere. Phrasing just need to be clear that intent is (dwc:verbatimLatitude and dwc:verbatimLongitude) or dwc:verbatimCoordinates.

[ArthurChapman]

Don't forget the "plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS"

I have edited accordingly

[Tasilee]

In accordance with discussions 16 April, I have edited the Expected Response to

INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude was not EMPTY, or either dwc:verbatimLatitude and dwc:verbatimLongitude, or dwc:verbatimCoordinates were not unambiguously interpretable into valid coordinates; FILLED_IN the values of dwc:decimalLatitude and dwc:decimalLongitude if unambiguous values can be interpreted from dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS; otherwise NOT_AMENDED

I added an IF phrasing after the FILLED_IN...to align with current usage

[chicoreus]

Changed case of "epsg" to "EPSG" to match usual usage of this pseudo-namespace. Added an example of a conversion of a verbatim UTM coordinate.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted"

[Tasilee]

This test is a mess.

First up, I believe we are FILLING_IN dwc:decimalLatitude, and dwc:decimalLongitude from dwc:verbatimCoordinates and dwc:verbatimLatitude and dwc:verbatimLongitude.

We are NOT converting with this AMENDMENT.

The examples are crap. Don't know why, but they are.

dwc:verbatimSRS and dwc:verbatimCoordinateSystem are Information Elements CONSULTED. Values in either could assist INTERPRETATION of dwc:verbatimCoordinates, dwc:verbatimLatitude and dwc:verbatimLongitude.

Needs serious discussion. My take is to get it simple if we can.

[tucotuco]

Changed Expected Response from:

INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude were not EMPTY, or either dwc:verbatimLatitude and dwc:verbatimLongitude, or dwc:verbatimCoordinates were not unambiguously interpretable into valid coordinates; FILLED_IN the values of dwc:decimalLatitude and dwc:decimalLongitude if unambiguous values can be interpreted from dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS; otherwise NOT_AMENDED

to:

INTERNAL_PREREQUISITES_NOT_MET if 1) either dwc:decimalLatitude or dwc:decimalLongitude were not EMPTY, or 2) dwc:verbatimLatitude and dwc:verbatimLongitude and dwc:verbatimCoordinates were all EMPTY; FILLED_IN the values of dwc:decimalLatitude, dwc:decimalLongitude and dwc:geodeticDatum (provided that the verbatim coordinates can be unambiguously interpreted as geographic coordinates) from a) dwc:verbatimCoordinates and dwc:verbatimSRS or b) dwc:verbatimLatitude, dwc:verbatimLongitude and dwc:verbatimSRS; otherwise NOT_AMENDED.

This change is to highlight that the coordinate reference system is an integral component of the coordinates and must accompany them from verbatim to the decimal format.

Notes also updated to emphasize that coordinate transformations are not part of this test.

[tucotuco]

Further amended the Expected Response from:

INTERNAL_PREREQUISITES_NOT_MET if 1) either dwc:decimalLatitude or dwc:decimalLongitude were not EMPTY, or 2) dwc:verbatimLatitude and dwc:verbatimLongitude and dwc:verbatimCoordinates were all EMPTY; FILLED_IN the values of dwc:decimalLatitude, dwc:decimalLongitude and dwc:geodeticDatum (provided that the verbatim coordinates can be unambiguously interpreted as geographic coordinates) from a) dwc:verbatimCoordinates and dwc:verbatimSRS or b) dwc:verbatimLatitude, dwc:verbatimLongitude and dwc:verbatimSRS; otherwise NOT_AMENDED.

to:

INTERNAL_PREREQUISITES_NOT_MET if 1) either dwc:decimalLatitude or dwc:decimalLongitude were not EMPTY, or 2) dwc:verbatimLatitude and dwc:verbatimLongitude and dwc:verbatimCoordinates were all EMPTY; FILLED_IN the values of dwc:decimalLatitude, dwc:decimalLongitude and dwc:geodeticDatum (provided that the dwc:verbatimCoordinates can be unambiguously interpreted as geographic coordinates) from 1) dwc:verbatimLatitude, dwc:verbatimLongitude and dwc:verbatimSRS or 2) dwc:verbatimCoordinates and dwc:verbatimSRS; otherwise NOT_AMENDED.

This change was to change the suggested order of interpretation to prioritize dwc:verbatimLatitude and dwc:Longitude over dwc:verbatimCoordinates because the former is more explicit, and to change a) and b) for 1) and 2) for consistency.

Also added a note about the consistency of dwc:verbatimLatitude and dwc:Longitude with dwc:verbatimCoordinates.

[Tasilee]

Changed to two Examples: One that infers priority in interpreting verbatim latitude and longitude over dwc:verbatimCoordinates and one that infers a coordinate system conversion which this test doesn't support.

[chicoreus]

Suggested change from dataID 113

from

dwc:verbatimLatitude and dwc:verbatimLongitude and dwc:verbatimCoordinates were all EMPTY;

to

dwc:verbatimCoordinates and one of dwc:verbatimLatitude and dwc:verbatimLongitude were EMPTY;

[ArthurChapman]

Updated Expected Response in line with above comment and updated "Specification Last Updated"