TG2-AMENDMENT_GEODETICDATUM_STANDARDIZED

[iDigBioBot]

TestField	Value
GUID	0345b325-836d-4235-96d0-3b5caf150fc0
Label	AMENDMENT_GEODETICDATUM_STANDARDIZED
Description	Propose amendment to the value of dwc:geodeticDatum using bdq:sourceAuthority.
TestType	Amendment
Darwin Core Class	Location
Information Elements ActedUpon	dwc:geodeticDatum
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority was not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is EMPTY; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a value in bdq:sourceAuthority; otherwise NOT_AMENDED
Data Quality Dimension	Conformance
Term-Actions	GEODETICDATUM_STANDARDIZED
Parameter(s)	dwc:geodeticDatum vocabulary
Source Authority	bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}
Specification Last Updated	2023-09-17
Examples	[dwc:geodeticDatum="WGS84": Response.status=AMENDED, Response.result=dwc:geodeticDatum="EPSG:4326", Response.comment="dwc:geodeticDatum contains a valid code in the bdq:sourceAuthority"]
	[dwc:geodeticDatum="WGS8": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:geodeticDatum contains an ambiguous value"]
Source	Paul Morris
References	Maptiler (2019). EPSG.io (https://epsg.io) IOGP (2019). About the EPSG Dataset (https://epsg.org/) Spatial Reference (2013). Find your references in any number of formats (https://spatialreference.org) Geomatic Solutions (2018). Georepository. Version 9.0.0.1062 (https://georepository.com) Chapman, AD and Wieczorek, JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat (https://doi.org/10.15468/doc-gg7h-s853)
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes	Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326".

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
Split from GEODETIC_DATUM_ASSUMED_DEFAULT (AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT)

[chicoreus]

There's a significant danger in translating a string "WGS 84" to an epsg code, in that the value used in dwc:geodeticDatum might have been intended to mean the datum, or the elipsoid, or the coordinate system. Translating dwc:geodeticDatum=WGS84 to epsg:6326 may not have been the intent, epsg:7030 (the elipsoid) or epgs:4326 (the geodetic coordinate system) may have been the intent.

[chicoreus]

Parameter is not needed for this test, the vocabulary expected by dwc;geodeticDatum is the EPSG vocabulary. Parameter might be needed to specify if the expected values are in the form https://epsg.io/4326, EPSG:4326, or WGS84, but the EPSG vocabulary is the one that everyone converges on, and different user communities are not likely to want different vocabularies for this test. The specification of the vocabulary should go into the specification or into the notes, not a parameter.

[Tasilee]

I agree @chicoreus. If there is no choice, there is no Parameter. I would however add "http://epsg.io/" to the References.

[tucotuco]

I wholeheartedly disagree. The source authority does nothing to help us amend. We need a lookup source, and that ought to be given, or it ought to be developed as a vocabulary of values and set in stone as the source the way you are trying to do here with epsg.io.

[Tasilee]

@tucotuco. This relates directly to #178 obviously. We either assume test code pattern matching against a vocab of values or we can't amend. I can't see a thesaurus ever being comprehensive enough to handle lookup of all the user values (because users are so ingenious - https://github.com/tdwg/dwc-qa/tree/master/data).

So, I still argue that this test is not parameterised as there is one bdq:sourceAuthority and we are interpreting against values in that authority. With #102 we have my 'Parameterised scenario 2' where there is an option as to which geodetic datum would be appropriate.

[tucotuco]

I maintain that my argument applies to every vocabulary or values. We don't have to anticipate every possible variation, we have to respond to the ones that surface. We already have a good start on this at https://github.com/VertNet/DwCVocabs/blob/master/vocabs/geodeticDatum.csv, based on what came through VertNet migrators to date. We can do the same with the GBIF distinct values.

[Tasilee]

@tucotuco: Given your well-reasoned arguments in #178, I'm a convert to thesauri.

Let's figure where this AMENDMENT sits.

It isn't Parameterized due to bdq:sourceAuthority as we are saying there is no choice.

It is Parameterized because the test installer has to choose a default geodetic datum.

Is it Parameterised due to the need for a choice of thesauri?

[tucotuco]

To me, the defaultDatum parameter is right, and there will a default defaultDatum of epsg:4326. The source of the controlled vocabulary is https://epsg.io/4326, but not every code in epsg is viable as a value for geodeticDatum. So, even in the case of VALIDATION tests, nothing is telling implementors what to do with https://epsg.io/4326. Thus, to me, a thesaurus that can tell people the viable and preferred values is essential. If it can interpret completely non-standard values as well, it will be really useful. The only remaining question is whether to proscribe one for the test. That seems not very open to community choice or the vagaries of time, Thus, to me, a parameter allowing one to say what service they want to use is important, and hurts nothing if there is only one in existence, especially if the test is supposed to provide a default source authority. Maybe the big problem is "source authority". Sure, we want to use epsg, but we can't as is. We need a source authority that works for what the test needs,

[ArthurChapman]

1). Perhaps we need to rewrite the example in this one to:
dwc:geodeticDatum is "WGS:84", propose amendment dwc:geodeticDatum="epsg:4326"

2). @tucotuco epsg.io does have an API ("The web has also API allowing integration of the search functionality and transformations in third party applications." - https://epsg.io/about) but not sure if this gives us what we want - I have not looked at it. We do also have the list of currently acceptable datums in the appendix to our in prep Georeferencing paper. This doesn't include synonyms. I don't think it is our job to develop and maintain a thesaurus of datums though. Can we loby EPSG to develop what we need (small fry that we are)?

[tucotuco]

epsg.io does have a REST API. See https://github.com/maptiler/epsg.io.

[tucotuco]

1). Perhaps we need to rewrite the example in this one to:
dwc:geodeticDatum is "WGS:84", propose amendment dwc:geodeticDatum="epsg:4326"

Agreed and updated.

2). @tucotuco epsg.io does have an API ("The web has also API allowing integration of the search functionality and transformations in third party applications." - https://epsg.io/about) but not sure if this gives us what we want - I have not looked at it. We do also have the list of currently acceptable datums in the appendix to our in prep Georeferencing paper. This doesn't include synonyms. I don't think it is our job to develop and maintain a thesaurus of datums though. Can we loby EPSG to develop what we need (small fry that we are)?

There is still a problem here with the source authority. epsg.io is not going to be able to do the standardization, they only have the controlled vocabulary, not the thesaurus. GBIF is committed to having the datum thesaurus in the vocabulary services with lookups to epsg codes. I have volunteered to do the mappings.

I updated the Notes as well, from
'[bdq:sourceAuthority = https://epsg.io]. Darwin Core recommends best practice is to use the EPSG code as a controlled vocabulary to provide a CRS, if known. Otherwise use a controlled vocabulary for the name or code of the geodetic datum, if known. Otherwise use a controlled vocabulary for the name or code of the ellipsoid, if known. If none of these is known, use the value "unknown". The amended value should be an EPSG code for a geodetic datum or an ellipsoid. The reference vocabularies of values for geodetic datums and ellipsoids need to be developed and published and include alternative representations of the datum strings, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "WGS84".'.

to

'[bdq:sourceAuthority = GBIF geodeticDatum thesaurus, when available]. Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326".'

[ymgan]

Hello,

I noticed that under the notes of this issue and the Georeferencing Best Practice, the following is mentioned

If none of these is known, use the explicit value "not recorded".

However in Darwin Core Quick Reference Guide for the term geodeticDatum, it says

If none of these is known, use the value unknown.

Would it be sensible to align the tests, Darwin Core documentation and the Best Practice Guide? Thank you!

[Tasilee]

@tucotuco ?

[tucotuco]

It's an unfortunate inconsistency. The recommendation for "not recorded" has been around a lot longer and is slightly more correct more often, because sometimes a datum in knowable even if not recorded. I recommend, "not recorded" and think an issue should be opened for Darwin Core with the suggestion to make the non-normative change to the comment and examples for Darwin Core.

[ArthurChapman]

EPSG:4326 changed to epsg:4326 in Example

[ArthurChapman]

Changed epsg:4326 to EPSG:4326 in Notes

[Tasilee]

Amended the Source Authority entry to align with @chicoreus syntax

bdq:sourceAuthority = GBIF geodeticDatum thesaurus, when available

to

bdq:sourceAuthority default = "GBIF geodeticDatum thesaurus" (when available)

[ArthurChapman]

Why aren't we using epsg.io for this one as we do other Datum references? I remember some discussion about this, bnut it doesn't look consistent.

[Tasilee]

Fair question

[tucotuco]

Because epsg.io is not a folksonomy with synonyms. The source for this one has to be something that can give us a standard epsg code for a string that is unambiguously mappable to that epsg code. EPSG.io is useless for this.

[ArthurChapman]

On advice from EPSG - changed in References: "https://www.epsg.org/" to "https://epsg.org/"

[tucotuco]

The work I did for geodeticDatum lookup is neither used nor exposed by GBIF currently (see gbif/vocabulary#71 (comment)). The CSV file that let's you do a lookup can be found at https://media.githubusercontent.com/media/VertNet/DwCVocabs/master/vocabs/geodeticDatum.csv

[Tasilee]

That is an amazing piece of work @tucotuco! Well done!

One point relating to #59 based on recent discussions, I presume we would still use something like https://apps.epsg.org/api/swagger/ui/index#/Datum as an endpoint?

[Tasilee]

Changed Parameter(s) from

bdq:geodeticDatum

to

dwc:geodeticDatum vocabulary

and changed Source Authority from

bdq:sourceAuthority = "EPSG" {[https://epsg.org/]} {API endpoint for dwc:geodeticDatum [https://geodeticDatum to/EPSG/code/thesaurus/..." (TBA)

to

bdq:sourceAuthority = "EPSG" {[https://epsg.org/]} {dwc:geodeticDatum vocabulary [https://media.githubusercontent.com/media/VertNet/DwCVocabs/master/vocabs/geodeticDatum.csv]}

[tucotuco]

One point relating to #59 based on recent discussions, I presume we would still use something like https://apps.epsg.org/api/swagger/ui/index#/Datum as an endpoint?

Yes. We can use the epsg code found to test if it is a standard code via that endpoint.

[ArthurChapman]

Does this mean we can delete bdq:geodeticDatum from the Vocabulary? I don't think we use it anywhere else.

[Tasilee]

@ArthurChapman: Indeed, there are no references to bdq:geodeticDatum.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted". Also changed "Field" to "TestField" and "Output Type" to "TestType".

[Tasilee]

Changed Source Authority from

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}

to

bdq:sourceAuthority default = "GBIF GeodeticDatum Vocabulary" [https://api.gbif.org/v1/vocabularies/GeodeticDatum]} {"dwc:lifeStage vocabulary API" [https://api.gbif.org/v1/vocabularies/GeodeticDatum/concepts]}

[ArthurChapman]

@Tasilee - some errors in that - in the second part you mention Life Stage - I thought we were following EPSG and not GBIF for GeodeticDatum - or have I missed something @tucotuco ?

[tucotuco]

EPSG for dwc:geodeticDatum

[Tasilee]

Reverted to EPSG