TG2-AMENDMENT_GEODETICDATUM_STANDARDIZED

[iDigBioBot]

TestField	Value
GUID	0345b325-836d-4235-96d0-3b5caf150fc0
Label	AMENDMENT_GEODETICDATUM_STANDARDIZED
Description	Proposes an amendment to the value of dwc:geodeticDatum using the bdq:sourceAuthority.
TestType	Amendment
Darwin Core Class	dcterms:Location
Information Elements ActedUpon	dwc:geodeticDatum
Information Elements Consulted
Expected Response	EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, Ellipsoid or a CRS appropriate for a 2D geographic coordinate in degrees, or as the value "not recorded"; otherwise NOT_AMENDED
Data Quality Dimension	Conformance
Term-Actions	GEODETICDATUM_STANDARDIZED
Parameter(s)
Source Authority	bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}
Specification Last Updated	2025-03-03
Examples	[dwc:geodeticDatum="WGS84": Response.status=AMENDED, Response.result=dwc:geodeticDatum="EPSG:4326", Response.comment="dwc:geodeticDatum contains a valid code in the bdq:sourceAuthority"]
	[dwc:geodeticDatum="WGS8": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:geodeticDatum contains an ambiguous value"]
Source	Paul Morris
References	Maptiler (2019) EPSG.io. https://epsg.io/ EPSG (2024) About the EPSG Dataset. https://epsg.org/ Spatial Reference (2024) What is SpatialReference.org. https://spatialreference.org/ Geomatic Solutions (2018) Georepository. Version 9.0.0.1062. https://georepository.com/ Chapman AD and Wieczorek JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat. https://doi.org/10.15468/doc-gg7h-s853 Wieczorek C and Wieczorek J (2021) Georeferencing Calculator. http://georeferencing.org/georefcalculator/gc.html
Example Implementations (Mechanisms)	Kurator/FilteredPush geo_ref_qc Library DOI: 10.5281/zenodo.14064324
Link to Specification Source Code	https://github.com/FilteredPush/geo_ref_qc/blob/v2.0.1/src/main/java/org/filteredpush/qc/georeference/DwCGeoRefDQ.java#L1568
Notes	Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io/) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326". NB. Do NOT change one datum to any other datum no matter how close they are or may appear to be. The same treatment should be given to all datums, which is to use their transformation algorithms to get the equivalent in epsg:4326. For reference, a vocabulary of synonyms for EPSG codes for values of dwc:geodeticDatum can be found at https://registry.gbif.org/vocabulary/GeodeticDatum/concepts and and for more information on obtaining the EPSG dataset, see https://docs.geotools.org/latest/userguide/library/referencing/epsg.html. For the purposes of this test "not recorded" is not a value in the bdq:sourceAuthority and should result in NOT_AMENDED.

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
Split from GEODETIC_DATUM_ASSUMED_DEFAULT (AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT)

[chicoreus]

There's a significant danger in translating a string "WGS 84" to an epsg code, in that the value used in dwc:geodeticDatum might have been intended to mean the datum, or the elipsoid, or the coordinate system. Translating dwc:geodeticDatum=WGS84 to epsg:6326 may not have been the intent, epsg:7030 (the elipsoid) or epgs:4326 (the geodetic coordinate system) may have been the intent.

[chicoreus]

Parameter is not needed for this test, the vocabulary expected by dwc;geodeticDatum is the EPSG vocabulary. Parameter might be needed to specify if the expected values are in the form https://epsg.io/4326, EPSG:4326, or WGS84, but the EPSG vocabulary is the one that everyone converges on, and different user communities are not likely to want different vocabularies for this test. The specification of the vocabulary should go into the specification or into the notes, not a parameter.

[Tasilee]

I agree @chicoreus. If there is no choice, there is no Parameter. I would however add "http://epsg.io/" to the References.

[tucotuco]

I wholeheartedly disagree. The source authority does nothing to help us amend. We need a lookup source, and that ought to be given, or it ought to be developed as a vocabulary of values and set in stone as the source the way you are trying to do here with epsg.io.

[Tasilee]

@tucotuco. This relates directly to #178 obviously. We either assume test code pattern matching against a vocab of values or we can't amend. I can't see a thesaurus ever being comprehensive enough to handle lookup of all the user values (because users are so ingenious - https://github.com/tdwg/dwc-qa/tree/master/data).

So, I still argue that this test is not parameterised as there is one bdq:sourceAuthority and we are interpreting against values in that authority. With #102 we have my 'Parameterised scenario 2' where there is an option as to which geodetic datum would be appropriate.

[tucotuco]

I maintain that my argument applies to every vocabulary or values. We don't have to anticipate every possible variation, we have to respond to the ones that surface. We already have a good start on this at https://github.com/VertNet/DwCVocabs/blob/master/vocabs/geodeticDatum.csv, based on what came through VertNet migrators to date. We can do the same with the GBIF distinct values.

[Tasilee]

@tucotuco: Given your well-reasoned arguments in #178, I'm a convert to thesauri.

Let's figure where this AMENDMENT sits.

It isn't Parameterized due to bdq:sourceAuthority as we are saying there is no choice.

It is Parameterized because the test installer has to choose a default geodetic datum.

Is it Parameterised due to the need for a choice of thesauri?

[tucotuco]

To me, the defaultDatum parameter is right, and there will a default defaultDatum of epsg:4326. The source of the controlled vocabulary is https://epsg.io/4326, but not every code in epsg is viable as a value for geodeticDatum. So, even in the case of VALIDATION tests, nothing is telling implementors what to do with https://epsg.io/4326. Thus, to me, a thesaurus that can tell people the viable and preferred values is essential. If it can interpret completely non-standard values as well, it will be really useful. The only remaining question is whether to proscribe one for the test. That seems not very open to community choice or the vagaries of time, Thus, to me, a parameter allowing one to say what service they want to use is important, and hurts nothing if there is only one in existence, especially if the test is supposed to provide a default source authority. Maybe the big problem is "source authority". Sure, we want to use epsg, but we can't as is. We need a source authority that works for what the test needs,

[ArthurChapman]

1). Perhaps we need to rewrite the example in this one to:
dwc:geodeticDatum is "WGS:84", propose amendment dwc:geodeticDatum="epsg:4326"

2). @tucotuco epsg.io does have an API ("The web has also API allowing integration of the search functionality and transformations in third party applications." - https://epsg.io/about) but not sure if this gives us what we want - I have not looked at it. We do also have the list of currently acceptable datums in the appendix to our in prep Georeferencing paper. This doesn't include synonyms. I don't think it is our job to develop and maintain a thesaurus of datums though. Can we loby EPSG to develop what we need (small fry that we are)?

[tucotuco]

epsg.io does have a REST API. See https://github.com/maptiler/epsg.io.

[tucotuco]

1). Perhaps we need to rewrite the example in this one to:
dwc:geodeticDatum is "WGS:84", propose amendment dwc:geodeticDatum="epsg:4326"

Agreed and updated.

2). @tucotuco epsg.io does have an API ("The web has also API allowing integration of the search functionality and transformations in third party applications." - https://epsg.io/about) but not sure if this gives us what we want - I have not looked at it. We do also have the list of currently acceptable datums in the appendix to our in prep Georeferencing paper. This doesn't include synonyms. I don't think it is our job to develop and maintain a thesaurus of datums though. Can we loby EPSG to develop what we need (small fry that we are)?

There is still a problem here with the source authority. epsg.io is not going to be able to do the standardization, they only have the controlled vocabulary, not the thesaurus. GBIF is committed to having the datum thesaurus in the vocabulary services with lookups to epsg codes. I have volunteered to do the mappings.

I updated the Notes as well, from
'[bdq:sourceAuthority = https://epsg.io]. Darwin Core recommends best practice is to use the EPSG code as a controlled vocabulary to provide a CRS, if known. Otherwise use a controlled vocabulary for the name or code of the geodetic datum, if known. Otherwise use a controlled vocabulary for the name or code of the ellipsoid, if known. If none of these is known, use the value "unknown". The amended value should be an EPSG code for a geodetic datum or an ellipsoid. The reference vocabularies of values for geodetic datums and ellipsoids need to be developed and published and include alternative representations of the datum strings, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "WGS84".'.

to

'[bdq:sourceAuthority = GBIF geodeticDatum thesaurus, when available]. Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326".'

[tucotuco]

Georeferencing Best Practices recommends "not recorded". Darwin Core says "unknown". We could consider these as synonyms. GBIF's GeodeticDatum vocabulary does not have either of these values, and should have both.

[chicoreus]

Getting into the semantics, we could change "unknown" (and case variants) to "not recorded", as noted in #59 treating the value in Darwin Core as an incorrect synonym for the value recommended by the georeferencing best practices.

[tucotuco]

I guess it is our choice what recommendation to make. I can't create a new issue for Darwin Core right now, but one should be made to follow Georeferencing Best Practices.

[ArthurChapman]

"unknown" and "not recorded" can mean different things under different circumstances. I prefer what we did in the Georeferencing Best Practices, so I guess, we should create a new issue for Darwin Core. And in that case we should allow "not recorded" as a valid value and treat "unknown" as a synonym. Perhaps we should also inform GBIF so that they can make some changes to their GeodeticDatum vocabulary.

[ymgan]

I created an issue about this in Darwin Core last year tdwg/dwc#437 based on @tucotuco recommendation on #60 (comment)

[chicoreus]

tdwg/dwc#437 is currently tagged as needing a templated proposal.

[Tasilee]

As with #59, if dwc:geodeticDatum contains values such as "unknown" or "not recorded", then it is effectively bdq:Empty so we trigger "INTERNAL_PREREQUISITES_NOT_MET", surely?

[chicoreus]

@Tasilee Absolutely not. The georeferences best pracices guide gives "not recorded" as a valid controlled value to use in this term. We therefore MUST support this value. This is not something we have a choice over. There can be absolutely no disgreement about this. The question is whether to also support "unknown". @ArthurChapman has answered this as "No" by filing change requests on the relevant Darwin Core terms.

[chicoreus]

This doesn't need a change in the expected response:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a value in the bdq:sourceAuthority; otherwise NOT_AMENDED

We can handle "not recorded" by adding to the notes something like:

The value "not recorded" should be treated as an allowed value, and strings with variations in case or whitespace of this value should be conformed to "not recorded".

[chicoreus]

Following on the discussion in #59, propose adding the following text to the end of the note:

For the purposes of this test "not recorded" is not a value in the sourceAuthority and should result in NOT_AMENDED.

[Tasilee]

Thanks @chicoreus : Added

"For the purposes of this test "not recorded" is not a value in the bdq:sourceAuthority and should result in NOT_AMENDED."

to the end of Notes.

[chicoreus]

Following offline discussion and notes on #59, recommend changing expected response from:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a value in the bdq:sourceAuthority; otherwise NOT_AMENDED

To:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as the value "not recorded" or as valid code from the bdq:sourceAuthority (in the form Authority:Number) for a CRS, Datum, or ellipsoid appropriate for a 2D geographic coordinate in degrees; otherwise NOT_AMENDED

Source authority can be:

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index]}

Notes should mention more information on obtaining the EPSG dataset, particularly https://docs.geotools.org/latest/userguide/library/referencing/epsg.html

[ArthurChapman]

Just wondering if we should turn it around a bit

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a CRS, Datum, or ellipsoid appropriate for a 2D geographic coordinate in degrees or as the value "not recorded"; otherwise NOT_AMENDED

[tucotuco]

That seems concise and correct.

[chicoreus]

If we take @tucotuco slightly more expansive scope for datum and ellipsoid:

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, Ellipsoid or a CRS appropriate for a 2D geographic coordinate in degrees, or as the value "not recorded"; otherwise NOT_AMENDED

[tucotuco]

Even better.

[Tasilee]

I agree.

[Tasilee]

Reviewing: #59 uses a more generic Expected Response and GBIF as the Source Authority so, how aligned should this and #59 be?

[Tasilee]

Due to browser caching, I was not viewing the latest comments on #59, so ignore prior post. I will now edit the Expected Responses and Source Authority on #59 and this test...

[Tasilee]

Updated Expected Response from

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a value in the bdq:sourceAuthority; otherwise NOT_AMENDED

to

EXTERNAL_PREREQUISITES_NOT_MET if the bdq:sourceAuthority is not available; INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is bdq:Empty; AMENDED the value of dwc:geodeticDatum if it could be unambiguously interpreted as a valid code from the bdq:sourceAuthority (in the form Authority:Number) for a Datum, Ellipsoid or a CRS appropriate for a 2D geographic coordinate in degrees, or as the value "not recorded"; otherwise NOT_AMENDED

...and updated Specification Last Updated

[Tasilee]

Updated Source Authority from

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index#/Datum]}

to

bdq:sourceAuthority = "EPSG" {[https://epsg.org]} {API for EPSG codes [https://apps.epsg.org/api/swagger/ui/index]}

and Notes from

Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io/) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326". NB. Do NOT change one datum to any other datum no matter how close they are or may appear to be. The same treatment should be given to all datums, which is to use their transformation algorithms to get the equivalent in epsg:4326. For reference, a vocabulary of synonyms for EPSG codes for values of dwc:geodeticDatum can be found at https://registry.gbif.org/vocabulary/GeodeticDatum/concepts. For the purposes of this test "not recorded" is not a value in the bdq:sourceAuthority and should result in NOT_AMENDED.

to

Chapman and Wieczorek (2020) recommend best practice is to use EPSG codes (https://epsg.io/) as a controlled vocabulary. Ideally, amend to the EPSG code for the geographic coordinate reference system (CRS), if known. Otherwise use the EPSG code for the geodetic datum, if known. Otherwise use the EPSG code of the ellipsoid, if known. If none of these is known, use the explicit value "not recorded". The reference vocabularies of values for geodetic datums and ellipsoids needs to be made available should map alternative representations of dwc:geodeticDatum strings to EPSG codes, such as "WGS84", "WGS_84", "WGS:84", "WGS 84" all with standard value "epsg:4326". NB. Do NOT change one datum to any other datum no matter how close they are or may appear to be. The same treatment should be given to all datums, which is to use their transformation algorithms to get the equivalent in epsg:4326. For reference, a vocabulary of synonyms for EPSG codes for values of dwc:geodeticDatum can be found at https://registry.gbif.org/vocabulary/GeodeticDatum/concepts and and for more information on obtaining the EPSG dataset, see https://docs.geotools.org/latest/userguide/library/referencing/epsg.html. For the purposes of this test "not recorded" is not a value in the bdq:sourceAuthority and should result in NOT_AMENDED.

[chicoreus]

Corrected typo in expected response: EEXTERNAL_PREREQUISITES_NOT_MET to EXTERNAL_PREREQUISITES_NOT_MET