TG2-AMENDMENT_EVENT_FROM_EVENTDATE

[iDigBioBot]

TestField	Value
GUID	710fe118-17e1-440f-b428-88ba3f547d6d
Label	AMENDMENT_EVENT_FROM_EVENTDATE
Description	Proposes an amendment to values in any of dwc:year, dwc:month, dwc:day, dwc:startDayOfYear or dwc:endDayOfYear from the content of dwc:eventDate.
TestType	Amendment
Darwin Core Class	dwc:Event
Information Elements ActedUpon	dwc:year
	dwc:month
	dwc:day
	dwc:startDayOfYear
	dwc:endDayOfYear
Information Elements Consulted	dwc:eventDate
Expected Response	INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is bdq:Empty or contains an invalid value according to ISO 8601; FILLED_IN if any of (1) dwc:day from dwc:eventDate if dwc:day is bdq:Empty and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is bdq:Empty and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is bdq:Empty and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are bdq:Empty and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.
Data Quality Dimension	Completeness
Term-Actions	EVENT_FROM_EVENTDATE
Parameter(s)
Source Authority
Specification Last Updated	2024-09-16
Examples	[dwc:eventDate="2023-01-26", dwc:year="2023", dwc:month="", dwc:day="", dwc:startDayOfYear="", dwc:endDayOfYear="": Response.status=FILLED_IN, Response.result=dwc:month="1", dwc:day="26", dwc:startDayOfYear="26", dwc:endDayOfYear="26", Response.comment="dwc:month, dwc:day, dwc:startDayOfyear and dwc:endDayOfYear filled in from dwc:eventDate"]
	[dwc:eventDate="2023", dwc:year="2023", dwc:month="", dwc:day="", dwc:startDayOfYear="", dwc:endDayOfYear="": Response.status=NOT_AMENDED, Response.result=, Response.comment="No amendments possible"]
Source	VertNet
References	ISO (2019) ISO 8601-1:2019(en) Date and time — Representations for information interchange — Part 1: Basic rules. https://www.iso.org/obp/ui/ Wikipedia (2020) ISO 8601. https://en.wikipedia.org/wiki/ISO_8601 Library of Congress (2019) Extended Date/Time Format (EDTF). https://www.loc.gov/standards/datetime/
Example Implementations (Mechanisms)	Kurator:event_date_qc
Link to Specification Source Code	FilteredPush event_date_qc DwCEventDQ.amendmentEventFromEventdate() unit test in DwcEventDQTest
Notes	Only fields that are empty will be have changes proposed, and only if dwc:eventDate has a valid ISO 8601-1 date. The dwc:eventDate is the canonical form of the event date (it is the first trusted form). If event date does not contain a range, dwc:startDayOfYear = dwc:endDayOfYear. Time (as compared to date) is not deemed a CORE component. Note, see sequencing tests section of standards document, run this amendment after any other amendment which may affect dwc:eventDate

[iDigBioBot]

Comment by Paul Morris (@chicoreus) migrated from spreadsheet:
One way of simplifying the core test suite would be to identify a small set of primary fields in cases where Darwin Core allows for multiple representations of the data (Event fields are the clearest example of this), and only propose amendments that work to fill in the primary fields from secondary fields (eventDate from day, month, year, verbatimEventDate, startDayOfYear, endDayOfYear, eventTime), and not include in the core suite ammendments that fill in secondary fields from the primary fields.

[iDigBioBot]

Comment by Arthur Chapman (@ArthurChapman) migrated from spreadsheet:
I agree with Paul ghere

[iDigBioBot]

Comment by Paula Zermoglio (@pzermoglio) migrated from spreadsheet:
Agree too. However, you may piss some people off, eg those that prefer using YMD instead of eventDate?

[ArthurChapman]

@JohnW wanted to be able to back populate - at least for year - as many people want to extract just the year from the data.

[chicoreus]

The danger with backpopulating is that eventDate is capable of handling richer information than the atomic fields (date ranges and date ranges which span more than one year). A consumer who wants to simply obtain the year from dwc:year does so at their own peril if dwc:eventDate contains a date range which spans more than one year. That, however, is a let the consumer beware kind of issue. We shouldn't advocate back populating because people may want to use the data, as it potentially is unfit for their use, but I'm entirely in favor of back populating in order to make data sets consistent in their presentation - filling in all fields that can be filled in.

[chicoreus]

I propose changing the description to: One or more empty component terms of the dwc:Event class (dwc:year, dwc:month, dwc:day, dwc:startDayOfYear, dwc:endDayOfYear) have been filled in from a valid value in the term dwc:eventDate.

The concept reflected in in the note " and only if dwc:eventDate has a valid ISO 8601:2004(E) date" should be reflected in the description. It is an important point for implementers.

[chicoreus]

There is an inter-amendment workflow dependency to note here - this amendment should run after all other amendments that may affect the value of eventDate. (i.e. fill in the event date from the verbatimEventDate, then fill in year/month/day/startDayOfYear/endDayOfYear from the interpreted eventDate value, etc.). I've added a note to this effect to the prerequisites.

[chicoreus]

We should note that time is not included here as it is not considered core. In working with an implementation of this, I've found extracting time from eventDate to be fraught with all sorts of concerns (which aren't going to be core concerns), including handling time zone and handling times on eventDates which involve ranges. Non-trivial to specify appropriate behaviors, and those aren't core. We are much safer not having this particular test propose to fill in eventTime.

[chicoreus]

We do need to specify if endDayOfYear is expected to be filled in if the eventDate represents a single day.

[ArthurChapman]

@chicoreus - wrt to workflow - I have added "After #33, #49, #86, #93, #132" for test #52 - see the circulated workflow document

[ArthurChapman]

@chicoreus - fully agree wrt note on time.

[Tasilee]

I have reviewed the parameters and notes (all good) and believe we have a useable outcome.

[chicoreus]

@Tasilee I like that phrasing.

[ArthurChapman]

I am happy with that @Tasilee but note minor correction (1) dwc.day from dwc:eventDate.... (was 1) dwc:day from day from dwc:eventDate...

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN 1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer, 2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month, 3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, 4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

[Tasilee]

I have edited the Expected response (and Specification Last Updated) accordingly. NEEDS WORK??

[ArthurChapman]

I think NEEDS WORK can be turned off.

Further to a post elsewhere; we have in different tests:

• (1), (2), (3)
• 1), 2), 3)
• a), b), c)
• others?

[Tasilee]

See #43 (comment)

[chicoreus]

Updated phrasing of the note, Amendments do not amend data, they propose changes (in Response.result, with a Response.status of FILLED_IN (only FILLED_IN in this case) or AMENDED). We need to be careful that we don't use these constants as verbs.

[chicoreus]

Added missing "or finer" from clause 2 for month. Now consistent with day and year clauses, otherwise implemetors would not fill in month if the event date had a precision of one day.

[chicoreus]

See discussion in #67 and issue #204, likely needs a little bit of work to be consistent with the related tests.

[chicoreus]

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

[Tasilee]

Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted". Also changed "Field" to "TestField" and "Output Type" to "TestType".

[chicoreus]

Current evaluation of validation data dataid 320 (from @Tasilee spreadsheet v68) suggests the numbered clauses in FILLED_IN need to be separated by "or":

320 #52 Fail got FILLED_IN expected NOT_AMENDED Provided value for dwc:eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z] represents a range of more than one year.|Provided dwc:eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z] spans more than one year, can't fill in dwc:year.|Added startDayOfYear [60] from start day of eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z].|Added endDayOfYear [132] from end day of eventDate [2007-03-01T13:00:00Z/2008-05-11T15:30:00Z].

Interpretation in implementation (which is in line with the expectations of FILLED_IN) is that if any of the elements are FILLED_IN then that is used as the Response.status, only if no elements are filled in, then NOT_AMENDED is used.

[Tasilee]

That seems reasonable. I've edited DataID 320 to FILLED_IN

Also changed Expected Response from

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; (FILLED_IN dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day. FILLED_IN dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month. FILLED_IN dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year. FILLED_IN dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better) otherwise NOT_AMENDED.

We can't use 'OR' because that doesn't allow for multiple fills. Other option is to qualify 'NOT_AMENDED' to something like

If all of dwc:day, dwc:month, dwc:year, dwc:startDayOfYer and dwc:endDayOfYear were not FILLED_IN, then NOT_AMENDED.

[chicoreus]

How about:

FILLED_IN if any of (1)...

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN if any of (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

[Tasilee]

Looks good to me @chicoreus

Changed

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or contains an invalid value according to ISO 8601-1; FILLED_IN if any of (1) dwc:day from dwc:eventDate if dwc:day is EMPTY and dwc:eventDate has a precision of a day or finer and is within a single day, (2) dwc:month from dwc:eventDate if dwc:month is EMPTY and dwc:eventDate has a precision of a single month or finer and is within a single month, (3) dwc:year from dwc:eventDate if dwc:year is EMPTY and dwc:eventDate has a precision of a single year or finer and is within a single year, (4) dwc:startDayOfYear and dwc:endDayOfYear if they are EMPTY and dwc:eventDate has a precision of a day or better; otherwise NOT_AMENDED.

and updated 'Specification Last Updated'

[Tasilee]

Changed reference in Expected Response from ISO 8601-1 to ISO 8601