TG2-MEASURE_EVENTDATE_DURATIONINSECONDS

[godfoder]

TestField	Value
GUID	56b6c695-adf1-418e-95d2-da04cad7be53
Label	MEASURE_EVENTDATE_DURATIONINSECONDS
Description	What is the duration of dwc:eventDate in seconds?
TestType	Measure
Darwin Core Class	Event
Information Elements ActedUpon	dwc:eventDate
Information Elements Consulted
Expected Response	INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or if the value of dwc:eventDate is not a valid ISO 8601-1 date; otherwise RUN_HAS_RESULT with the result being the duration (sensu ISO 8601-1) expressed in the dwc:eventDate, in seconds.
Data Quality Dimension	Resolution
Term-Actions	EVENTDATE_DURATIONINSECONDS
Parameter(s)
Source Authority
Specification Last Updated	2023-09-18
Examples	[dwc:eventDate="1880-05-08/10": Response.status=RUN_HAS_RESULT, Response.result="259200", Response.comment="dwc:eventDate duration is 3 days = 259,200 seconds"]
	[dwc:eventDate="95": Response.status=INTERNAL_PREREQUISITES_NOT_MET, Response.result=NOT_REPORTED, Response.comment="dwc:eventDate does not contain a valid ISO 8601-1:2019 date"]
Source	Alex Thompson
References	ISO 8601-1: Latest version (currently 2019(en)) Date and time — Representations for information interchange — Part 1: Basic rules (https://www.iso.org/obp/ui/#iso:std:iso:8601:-1:ed-1:v1:en) Wikipedia (2020). ISO 8601 (https://en.wikipedia.org/wiki/ISO_8601)
Example Implementations (Mechanisms)	FilteredPush/Kurator:event_date_qc 10.5281/zenodo.596795.
Link to Specification Source Code	event_date_qc v3.0.0 DwCEventDQ.measureEventdateDurationinseconds()
Notes	The duration of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days (but not leap seconds) in durations that encompass them. Consumers should treat assertions about event date duration as approximations, see: https://xkcd.com/2867/

[chicoreus]

Isn't this a duplicate of #124?

Information elements should only include eventDate.

Remove the " expected to run this test after the amendments to populate dwc:eventDate." All measures are expected to be run in a pre-amendment phase and in a post-amendment phase.

[ArthurChapman]

Should this one be better named MEASURE_EVENTDATE_PRECISIONINSECONDS?

[Tasilee]

Yes indeed. Changed in label and table, where Term-Actions was also incorrect.

[chicoreus]

This test has no path to a response state of NOT_RUN, so the last clause shouldn't be included in the specification, similarly, "REPORTED" is confusing, this means Result.State=RUN_HAS_RESULT Result.Value={duration in seconds, long integer}.

Suggest changing: "INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; REPORT on the length of the period expressed in the dwc:eventDate in seconds; otherwise NOT_REPORTED" to:

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; otherwise RUN_HAS_RESULT with the result value being the length of the period expressed in the dwc:eventDate in seconds"

[ArthurChapman]

@chicoreus - the suggested change looks OK to me.

[chicoreus]

@ArthurChapman great. I've updated accordingly.

[Tasilee]

I've also updated the TESTs dump worksheet accordingly and I think I have already done the test data examples correctly.

[chicoreus]

We probably need to add guidance about leap seconds to the notes. For the purposes of CORE, I would suggest that the guidance be to ignore leap seconds when calculating duration. Standard time libraries that implementers would use tend to ignore leap seconds (e.g. posix seconds from the epoch (affecting python Time), joda time (which excludes leap seconds), and java Time (which treats all days as 86400 seconds), etc.). Including leap seconds is likely to confuse end users, for example by causing event dates known to a day that includes a leap second to be excluded from use by a filter on <=86400 seconds). CORE uses, from the TG3 results and discussed here where we discussed including dwc:eventTime and decided not to include it in core, don't include time, so feels safer for core uses to be explicit in the notes about not including leap seconds.

[chicoreus]

I propose changing the notes from:

The length of a day is 86400 seconds. Leap days and leap seconds affect the duration of some months and some years.

to:

The length of a day is 86400 seconds. Leap days and leap seconds affect the duration of some months and some years, implementations should not include leap seconds, and treat all days as 86400 seconds, but should include leap days.

[ArthurChapman]

I am happy with that.

[Tasilee]

I don't understand it. Could it be reworded for simpletons like me? It needs to be self contained or reference definitions.

[tucotuco]

Would this suffice?

The length of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days.

[Tasilee]

Thanks @tucotuco. Better, but we have two elephants in the room!

1. We have no dwc:coordinatePrecision test so why this one? I do wonder.
2. Are we REALLY talking about a RANGE or a PRECISION? The expected response reads

..."RUN_HAS_RESULT with the result value being the length of the period expressed in the dwc:eventDate in seconds"

This implies a RANGE, yet the example

"dwc:eventDate="1880-05-08" has a precision of 86400 seconds. dwc:eventDate="1880-05-08/10" has a precision of 86400 seconds."

IS a PRECISION!

If this test is about PRECISION, and we think it is CORE, then surely the Expected Response should be more like

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 date; otherwise RUN_HAS_RESULT with the value in seconds of the coarsest temporal unit of dwc:eventDate.

For example, dwc:eventDate="1949-01-15T12:34/1949-01-20" would have a result of 86400 seconds.

Leap anything is irrelevant?

[ArthurChapman]

Good point @Tasilee

I'd like to question why we have this test as a Core Test. It is unlike any other test we have. It is a measure unlike the other measures. I can't see how it would be valuable in testing the quality of a dataset, record etc. I am having trouble envisioning where a user would want to run this and what value they would get out of it wrt to data quality.

[tucotuco]

Very astute @Tasilee . That's thinking inside the box. ;-) I agree that precision is not particularly useful, but the uncertainty is, and that is what this measure is really about ("the result value being the length of the period expressed in the dwc:eventDate in seconds"). It would let people filter out records that were less specific than a day (> 86400), which is otherwise not trivial to do.

To fix this test, I would rename it to something like TG2-MEASTURE_EVENTDATE_DURATIONINSECONDS and amend the incorrect example,

dwc:eventDate="1880-05-08/10" has a precision of 86400 seconds.

to

dwc:eventDate="1880-05-08/10" has a precision of 259200 seconds.

[cboelling]

I think what you are thinking of (requiring) @cboelling is not a Data Quality test but a new term (Darwin Core?) for the Duration. That would have to be taken up in a different forum.

I agree, @ArthurChapman, but I think that some of the difficulties exposed by the discussion here, trying to specify this test, stem from what I think is an ambiguity in how dwc:EventDate is defined and also in how it is actually being used, as pointed out above.

Looking at this from an adopter's perspective, I understand that what the test reports back is the length of a time interval, a duration, expressed in seconds in this instance, from what it finds as value in a dwc:EventDate token. The significance of that duration with regard to the actual or assumed temporal extension of the event or the actual or assumed duration of observing the event may be quite different in different cases. Therefore, based on the current definitions for the test and for dwc:EventDate, I find the term "DURATION" more plausible for describing the test than the alternatives that have been suggested.

[Tasilee]

This one is yet to be finalized, at least here. Two votes for "MEASURE_EVENTDATE_PERIODINSECONDS"?

[ArthurChapman]

Last email from @chicoreus suggested MEASURE_EVENTDATE_DURATIONINSECONDS

"The question is what to call this....

An option might be to drop the word, and call it MEASURE_EVENTDATE_INSECONDS....

Or use the ISO consistent concept MEASURE_EVENTDATE_DURATIONINSECONDS"

as the duration of the Event Date in seconds is what we are measuring - I vote for that (MEASURE_EVENTDATE_DURATIONINSECONDS)

[ArthurChapman]

Note you also have another vote for DURATIONINSECONDS from @cboelling above

[Tasilee]

If we want to use "duration" then the Expected Response should be updated accordingly.

"EVENTDATE_INSECONDS" is ambiguous.

[ArthurChapman]

I'd leave wording of Expected Response to @chicoreus and @tucotuco - but it appears from the recent runs carried out by @chicoreus - we are measuring the duration. i.e. dwc:eventDate="2007-03-01T13:00:00Z/2008-05-11T15:30:00Z" = 37765800 which is different to what we were measuring before where we had it = "0.01"

[tucotuco]

In an effort to try to wrap this one up, I recommend

Label: TG2_MEASURE_EVENTDATE_DURATIONINSECONDS
Expected Response: INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 value; otherwise RUN_HAS_RESULT with the result value being the length of the duration (sensu ISO 8601-1:2019) expressed in the dwc:eventDate, in seconds.
Term Actions: EVENTDATE_DURATIONINSECONDS
Example: dwc:eventDate="1880-05-08" has a duration of 86400 seconds. dwc:eventDate="1880-05-08/10" has a duration of 259200 seconds.
Notes: The duration of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days in durations that encompass them.

[ArthurChapman]

Looks good to me - do you want to include leap seconds in the notes

Notes: The duration of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days (but not leap seconds) in durations that encompass them.

[tucotuco]

I have no attachment to any mention of leap seconds. Not even the IERS is convinced that it was ever a good idea or that the practice should continue.

[chicoreus]

On Tue, 15 Mar 2022 17:59:19 -0700 John Wieczorek ***@***.***> wrote: I have no attachment to any mention of leap seconds. Not even the IERS is convinced that it was ever a good idea or that the practice should continue.

To make it clear for implementors, I suggest we explicitly state in the notes that leap seconds should be excluded. @ArthurChapman's language works fine: **Notes**: The duration of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days (but not leap seconds) in durations that encompass them.

[cboelling]

In an effort to try to wrap this one up, I recommend

Label: TG2_MEASURE_EVENTDATE_DURATIONINSECONDS
Expected Response: INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 value; otherwise RUN_HAS_RESULT with the result value being the length of the duration (sensu ISO 8601-1:2019) expressed in the dwc:eventDate, in seconds.
Term Actions: EVENTDATE_DURATIONINSECONDS
Example: dwc:eventDate="1880-05-08" has a duration of 86400 seconds. dwc:eventDate="1880-05-08/10" has a duration of 259200 seconds.
Notes: The duration of a day is 86400 seconds. Implementations should treat all days as 86400 seconds, but should include leap days in durations that encompass them.

To me this looks good, but I propose to change the wording in EXPECTED RESPONSE to: otherwise RUN_HAS_RESULT with the result value being the length of the duration of the time interval (sensu ISO 8601-1:2019) expressed in the dwc:eventDate, in seconds.

In ISO 8601, durations don't have length, they are length (of time).

[Tasilee]

I have updated the name of the measure to DURATION and edited the relevant specifications and example.

[Tasilee]

Should we have an entry

| Source Authority | bdq:sourceAuthority is "ISO 8601-1:2019" [https://www.iso.org/obp/ui/] |

?

[tucotuco]

I wouldn't think so if we have established that bdq:sourceAuthority should be used for vocabularies.

[Tasilee]

...Just testing :)....

[ArthurChapman]

I agree with @tucotuco Keep the bdq:sourceAuthority for vocabularies where we are looking up a value - use just ISO... (e.g. in dates) where we are using it as a format (not a vocabulary as we are not looking up values).

[Tasilee]

I've edited the Expected Response according to @tucotuco suggestion:

From

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or does not contain a valid ISO 8601-1:2019 value; otherwise RUN_HAS_RESULT with the result being the duration (sensu ISO 8601-1:2019) expressed in the dwc:eventDate, in seconds.

to

INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is EMPTY or if the value of dwc:eventDate is not a valid ISO 8601-1 date; otherwise RUN_HAS_RESULT with the result being the duration (sensu ISO 8601-1) expressed in the dwc:eventDate, in seconds.

and updated the References

[Tasilee]

I have updated the ISO Reference link

[chicoreus]

Added reference to https://xkcd.com/2867/ to note. This can probably also be added as a citation.