More lenient interpretation of individualCount NULL

[timrobertson100]

It is relatively common that numerical values that should be NULL are presented as 0.

A consequence is that GBIF will infer absence when individualCount is stated as 0 but not intended. When an explicit statement of presence is given in occurrenceStatus we should be more lenient e.g.:

If (individualCount == 0 AND occurrenceStatus == ‘present’) then individualCount = NULL

This would be implemented somewhere around here

[timrobertson100]

@MattBlissett can you please provide your opinion or implement if you agree?

[tucotuco]

I agree with this interpretation. Whether you put it in practice or no, I think we should also document the best practice in the term commentaries (see ). What do you think about documenting what GBIF does in the Darwin Core Q&A? Is that too much a moving target to keep up with? If so, could we point to documentation done by GBIF within DwC Q&A?

…

On Wed, Mar 20, 2019 at 9:11 AM Tim Robertson ***@***.***> wrote: @MattBlissett <https://github.com/MattBlissett> can you please provide your opinion or implement if you agree? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#94 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAcP640-IU2RpvkpOGhSKzajGxM6Rkc1ks5vYiV7gaJpZM4b_Jl5> .

[MattBlissett]

This affects about 20,000,000 records.

We don't yet interpret occurrenceStatus. That can wait for pipelines, so I will handle the existing cases: present, Present and Presente and individualCount being 0.

What did you mean by "GBIF will infer absence"?

John: I think we will work on improving the documentation around GBIF interpretation later this year, at the moment too many questions require me to read the source code to see what we do.

[MattBlissett]

This was implemented in pipelines, resulting in an issue: https://api.gbif.org/v1/occurrence/search?issue=INDIVIDUAL_COUNT_CONFLICTS_WITH_OCCURRENCE_STATUS

The interpretation is according to this table: gbif/pipelines#268 (comment)