iNaturalist API 504 Gateway Time-out

[jhpoelen]

@kueda Hi!

I noticed that the iNaturalist updates aren't coming in as expected. For some reason, 504 Gateway Time-out are reported when requesting urls like:

https://www.inaturalist.org/observation_field_values.json?type=taxon&page=1&per_page=100&quality_grade=research

Please let me know if there's anything I can do to work around this issue.

Thanks!

Please note I am still hearing back from you about lingering #427 ideas. Also, note that changes to ResourceRelations extension in DwC-A have been approved and introduced into DwC. You can find the rigorous paper trail at tdwg/dwc#283 .

[jhpoelen]

@kueda note that due to some caching issue, iNaturalist record dropped out of GloBI for a little. Have implemented a fix, and rebuilding the GloBI index. Apologies for the interruption.

Root cause - I reduced the page size as suggested. And, by doing so, the cached version of iNaturalist could no longer be used (e.g., the request urls changed, so cached content didn't line up).

Temporary fix - revert back to page size of 100.

Long term fix - make a cached version of the reduced page size inaturalist first, then use it to rebuild the GloBI index.

[jhpoelen]

Temporary fix resulted in iNaturalist observations being indexed by GloBI again. (see attached results from https://www.globalbioticinteractions.org/?accordingTo=globi%3Aglobalbioticinteractions%2Finaturalist&interactionType=eats&sourceTaxon=Enhydra%20lutris ) that includes, among many others (>200k observations), this sea otter munching on some tasty marine life https://www.inaturalist.org/observations/77386996 .

@kueda - I am assuming that GloBI's badge next to the "this observation is features on ..." will reappear after your iNaturalist robot will make another round at asking GloBI for which iNaturalist observations were indexed.

[kueda]

Yes, if everything's working correctly, the badges should re-appear some time this week.

[jhpoelen]

Confirmed that the GloBI badge for https://www.inaturalist.org/observations/77386996 is now appearing again . See attached screenshot from today, 1 April 2022.