Resources, APIs, etc

[blahah]

Thread to collect resources

[blahah]

via @hubgit: the pubmed retractions API report https://rpubs.com/neilfws/65778

[blahah]

crossref have a repo which shows an undocumented part of the API for retrieving retractions: https://github.com/CrossRef/retractions

right now it only has results from one publisher

[blahah]

I experimented with the crossref API and found that other publishers publish retraction notices as different kinds of updates, so that's one place to look - also some don't make them updates at all, so retraction AND notice in the title should be a start

[blahah]

Turns out to be pretty simple to get retractions from the PubMed API, where they are helpfully marked with pubtype = ['Journal Article', 'Retracted Publication']...

So we first use bionode-ncbi to search PubMed for the string "Retracted Publication", then filter the results using jq to ensure that the string is actually in the pubtype field:

bionode-ncbi search pubmed "\"Retracted Publication\"" > pubmed_retracted.json
cat pubmed_retracted.json | jq -c 'select(.pubtype[] | inside("Retracted Publication"))'

As of today, this gives 4,997 retractions:

cat pubmed_retracted.json | jq -c 'select(.pubtype[] | inside("Retracted Publication"))' | wc -l
4997

Of those, 3,728 have DOIs:

cat pubmed_retracted.json | \
  jq -c 'select(.pubtype[] | inside("Retracted Publication")) | .articleids | .[] | select(.idtype == "doi") | .value' | \
  wc -l
3728

[blahah]

I made a command-line tool to achieve similar to the above, but for crossref: https://github.com/fathomlabs/crossref-cli

I'm now working my way through the different update types to see which ones we should use.

Here's the full list:

addendum
article
book-review
clarification
comment
communications
contributed-paper
corrected-article
correction
Correction
correspondence
corrigendum
Corrigendum
err
erratum
Erratum
expression_of_concern
expression-of-concern
interesting_update
invited-article
invited-paper
new_edition
new_version
note-discuss
Oopsies
point-counterpoint
print
publisher-note
removal
retraction
Retraction
retration
sample_update
tg-report
this_is_some_update_23
unknown
withdrawal

• withdrawal in the context of some papers means that the information in the article has been superseded by more recent evidence, e.g. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD000170.pub2/full
• retraction (and the much less used Retraction) means what it says, but only a couple of publishers use this

[blahah]

Update entries in CrossRef have both a type and a label field - here's the set of labels:

Addendum
article
book-review
Clarification
comment
communications
contributed-paper
corrected-article
Correction
correspondence
Corrigendum
Erratum
expression-of-concern
Expression of concern
interesting_update
invited-article
invited-paper
New edition
New version
note-discuss
Oopsies
point-counterpoint
print
publisher-note
Removal
Retraction
retration
sample_update
tg-report
this_is_some_update_23
unknown
Withdrawal

[blahah]

yes, retration is a label 🤔

[blahah]

OK, these are the types we probably want to include:

corrected-article
correction
Correction
correspondence
corrigendum
Corrigendum
err
erratum
Erratum
expression_of_concern
expression-of-concern
Removal
Retraction
retration
Withdrawal

[blahah]

the above is now implemented in https://github.com/fathomlabs/crossref-retractions