OSrPRE contribution to OASPA consortium of publishers #RapidReviewC19

[dasaderi]

Updated: June 9th, 2020

This is a Master Issue to discuss how OSrPRE can optimize for this initiative. Individual issues with the agreed upon change should be made for each point.

A group of publishers + us at OSrPRE has signed an open letter that came out today April 27, to basically do 2 things:

• Recruit researchers with relevant expertise to be reviewers for COVID-19 manuscripts submitted to those journals and/or to rapidly review preprints in platfroms such as OSrPRE;

• Share those contacts among the signatories of the letter to optimize peer review. Volunteer reviewers (let's call them "RapidReviewersC19" are asked to submit at google form in which they have to indicate their ORCID iD.

The goal is to increase the S/N ratio of the massive amount of COVID-19 work that is submitted for publications everyday both to journals and as preprints.

To improve how the OSrPRE platform can help in this workflow, I propose the following tasks are discussed and moved to rapid implementation on OSrPRE:

Identify RapidReviewersC19 users on OSrPRE

• Add an attribute for a group to the user profile #100 - 🛵 , A topic/tag dashboard view for folks to understand information within a tag/topic #80
• Because we will have access to the ORCID iD of the RapidReviewersC19 as google sheet output of the form, we can have a way to match the orcid id of the OSrPRE users to that list and automatically tag them as part of that group. I would propose to add that tag to the user profile.
• Update: We will address this with the dashboard view described in A topic/tag dashboard view for folks to understand information within a tag/topic #80.

Aggregated Results

• Improvement on the aggregated rapid reviews dataviz  #79, A topic/tag dashboard view for folks to understand information within a tag/topic #80
• Add a way to only visualize reviews from this group when looking at the aggregated results.
• Update: The initiative is interested in all reviews related to COVID19, so this is not needed anymore. This will be addressed with the Tag Dashboard A topic/tag dashboard view for folks to understand information within a tag/topic #80 .

COVID-19 automatic tagging

• Add functionality to add tags to requests and make them searchable (e.g. COVID-19 tag) #83
• This is something I'd like implemented for everything. That is, an automatic way to search through the metadata (title and abstract) of the preprint to review or for which there is a request to review using covid-19 specific keywords and tag the content automatically, perhaps allowing the user to untag if a mistake is made by the system. I started issue Add functionality to add tags to requests and make them searchable (e.g. COVID-19 tag) #83 about this. Set of keywords we can use are here.
• Update: Automation will likely wait, but this will be addressed at the moment with the change to how tags implement search rather than manual tagging. Add functionality to add tags to requests and make them searchable (e.g. COVID-19 tag) #83

Data sharing statement

• Changes to Rapid Form Questions #115
• Add more specific language to the data question so that readers can differentiate between a preprint who has a statement saying that all data IS available + link vs. those who say data can be made available upon request. These are very different things that in a moment like this one can make a huge difference. We could also add a field that pops out if the user says YES to our existing data question and ask to paste the data statement from the preprint. This change can be specific to the form available to a member of the RapidReviewersC19 group, or something we add to every form.
• Update: We will edit the data question on the form to be clearer, but not change the questions overall until we have a plan for managing versions of the form (see Changes to Rapid Form Questions #115 for more detail).

Prioritization for publication

• Add a way to highlight preprints that should be prioritized for publication #97 - 🛵 , Changes to Rapid Form Questions #115
• Add a sentence at the end of the form that uses a Likert scale for answers to determine if the preprint should be prioritized for publication or to gauge how important the overall results are. For example, we could say "This preprint should be prioritized for rapid publication" with options to say
  o Strongly disagree
  o Disagree
  o Neutral
  o Agree
  o Strongly agree
  Again, this change can be specific to the form available to a member of the RapidReviewersC19 group, or something we add to every form. Issue Add a way to highlight preprints that should be prioritized for publication #97 is related to this.
• Update: We will not change the questions overall until we have a plan for managing versions of the form (see Changes to Rapid Form Questions #115 for more detail).

Reminder of the goal

• Add COVID call to action banner for the OSrPRE site #103, A topic/tag dashboard view for folks to understand information within a tag/topic #80
• Add header to the review form if reviewer is part of the RapidReviewersC19 group stating the goal of the rapid review. E.g., You are reviewing this preprint to support the RapidReviewC19 initiative with the goal of helping readers/journal editors prioritize content" or something like this. We can also link again to the CoC.
• Update: We should add a Call to Action Banner (Add COVID call to action banner for the OSrPRE site #103) on the homepage to bring people to the COVID19 Tag dashboard (A topic/tag dashboard view for folks to understand information within a tag/topic #80)

Reference OASPA Initiative on the homepage

• Add COVID call to action banner for the OSrPRE site #103, A topic/tag dashboard view for folks to understand information within a tag/topic #80
• Have some sort of reference to this initiative on the front page
• Update: We should add a Call to Action Banner (Add COVID call to action banner for the OSrPRE site #103) on the homepage/all pages to bring people to the COVID19 Tag dashboard (A topic/tag dashboard view for folks to understand information within a tag/topic #80)

Tagging the different subject areas

• Tagging different subject areas #132 - 🛵
• I’m not sure about how we would do this, but if you can provide us with a list of useful subject areas or subtopics (e.g., “clinical trial”, or “vaccine”, “modeling”), we could offer a drop down menu to the reviewers so that the review and associated preprint can be tagged for easier search by editors. This ontology was suggested by FAIRdata.
• Update: This was suggested by the OASPA publishers and can be parked until we have funding to cover these types of requests.

[dasaderi]

In today's call with publishers from this group, I was asked if we can give rapid reviews DOIs to integrate metadata in CrossRef. Right now we don't give DOIs to the rapid reviews, but we do via Zenodo, on PREreview. What we do have on OSrPRE is a "permalink" that associates the preprint to the reviews.

Thoughts @majohansson @georgiamoon? I think it's something we should think about, but not a priority.

[dasaderi]

@murkatr @georgiamoon I made some slides summarizing some of current OSrPRE features and proposed ideas for this issue to share with preprint workflow group from the open letter group https://docs.google.com/presentation/d/1_g1qr-dwR3bdSP9PJD_jBj9BmLf5ScEsGCtQmn4ZVbI/edit#slide=id.g76f2b40aae_1_80. It can be improved, but I think it can be useful when you are trying to communicate ideas to them in a meeting.

Also made a survey for j editors: https://forms.gle/VbayQ36LTv6mDAsD8.

[simsecdeploybot-ott]

@dasaderi this is super helpful. I'll review! I'm subscribed to the form replies as well, so seeing those as they come in.

[murkatr]

An update on the above unchecked-tasks:

Identify RapidReviewersC19 users on OSrPRE

• Add an attribute for a group to the user profile #100 - 🛵 , A topic/tag dashboard view for folks to understand information within a tag/topic #80
  Because we will have access to the ORCID iD of the RapidReviewersC19 as google sheet output of the form, we can have a way to match the orcid id of the OSrPRE users to that list and automatically tag them as part of that group. I would propose to add that tag to the user profile.
  Update: A topic/tag dashboard view for folks to understand information within a tag/topic #80 is complete. Add an attribute for a group to the user profile #100 is now issue #38 in the New Merge Platform project.

Aggregated Results

• Improvement on the aggregated rapid reviews dataviz  #79, A topic/tag dashboard view for folks to understand information within a tag/topic #80
  Add a way to only visualize reviews from this group when looking at the aggregated results.
  Update: A topic/tag dashboard view for folks to understand information within a tag/topic #80 is complete. @murkatr will discuss Improvement on the aggregated rapid reviews dataviz  #79 with @dasaderi.

Prioritization for publication

• Add a way to highlight preprints that should be prioritized for publication #97 - 🛵 , Changes to Rapid Form Questions #115
  Add a sentence at the end of the form that uses a Likert scale for answers to determine if the preprint should be prioritized for publication or to gauge how important the overall results are. For example, we could say "This preprint should be prioritized for rapid publication" with options to say
  o Strongly disagree
  o Disagree
  o Neutral
  o Agree
  o Strongly agree
  Again, this change can be specific to the form available to a member of the RapidReviewersC19 group, or something we add to every form. Issue Add a way to highlight preprints that should be prioritized for publication #97 is related to this.
  Update: @murkatr will discuss Add a way to highlight preprints that should be prioritized for publication #97 and Changes to Rapid Form Questions #115 with @dasaderi.

Tagging the different subject areas

• Tagging different subject areas #132 - 🛵
  I’m not sure about how we would do this, but if you can provide us with a list of useful subject areas or subtopics (e.g., “clinical trial”, or “vaccine”, “modeling”), we could offer a drop down menu to the reviewers so that the review and associated preprint can be tagged for easier search by editors. This ontology was suggested by FAIRdata.
  Update: @murkatr will discuss Tagging different subject areas #132 with @dasaderi.