This working group (WG) aims to gather responsible and efficient quality indicators to facilitate initial quality screening and assessment of research outputs in situations where in-depth qualitative assessment via peer-review is not feasible. The challenge is to preserve the advantages of indicators – objectivity and efficiency – while improving their validity.
This WG aligns with the CoARA vision by complementing CoARA’s commitment to qualitative assessment with transparent and valid indicators:
✅ We do not always use indicators – but when we do, they need to be valid, reproducible, efficient and non-proprietary.
The indicators are supposed to pertain to a broad range of research outputs (including publications, data sets, research software), various assessment scenarios (hiring, promotion, funding, monitoring), and various units of assessment (including research outputs, researchers, institutions).
The WG has three objectives, tackled in three stages (Oct. 23 - Sept. 25):
Stage 1: Assessing the status quo: Which indicators are currently employed?
- We investigate this across varied disciplines and cultural contexts, specifically in the evaluation of researchers, awards, institutional assessments, and the progression of scientific advancement (like tenure, PhD, habilitation, professor titles, etc.).
- Starting with CoARA members, institutions are asked to provide information on their current usage of indicators via a structured survey.
- This set of already employed indicators gets augmented by new indicators that cover underrepresented aspects, such as open science aspects of research.
✅ The survey is available here (until 2025-06-08): https://surveys.osc.lmu.de/RMI The survey questions are published under a CC-BY4.0 license at Zenodo.
Stage 2: Critical evaluation of the indicators and recommendations. Based on the survey results, we develop guiding questions and recommendations (2a) when to use indicators (and when not), and (2b) how to integrate their usage with qualitative modes of assessment. This could be a decision path diagram, and will not be a “one size fits all” recommendation.
Stage 3: Strategic plan for dissemination of recommendations. Share findings, strategies, and recommendations with a wider audience, emphasizing their applicability across disciplines and cultures.
High involvement: You are welcome to join as a WG member institution. For this, you should be a representative of an official CoARA member, and be ready to attend and contribute to the WG meetings (~quarterly). In the shortest time frame, this means that you can contribute to the development of the Stage 1 survey; in the longer run, you will be able to work with us more directly on the recommendations (though, of course, we’re always open to more impromptu input from any member of the broader community).
Medium involvement: Your institution can provide answers in the piloting phase of our Stage 1 survey and give feedback during the development of the recommendations.
Low involvement: Simply fill out the final survey (live until 2025-06-08).
The following CoARA members are active members of this working group (no specific ordering):
- German Psychological Society (Germany)
- Leibniz Association: IWM Institut für Wissensmedien (Germany)
- Leibniz Association: Leibniz Institute for Psychology, ZPID (Germany)
- BIH / Charité Berlin (Germany)
- Berlin University Alliance (Germany)
- Alexander von Humboldt Foundation
- Swiss Reproducibility Network (Switzerland)
- EPFL - Swiss Federal Institute of Technology in Lausanne (Switzerland)
- Italian Reproducibility Network (Italy)
- Università di Pisa (Italy)
- Roma Tre University, Department of Science (Italy)
- Italian Institute of Technology (Italy)
- IMT School for Advanced Studies Lucca (Italy)
- University of Ferrara (Italy)
- Università degli Studi di Milano (Italy)
- University of Messina (Italy)
- Aix-Marseille University (France)
- Université Paris Cité (France)
- Université Côte d'Azur (France)
- Université Lumière Lyon 2 (France)
- MNHN - Museum National d'Histoire Naturelle (France)
- University of Luxembourg (Luxembourg)
- Swedish Reproducibility Network (Sweden)
- Karlstad University (Sweden)
- Tbilisi Medical Academy (Georgia)
- Sociedad Científica de Bibliometría y Cienciometria (Peru)
- CERN - European Organization for Nuclear Research (Europe)
- UAS4EUROPE - Universities of Applied Sciences for Europe (Europe)
- OpenAIRE AMKE (Europe)
- GCPA - Good Clinical Practice Alliance (Europe)
- International Consortium of Research Staff Associations (Global)
- Molde University College (Norway)
- Poznan University of Economics and Business (Poland)
- University of Cyprus (Cyprus)
- Vrije Universiteit Brussel (Belgium)
- Hungarian Academy of Sciences (Hungary)
- Masaryk University (Czechia)
- Finnish Learned Societies (Finland)
- Häme University of Applied Sciences (Finland)
- BOKU University (Austria)
- University of Monterrey (Mexico)
- CRG - Centre for Genomic Regulation (Spain)
- AQuAs - Agency for Health Quality and Assessment of Catalonia (Spain)
- Young Academy of Slovenia (Slovenia)
- Corvinus University of Budapest (Hungary)
If you want to join the WG, please get in contact with us!
The steering committee consists of four persons:
- Katarzyna Nawrot (Poznan University of Economics and Business, Poland)
- Felix Schönbrodt (German Psychological Society)
- Rachel Heyard (Swiss Reproducibility Network)
- Davide Crepaldi (Italian Reproducibility Network)
Primary contacts towards the CoARA Steering Committee are Katarzyna Nawrot and Felix Schönbrodt.
In general WG meetings, each member organization sends one representative.
- "Members" of the WG are CoARA members (i.e., institutions), represented by a concrete person.
- "Experts" of the WG are individual persons who are experts in the field, but are not officially representing a CoARA member. The WG can (proactively) contact them for consultation.
- "Friends" of the WG are generally interested persons who want to simply stay informed.
You can directly subscribe to our public mailing list. Our public mailing list is specifically targeted at the "Friends" of our WG. If you want to unsubscribe please inform a member of the steering committee.
To become an official member organization of our WG fill in the application form. If you are accepted (which you most likely will - if you represent a COARA member), we will include you in all communication and meetings.
To become an expert of our WG fill in the registration form. We might contact you proactively for consultation or invite you to specific meetings of the WG.
For other direct inquiries, you can contact any member of the steering committee:
- Davide Crepaldi: dcrepaldi@sissa.it
- Rachel Heyard: rachel.heyard@uzh.ch
- Katarzyna Nawrot: katarzyna.nawrot@ue.poznan.pl
- Felix Schönbrodt: felix.schoenbrodt@psy.lmu.de