Workshop 4: Certification and Incentives WG #304
Replies: 17 comments 15 replies
-
|
Certification can apply to models but also to modelers. In the future, education programs may be developed to modelers may get certification. |
Beta Was this translation helpful? Give feedback.
-
|
OMF could provide training as a service. Maybe under a price. |
Beta Was this translation helpful? Give feedback.
-
|
Education and certification can have a connection |
Beta Was this translation helpful? Give feedback.
-
|
The purpose of the certification is to provide evidence that the model complies with the standards. It does not necessarily imply anything about the soundness of the science behind. |
Beta Was this translation helpful? Give feedback.
-
|
Who does the certification? |
Beta Was this translation helpful? Give feedback.
-
|
Certification may refer to a piece of a large chain of reused code or to a model component. This complicates the review process. |
Beta Was this translation helpful? Give feedback.
-
|
The quality of the submission checklist has an impact on the quality of the submissions and how complicated it can be to review them. |
Beta Was this translation helpful? Give feedback.
-
|
Could also ask that people who are submitting models for certification have to submit a few unit tests with it - this would encourage good practice and ease the work for the reviewer |
Beta Was this translation helpful? Give feedback.
-
|
Maybe at least a user guide and related test data. |
Beta Was this translation helpful? Give feedback.
-
|
An alternative model: nanohub.org |
Beta Was this translation helpful? Give feedback.
-
|
Community use metrics are monitored to identify models, organizations or universities are invited to help curate their content. |
Beta Was this translation helpful? Give feedback.
-
|
we need to separate two things, I felt. publishing and practice. if only publishing, it focuses on the science part of the model. but if comes into practice, standardization is most important. so only if model developer willing to power the model to real tools, our standardization will effectively works. |
Beta Was this translation helpful? Give feedback.
-
|
Crowd-sourced feedback. Students pursuing certification training may take this as assignment. |
Beta Was this translation helpful? Give feedback.
-
|
Can there be different levels of certification? |
Beta Was this translation helpful? Give feedback.
-
|
sure |
Beta Was this translation helpful? Give feedback.
-
|
One type of certification may offer to list the model in the OMF website as an additional incentive |
Beta Was this translation helpful? Give feedback.
-
|
One paid service could be to test the model's reproducibility, which could be the highest certification level. Another level could be for models that are submitted with appropriate unit tests. |
Beta Was this translation helpful? Give feedback.
-
|
like technology readness level， TRL |
Beta Was this translation helpful? Give feedback.
-
|
An example of levels of maturity - FAIR data maturity model https://www.rd-alliance.org/group/fair-data-maturity-model-wg/outcomes/fair-data-maturity-model-specification-and-guidelines-0 |
Beta Was this translation helpful? Give feedback.
-
|
Parallelism with CITI training: certification is required for certain activities |
Beta Was this translation helpful? Give feedback.
-
|
Incentives. A modeler would be interested in having their model certified if: |
Beta Was this translation helpful? Give feedback.
-
|
Journals require the certification stamp/badge |
Beta Was this translation helpful? Give feedback.
-
|
Funding agencies/sponsor favor certified outputs |
Beta Was this translation helpful? Give feedback.
-
|
Tenure and promotion. If other products are taken into account in the decision process, this could become an incentive. |
Beta Was this translation helpful? Give feedback.
-
|
Possibility of automatizing the verification that the most basic checklist is completed before the review process gets to a human |
Beta Was this translation helpful? Give feedback.
-
|
There is a real demand for training in good practices in modeling, could even be a paid service |
Beta Was this translation helpful? Give feedback.
-
|
The modeling community can incorporate ideas from the data community: eg trusted data repositories |
Beta Was this translation helpful? Give feedback.
-
|
Within the Agent Based Community, the CoMSES Net Library has a reputation. The library accepts other types of models, therefore it could be potentially expanded to support OMF goals |
Beta Was this translation helpful? Give feedback.
-
|
A related project: in our work on Open Source Hardware, Sloan funded some work to develop an API to the OSHWA hardware certification process (https://certification.oshwa.org/) ...the most interesting thing about this is that it allowed different repositories to directly embed certification within their own interfaces, rather than requiring project leads and designers to come to them... |
Beta Was this translation helpful? Give feedback.
-
|
Similarly, ELIXIR's TESS https://tess.elixir-europe.org/ enables people when creating a new event to embed code so that the event then shows in their training database automatically. |
Beta Was this translation helpful? Give feedback.
-
|
This is unrelated, I think, but I wonder whether there's an opportunity to develop a "Model Carpentry" curriculum that would extend/build from Software Carpentry into best practices of open/interoperable model software development and use |
Beta Was this translation helpful? Give feedback.
-
|
One more thing; somewhat adjacent, but the CoRe2 project is doing verification of data/code in a "replication package" at the moment of manuscript reviews (http://core2project.org/) ... am not sure how to think about the difference between certification and versions of peer code review |
Beta Was this translation helpful? Give feedback.
-
Meeting notes and discussions from workshop 4
Beta Was this translation helpful? Give feedback.
All reactions