Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "Annotation task" or "Labeling task" project type #1032

Open
tompollard opened this issue Apr 27, 2020 · 0 comments
Open

Add "Annotation task" or "Labeling task" project type #1032

tompollard opened this issue Apr 27, 2020 · 0 comments
Labels

Comments

@tompollard
Copy link
Member

tompollard commented Apr 27, 2020

(I've used "annotation" and "label" interchangeably in this issue. Is there a difference and if so can we define the difference in the thread?)

We would like the community to be able to contribute annotations/labels for datasets on PhysioNet. These labels would capture details of interest about data such as images (e.g. chest x-rays) or waveforms (e.g. ECGs).

Some of the key challenges in doing this are:

  • defining what type of labels are of interest (i.e. what exactly are we asking people to label?).
  • making labels available for download and reuse in an simple-to-manage, simple-to-understand manner.
  • incentivising the community to contribute.

To help address these challenges, my suggestion is that we introduce a new "Annotation task" (or "Labeling task") project type, which would perhaps be most similar to the "Challenge" project type.

Someone interested in gathering labels would create a new project in the usual way, specifying:

  • Title: e.g. Labelling sinus vs non-sinus ECG rhythms.
  • Background: What is the background to the task? Why are labels needed?
  • Goal: What is the ultimate goal of the task?
  • Participation: Who should participate in the task, and how?
  • Data description: What is the data that is being labelled?
  • Evaluation (?): Will the quality of annotations be evaluated?

The project page would include clear instructions for providing annotations. It would probably include a link to one (or multiple) tools for doing the task. As annotations are submitted (and perhaps approved by the contributor), they will be archived in the "Files" section of the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant