Support 5 different levels of uncertainty in user/machine annotations

[Janchorizo]

Decide on the way to approach and implement the necessary changes to support
specifying a floating number for the certainty level of an annotation. This includes
changing the used attribute (possibly to deg), changing the annotator to process
each specific tag and assign styles based on the numerical value, and adding
back-end support for annotations made using this new approach.

[michalkozak]

We have talked with Alex yesterday about it. We decided to add two new level values to our list. So the list will consists of the following 6 values:

• unknown
• very low
• low
• medium
• high
• very high

These values are for humans. So I suggest that @bug-rancher would add validation in the backend that values are from this list, and @Janchorizo could add these values to the annotator.

The degree attribute is for automatic annotations made by algorithms (values for this attribute are numbers between 0 and 1). But we don't need a parameter in API for the degree, because algorithms won't use API for annotating. However we need that all API that return annotations should return degree as well. I think that @bug-rancher can add degree to the returned JSON for each uncertainty annotation.

[Janchorizo]

There are currently just four supported values for this cert attribute according to TEI specification:

• high
• medium
• low
• unknown

Should I handle these new options already, or is this going to go along with a change
proposal for the specification?

[michalkozak]

I think you can handle these new options and simultaneously we will submit a change request to change the closed list to semi-open.

[bug-rancher]

...

• unknown
• very low
• low
• medium
• high
• very high

These values are for humans. So I suggest that @bug-rancher would add validation in the backend that values are from this list, and @Janchorizo could add these values to the annotator.

I added validation to all parameters with closed list of options ('category', 'locus', 'certainty') in Annotator's add annotation request, including new options in 'certainty' parameter.

... However we need that all API that return annotations should return degree as well. I think that @bug-rancher can add degree to the returned JSON for each uncertainty annotation.

When we add 'degree' attribute to element, it will be returned in 'get annotations' request by default, as any other attribute of this element, so we don't need to modify anything in this case.

All changes are on the api_vis branch.

Regards

[michalkozak]

We will have to add our own extension of TEI in a scope of projects. It is unlikely that the TEI Council will amend the specification see change request.

But we can generate such extension in every file uploaded to the platform. I will prepare such extension.

[michalkozak]

During uploading files to PROVIDEDH platform, each TEI file should have added the following line at top (but after if it exists):

<?xml-model href="https://providedh.ehum.psnc.pl/tei_providedh.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>

This schema contains an extension of the TEI schema attribute teidata.certainty to 6 values.

Moreover at endpoint https://providedh.ehum.psnc.pl/tei_providedh.rng the following file should be exposed: https://drive.google.com/drive/folders/1VfxaYv94DPeUXN8Zu7E6b3CIV4OZxFks

[bug-rancher]

During uploading files to PROVIDEDH platform, each TEI file should have added the following line at top (but after if it exists):

<?xml-model href="https://providedh.ehum.psnc.pl/tei_providedh.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>

This schema contains an extension of the TEI schema attribute teidata.certainty to 6 values.

Moreover at endpoint https://providedh.ehum.psnc.pl/tei_providedh.rng the following file should be exposed: https://drive.google.com/drive/folders/1VfxaYv94DPeUXN8Zu7E6b3CIV4OZxFks

Done.