Some spans are missing begin offset field

[elizlee]

Describe the bug
When we process annotated documents exported from Inception as JSON files, we use the begin-end offsets of each span to identify words and phrases. In some annotated documents, one of the spans will only list the end offset, which causes an error when we attempt to process such documents. We suspect that the issue might come from annotating spans that begin at the first index of the document.

Snippet
In this example snippet from exported JSON, span 5368 lacks a begin field:

    {
        "12": {
            "_type": "Sofa",
            "sofaNum": 1,
            "sofaID": "_InitialView",
            "mimeType": "text",
            "sofaString": "__DOCUMENT_TEXT_REDACTED_FOR_IP_REASONS__"
        },
        "5363": {
            "_type": "CTEventSpan",
            "sofa": 12,
            "begin": 31,
            "end": 38,
            "negative_example": false
        },
        "5368": {
            "_type": "CTEventSpan",
            "sofa": 12,
            "end": 10,
            "negative_example": false
        }
    }

To Reproduce
Steps to reproduce the behavior:

1. Create a new project
2. In that project, open a document
3. Create a span by highlighting one or more words starting from the very beginning of the document
4. Click "Export" and export the document in UIMA CAS JSON format
5. Navigate to the downloaded .zip file - unzip and open the .json file
6. Below the document text, you should see a span object that contains the field end, but not begin.
7. You can further investigate this by repeating steps 3-6, except by highlighting words in the middle of the document and checking that those spans, in contrast, contain a begin field.

Speculation as to cause
Because this only happens when end offsets are small, it might be the case that this is triggered when begin would have a value of zero. Perhaps something like @JsonInclude.NON_DEFAULT is being used? If you can point us to the relevant code, we are happy to look into this.

• Version and build ID: 2019-05-30 12:43:10, build f086cc6
• OS: macOS
• Browser: Chrome

[reckart]

Hm, ok. We use the JsonWriter from DKPro Core and indeed, it is by default configured to omit default values.

It arrives in INCEpTION via a module we re-use from WebAnno: JsonFormatSupport:

https://github.com/webanno/webanno/blob/5ed87ce971b2a15f818bb685a70223cc2d503d7b/webanno-io-json/src/main/java/de/tudarmstadt/ukp/clarin/webanno/json/JsonFormatSupport.java#L56

So, you could add PARAM_OMIT_DEFAULT_VALUES, false there in the call to createEngineDescription.

However, I wonder if it wouldn't make sense to report the issue upstream to the UIMA issue tracker to suggest always including begin/end offsets even while other 0 values might still be omitted.

[reckart]

We could also consider changing the default for PARAM_OMIT_DEFAULT_VALUES in DKPro Core.