Task 1: Inconsistnecies between durationUsedToJudgeMs and judgedAtUnixTS

[bruluk5w]

Hi,
I analyzed the FIRA annotations and have a few questions about how the judgment time was measured in FIRA.
As shown below the the durationUsedToJudgeMs is mostly shorter than the absolute time difference between consecutive judgements.
How does FIRA measure durationUsedToJudgeMs? The data suggests that it tracks user activity and can exclude at least some of an annotator's inactivity.

The more puzzling fact is, that the duration derived from the judgment timestamps can also be lower than what durationUsedToJudgeMs states? Can you verify that this is only due to annotators going back and editing the answer (increasing the value in durationUsedToJudgeMs while keeping the timestamp the same?
Cheers
Lukas

[sebastian-hofstaetter]

Hi,
as far as I am aware durationUsedToJudgeMs is measured on the client, and the unix timestamp on the server. Fira uses asynchronous (potentially delayed) uploads, this definitely introduces some inconsistencies. To be honest we never looked at that connection before. For the published data we also capped the durationUsedToJudgeMs at 3 minutes or so to prevent large outliers (because if the annotator left the browser window open for a while and then came back to it, the duration naturally would be larger). If you want to look at the duration per judgment, I would only look at durationUsedToJudgeMs. And the unix timestamp could be used for time of day analysis, or to determine sessions.
Best,
Sebastian