Jury submissions should be judged with low priority

[0xJoey]

Currently, Jury submissions have equal priority to contestant submissions.

[0xJoey]

@RagnarGrootKoerkamp

[RagnarGrootKoerkamp]

Context: We were re-adding problems for the main contest during the testsession, and the corresponding jury submissions took priority over team submissions from the testsession

[meisterT]

The normal prioritization should take of this naturally since you add all of these submissions as one team. See the comment and implementation here:
https://github.com/DOMjudge/domjudge/blob/main/webapp/src/Service/DOMJudgeService.php#L1228

Doesn't that match what you have been seeing?

[RagnarGrootKoerkamp]

@mpsijm Do you remember the precise behavior we saw? I wasn't paying close attention at the time.

[mpsijm]

We were doing two things during the test session:

1. Changing the time limit of a problem and rejudging all submissions for that problem. When doing so, you can select "priority" as "low", "medium", or "high" (which we set to "low", to not interfere with the test session). This works as expected, as the rejudgings were only run after all submissions for the test session were done being judged (15 minutes after the test session ended).
2. Completely deleting a problem and re-uploading it, because all test cases were changed. When doing so, we could not select a priority, and it seemed like the jury submissions of this problem had priority over the team submissions for the test session, because all the jury submissions were judged before it continued with the team submissions for the test session again. I don't think this matches the behaviour in the comments of the function that @meisterT mentioned.

Of course, we would have had less problems with this if there would have been more than one judge host alive during the test session, but ah well, that's what happened. 😂

[meisterT]

Thanks @mpsijm - that shouldn't have happened. I'll have a closer look to see what was going on and how to fix it.

[RagnarGrootKoerkamp]

Ah, one additional thing I just remembered: We were running the *testsession*, but we re-uploaded a problem in the *main contest*. So maybe the current prioritization only works inside a single contest, not across multiple contests. (Oh apparently I already mentioned this before - I forgot that I remembered 😅. But anyway this is probably the root cause.)

[meisterT]

I am working on a visualization that I want to add to DOMjudge eventually, but requires some tweaks before I can do so.

For this particular case, this is how it currently looks:

It shows what each judgehost was busy with. You clearly see some setup, the test session and the actual contest (where you added more machines in the middle?).

Zooming in, you can see that each judgehost is basically two rows, one for the judging it was working on and one for all the judgetasks, but the second row I labeled with the team ID to understand this issue better.

Zooming out again on the relevant time during the test session, I double clicked one judgetask by team 1 to highlight all of them:

From that it doesn't look like the submissions from team 1 took priority over normal team submissions. There's still the chance that I visualized stuff wrong since I had to filter out negative duration where I don't know how they came to be.

[meisterT]

Did you disable judging for some time during the test session (e.g. disable the judgehost or by an internal error)?

[vmcj]

Did you disable judging for some time during the test session (e.g. disable the judgehost or by an internal error)?

I think we added more judgehosts during the testsession, and 1 step for that is to pause all judgehosts.

[nickygerritsen]

Yep and during the contest we added even more. I felt like some Tobi guy stealing all hardware.

[mpsijm]

Let's see... If I remember correctly:

• Problem G has been removed and re-uploaded twice. The first re-upload was as problem ID 16 (which is now missing in the problems list), and I think this one happened during the test session. The second re-upload was after the test session, as problem ID 19. Because the old version of G (with ID 16) is now removed, could it be that this causes the missing data during that short time window during the test session?
• During the entire test session, we have had only one judge host running. It was only put down after all submissions from the test session were done judging. Right before the contest started, we had six judge hosts. It could very well be that more judge hosts have been added during the contest, but during the test session we definitely had only one.