Programming exercise: Instructor not informed that test case visibility changes do not take effect immediately

[BBesrour]

Describe the bug

As instructor, when you change the test cases visibility, you get a toast (notification) that the visibility of the test cases was successfully updated. This is misleading since students who submitted before this change can still see the test cases. The instructor should be informed that they have to re-trigger or re-evaluate the submissions

To Reproduce

Prerequisites:

• Programming exercise with test case visibility set to always (can be changed under Grading)
• 1 Student
• 1 Instructor

1. As student, submit something in the programming exercise
2. Wait for the result and make sure that you can see the test cases
3. As instructor, change the test case visibility to after due date
4. As student, reload the page. you should still see the test cases even after the visibility was changed

Expected behavior

The instructor should be warned that students who already submitted can still see the test cases and that they have to re-evaluate or re-trigger the submissions

Screenshots

No response

Which version of Artemis are you seeing the problem on?

7.4.1 (4c5382b)

What browsers are you seeing the problem on?

Microsoft Edge

Additional context

No response

Relevant log output

No response

[Strohgelaender]

There already is such an indicator on the top of the page that shows an exclamation mark if the changes are not in effect. Do you think that should be larger?

[BBesrour]

@Strohgelaender Oh, I didn't notice it. Yes, I believe the warning should be more noticeable. Or to mention it in the notification you get after saving the changes. This is debatable though.

[JohannesStoehr]

Re-evaluate will not help to hide feedback afterwards. While the new result will no longer show the feedback for that particular test case, the old results that are also accessible to the students still show the feedback.