Question Marks break Exact Test

[sparry0407]

If exact text in a test case contains a question mark, it breaks the comparison. so, if you have a test of:
output="ABC?"
This will fail a program that outputs:
ABC?

[jcrodriguez-dis]

I have not been able to reproduce this issue.
Please, show your vpl_evaluate.cases file and the student's program that does not pass the test.

Notice that your case definition does not accept anything after the "?" neither a newline.
To define a case that accepts exact outputs with and without a newline, you can use two outputs in this test case

output="ABC?"
output="ABC?
"

If some output defined matches the program output, the test case is passed.

[sparry0407]

This one works: [cid:0dd0a328-e4a2-4888-a525-4cafd820ede1] [cid:f2c86be4-1f3d-42c3-9937-b288b4d770e2][cid:15197f69-9cb1-48dd-90e2-ff377cfd5b35] This one does not: [cid:51c08b53-7b01-4551-886d-d4ea9693571b] [cid:ffb2cbef-bca9-44a5-823b-3a530ffa5f8a][cid:97353f6f-5dbc-4240-9818-fe0c43e7bf76]

…

________________________________ From: jcrodriguez-dis <notifications@github.com> Sent: 29 October 2020 11:01 To: jcrodriguez-dis/moodle-mod_vpl <moodle-mod_vpl@noreply.github.com> Cc: Stephen Parry <sparry@chapeltownacademy.com>; Author <author@noreply.github.com> Subject: Re: [jcrodriguez-dis/moodle-mod_vpl] Question Marks break Exact Test (#117) I have not been able to reproduce this issue. Please, show your vpl_evaluate.cases file and the student's program that does not pass the test. Notice that your case definition does not accept anything after the "?" neither a newline. To define a case that accepts exact outputs with and without a newline, you can use two outputs in this test case output="ABC?" output="ABC? " If some output defined matches the program output, the test case is passed. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#117 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQCFKOZR7F5EUYG3WI4VNZ3SNFDQPANCNFSM4TC5TLCA>.

[jcrodriguez-dis]

Sorry, I can not see your images.

I think that may be better to use the GitHub interface than reply by email.

You can use this link to access the thread

[sparry0407]

Correct:

Incorrect:

It also does the same if the question mark is in the body of the text instead of at the end.

[sparry0407]

I tried your suggestion, which worked. It does seem there is an inconsistency between how the tests are treated depending on whether question marks (and possibly other symbols) are in the text or not.

[jcrodriguez-dis]

Yes, you are right.

I have reviewed the code and found that the exact text matcher trims (cleans spaces, tabs, and newlines) the right side of the output if the searched text end with an alpha char.
This code was introduced to avoid frequent fails by newlines at the end of the output.
This code does that the "ABC" matches with print("ABC") in python, but really print("ABC") print also a newline.

I tried to reproduce the problem using C, but without including a "\n" at the end of the printf() this is why I don't found the problem.

Thanks for your report, I will fix this bug in the next release.

[sparry0407]

Thank you for your prompt and very helpful replies. I have another query but I will raise that separately.

…

________________________________ From: jcrodriguez-dis <notifications@github.com> Sent: 30 October 2020 12:38 To: jcrodriguez-dis/moodle-mod_vpl <moodle-mod_vpl@noreply.github.com> Cc: Stephen Parry <sparry@chapeltownacademy.com>; Author <author@noreply.github.com> Subject: Re: [jcrodriguez-dis/moodle-mod_vpl] Question Marks break Exact Test (#117) Yes, you are right. I have reviewed the code and found that the exact text matcher trims (cleans spaces, tabs, and newlines) the right side of the output if the searched text end with an alpha char. This code was introduced to avoid frequent fails by newlines at the end of the output. This code does that the "ABC" matches with print("ABC") in python, but really print("ABC") print also a newline. I tried to reproduce the problem using C, but without including a "\n" at the end of the printf() this is why I don't found the problem. Thanks for your report, I will fix this bug in the next release. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#117 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AQCFKO6BHXVDPDXFFJG2IL3SNKXVBANCNFSM4TC5TLCA>.

[jcrodriguez-dis]

Fixed in VPL 3.5