Gradescope does tests in a different order than in the jupyter notebook

[cindygr]

Describe the bug
The jupyter notebook created for the students has the tests in-line (where you put them). However, when the jn is autograded on Gradescope, I think all of the student's code is run FIRST, then the tests are evaluated at the end. So if, for example you declare a variable t = 3 and check that t = 3 in the grader check, it works fine in both cases UNLESS somewhere later in the file you say t =4, in which case the student's JN will say "test passed" but Gradescope will say 3 does not equal 4.

I am 90% sure this is what's happening (since when I made sure there were no duplicate variable names it got fixed). I am also about 70% sure that this was a recent change in... something.... because it only showed up when I tried to update my class this term. Autograder/tests that worked fine last term (Jan - Mar) worked fine, but when I re-ran them to check them this term, they don't...

To Reproduce
Steps to reproduce the behavior:

1. Create an otter-grader JN with the first question with a variable t = 3 in it
2. Create a test in that first question that has t = 3 in it
3. Somewhere after that test (in a second problem) set t = 4
4. Run otter assign... etc to generate the student JN and Gradescope autograder .zip file
5. Submit your "correct" JN
6. It will fail because in the test t = 4

Expected behavior
The tests should be executed at the same location in the JN as they are in the student version

Versions
Python 3.11, Otter 5.1.1, Gradescope (??) as of March 25

Additional context
The submitted code on gradescope is now formatted differently than it was before - all the grader.check cells have disappeared from the student files. I don't know if the two are related...

[chrispyles]

@cindygr I'm unable to reproduce this issue with the steps you provided. Can you provide some notebooks that produce this error? Feel free to DM me in slack (link in README) if you don't want to post them publicly.

[cindygr]

Sure I'm traveling at the moment but will back out the repo and post when I get back. Get Outlook for Android<https://aka.ms/AAb9ysg>

…

________________________________ From: Chris Pyles ***@***.***> Sent: Wednesday, April 10, 2024 11:11:58 PM To: ucbds-infra/otter-grader ***@***.***> Cc: Grimm, Cindy ***@***.***>; Mention ***@***.***> Subject: Re: [ucbds-infra/otter-grader] Gradescope does tests in a different order than in the jupyter notebook (Issue #793) [This email originated from outside of OSU. Use caution with links and attachments.] @cindygr<https://github.com/cindygr> I'm unable to reproduce this issue with the steps you provided. Can you provide some notebooks that produce this error? Feel free to DM me in slack (link in README) if you don't want to post them publicly. — Reply to this email directly, view it on GitHub<#793 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADDOR57FJ6SNSVYO4LXZF2LY4X5P5AVCNFSM6AAAAABFPHYVMWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYHA2TMMRXGE>. You are receiving this because you were mentioned.Message ID: ***@***.***>