Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variable are not shared across test cell in Python notebooks #426

Open
joelostblom opened this issue Oct 12, 2021 · 5 comments
Open

Variable are not shared across test cell in Python notebooks #426

joelostblom opened this issue Oct 12, 2021 · 5 comments
Labels
enhancement New feature or request

Comments

@joelostblom
Copy link
Contributor

In R, defining a variable in one test cell will make it available to all other test cells for that question. In Python, the variable needs to be assigned in each test cells, which means performing the same (potentially expensive) computation multiple times. It would be great if Python test would be consistent with how Otter works for R tests and share namespace for variables.

Versions
Otter 3.1.4
Python 3.9.7

@joelostblom joelostblom added the bug Something isn't working label Oct 12, 2021
@chrispyles
Copy link
Member

For Python, this is intentional. We don't want tests to clutter the student's global environment, so we duplicate it before running any tests on it. The only reason this behavior is present in R is because we haven't yet found a way of running the tests correctly without running them directly in the student's environment. We would like to standardize the behavior, but the goal would be to make R like Python, not Python like R.

@joelostblom
Copy link
Contributor Author

Thanks for the reply! It makes sense to restrict the test cells from cluttering the students environments. I think it would still be quite useful to be able to create test constants (both individual values and functions) that would be accessible to all tests for that particular question, would you consider adding a way to define these, e.g.

Test cell 1

# BEGIN TEST CONSTANT
def some_long_function_def(student_obj):
    return ...
# END TEST CONSTANT
test_cont = expensive_computation(student_obj)  # TEST CONSTANT

assert test_const['value1']
assert some_long_function_def(test_const['value1'])

Test cell 2

assert some_long_function_def(test_const['value2'])

The reason for not having both asserts in the same cell would be to be able to give partial points if one of them is correct.

@chrispyles chrispyles added enhancement New feature or request and removed bug Something isn't working labels Oct 20, 2021
@joelostblom
Copy link
Contributor Author

I believe it would be possible to implement the test constant idea via setting the Execution context for the doctest.

@stale
Copy link

stale bot commented Dec 23, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix This will not be worked on label Dec 23, 2021
@stale stale bot removed the wontfix This will not be worked on label Dec 23, 2021
@chrispyles chrispyles removed the pinned label May 24, 2022
@joelostblom
Copy link
Contributor Author

joelostblom commented Sep 28, 2023

I encountered this again and wanted to check in in case there is any other way to achieve what I would like to do. Currently I'm seeing a lot of duplication code in my test, for example in this instance where the only difference is in the assert messages:

image

Since you mentioned above that it is unlikely that sharing of variables will be implemented in the future, maybe the best solution is to keep trying to get #536 working, since that would allows writing everything in one cell. Although I don't think that would not allow for partial points per assert statement, the points are just split per test cell and not per assert, right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

2 participants