Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SCT somehow creates error when correct code submitted #10

Closed
hugobowne opened this issue May 27, 2016 · 9 comments
Closed

SCT somehow creates error when correct code submitted #10

hugobowne opened this issue May 27, 2016 · 9 comments
Labels

Comments

@hugobowne
Copy link

In https://campus.datacamp.com/courses/1113/2464?ex=3 ,

when we submit the solution code, python throws an error. this did not happen when SCT wasn't present.

see screenshot:

screen shot 2016-05-27 at 11 57 10 am

@vvnkr
Copy link
Member

vvnkr commented May 27, 2016

So these are issues of scoping. The problem is that test_function() and test_object_after_expression() will run parts of the code in the global scope, but word and shout_word are defined only in the function's scope.

In test_object_after_expression() you can use the argument context_vals to work around this problem. To context_vals you can assign a list of values, which will be assigned to the variable in the context its surrounding test. If it's used within test_function_definition(), the context variables will be the arguments of the function. For example,

*** = solution
def shout(word):
    shout_word = word + '!!!'
*** =sct
test_function_definition('shout', arg_names = False, body = lambda: test_object_after_expression("shout_word", context_vals = ["shouting this"]))

This test will test the object shout_word after running through the body of shout() with its only argument, word, set to "shouting this". In other words, using context_vals you can mock the values of the arguments while testing. Note that this test implies nothing about how you call the argument, you could call it foo instead of word and the test would still past.

You can also use test_object_after_expression() within a test_for_loop() for example, in this case the context_vals will be the values of the loop variable.

If you use test_function() will currently just run in the global scope, and there's no way to set the arguments when running the tests.

I'm thinking of ways to make this less complex and more intuitive.

I think it might be easier to use the outputs argument of test_function_definition() to test whether correct values are printed by the function.

@hugobowne
Copy link
Author

@vincentvankrunkelsven thank you! this workaround doesn't look as complex as some that were used during the dev of testwhat ;-)

@franciscastro try this out (should work for both problems that we had) and we'll let @vincentvankrunkelsven know how it goes asap

@hugobowne
Copy link
Author

@franciscastro about to try this now

@franciscastro
Copy link

It doesn't seem to work for cases when I have to test an object within a function definition, but the function does not have parameters.

05-27-2016-test-function-in-def-error-5-1

@hugobowne
Copy link
Author

@vincentvankrunkelsven the context_vals workaround doesn't seem to be working for us.

In the case that the function we're defining takes no arguments we have tried

  • context_vals = [] (then we get above error 'list index out of range')
    • we have also tried along context_vals as a list of one element and in this case we get the 'not defined' error.

this occurs in https://campus.datacamp.com/courses/1113/2464?ex=3

Also, in https://campus.datacamp.com/courses/1113/2464?ex=4 , the function in which takes an argument, using context_vals=["congratulations!!!"] throws the not defined error. see screenshot.

any other workarounds?

It looks like @franciscastro 's SCTs are written correctly but maybe you want to take a quick look also

screen shot 2016-05-27 at 4 58 41 pm

@vvnkr
Copy link
Member

vvnkr commented May 29, 2016

Seems like it doesn't work as I expected, I'll have a look and it'll be fixed in the next patch. That will probably be put live Monday evening my time, so that would be Monday around noon for you.

@vvnkr
Copy link
Member

vvnkr commented May 30, 2016

Ok so I had a look at your problems:

  • For exercise 3: setting context_vals = [] caused an error. However for the effect you desire, just don't define context_vals, it will be set to None by default, and this will provide the correct behaviour. I added a check in a new version, that causes [] to be treated like None for the context_vals argument.
  • For exercise 4: the problem here is not caused by test_object_after_expression(), but rather by test_function() in your fourth test_function_definition(). As I mentioned before, test_function() is by design only capable of testing arguments in the global scope. (It doesn't have something like context_vals, I think it maybe should). You can check whether the function print() is used by using test_function('print', args=[]) and you can check if the correct output is given in the function definition by using: outputs of the test_function_definition() function. I'm working on a better way of handling the scopes, so test_function() can be used properly within other test functions.

@hugobowne
Copy link
Author

@vincentvankrunkelsven neither of these seem to be working still. perhaps you and I should speak for 5min and/or you could write an example SCT that you think should work?

@vvnkr
Copy link
Member

vvnkr commented Jun 2, 2016

They are fixed

@vvnkr vvnkr closed this as completed Jun 2, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants