Spec Refactor: Object checks

[machow]

Problem

Some tests are extensions of test_object, either by testing the "value" of an object, or by testing something code (i.e. AST) related.

value tests check isDefinedInProcess:

• test_dictionary
• test_data_frame
• test_mc???
• test_object_after_expression

code tests are similar in that they involve grabbing something by "name" in the AST:

• test_object_accessed (mentioned in the specs)

Solution

Collapse the "isDefinedInProcess" checks into single place, with chained methods for further checks, like check_object('d').is_dictionary(with_keys = ['a', 'b']), or something.

Misc

• isDefinedProcess is redundant, as included w/in DefinedProcessTest

[filipsch]

DefinedProcessTest is checks if it is defined (on the solution side, no test), DefinedProcessTest uses isDefinedProcess to build a test to check if it is defined (on the student side).

I wouldn't include test_mc() in the test_object() function..

[machow]

The spec uses test_object_accessed in this way...

ex().check_object("arr.shape").check_accessed()

But AFAIK the current form of test_object doesn't count properties as valid input. For example..

*** =solution

class X:
    a = 1

X.a

*** =sct

test_object_accessed('X.a')    # passes fine
test_object('X.a')                     # fails

In a sense, should initial test of test_object and the full purpose of (the new) check_object just be to assert that something exists? For example, that the statement X.a doesn't raise an error in either process. Or should test_object_accessed be treated as a fundamentally different test (since its current implementation on uses the AST.

The key question here is under the new spec, what does it mean to say ex().check_object('arr.shape')? It sounds more similar to test_expression_result('arr.shape') than the current implementation of test_object. My vote is to treat test_object_accessed as more like test_student_typed than the object tests.

[filipsch]

Good you raise this.

I think indeed that test_object_accessed() is fundamentally different from test_object(), just like test_function() is fundamentally different from test_function_definition().

[machow]

Now that I've refactored tasks.py a few times, and spec'd out subtests, I've realized that test_object and friends can be expressed as parser output with a "part" comprised of just an AST name node. So something like check_object('a').has_equal_value('msg') and check_list_comp(0).comp_iter().has_equal_value('msg') would be doing similar things.

TODO:

(just need to tweak the implementation on the spec-test-object branch)

• implement a has_key func
• implement a isinstance func
• check_object spec'd
• check_dict spec'd
• check_df spec'd

[machow]

check_object, is_instance, has_key, check_dict, check_df need to be exposed to SCT creators....

[machow]

On prod in commit ee99fb5