The problem of easy problems

[marijanbeg]

Is your feature request related to a problem? Please describe.
We demonstrate the problem we encounter in example 01-easy-problems in https://github.com/marijanbeg/pybryt-examples

Summary

A student has an exercise to write a function with signature maximum(a), which finds and returns the largest element in a list a. The solution we expect in a beginner-level Python course is:

def maximum(a):
    res = a[0]
    for i in a:
        if i > res:
            res = i

    return res

The reference solution would be:

def maximum(a):
    res = a[0]
    pybryt.Value(res,
                 name='initial_value',
                 success_message='SUCCESS: Great! You declare the first element to be the largest before the loop.',
                 failure_message='ERROR: Hmmm... Did you declare the first element to be largest before the loop?')
    for i in a:
        if i > res:
            res = i
            pybryt.Value(res,
                         name='larger',
                         success_message='SUCCESS: Very nice! You are finding larger elements.',
                         failure_message='ERROR: Hmmm... Are you finding the larger elements than the declared one.')

    pybryt.Value(res,
                 name='res',
                 success_message='SUCCESS: Wow! You found the largest element.',
                 failure_message='ERROR: Hmmm... Something is wrong in your function.')
    return res


pybryt.Value(maximum([-3, 1, 0, 5, 19]), name='solution')

However, whatever the solution of the student's code is, we are not able to validate it because all elements of the input list are in the footprint anyway because of the for i in a loop, which validates any PyBryt annotation from the reference solution. Exercises like this are very common in beginner-level coding exercises where feedback on the student's implementation (PyBryt's main power) is essential.

[ranigb]

What are the kind of mistakes you see students do?

I can think of several possible errors:

1. not iterating over the entire list
2. doing for i in rage(0,a) instead of for i in a
3. not updating the maximal value

I can see how to catch 1&2 with a reference implementation. To check for the 3rd one a standard unit test can be used or using pybryt by adding to the reference implementation the line:
pybryt.Value('__solution', name='solution')

and when running students solutions wrapping their function with
if maximum([-3, 1, 0, 5, 19]) == 19): message = '__solution'

[marijanbeg]

Note to self: As discussed in the tech meeting, pre-processing footprints depending on whether values come from the variable with the same name could be one way forward. @rolotumazi has possibly an even better solution and he will summarise it in this issue soon.

[rolotumazi]

So, I'll briefly outline what me and @marijanbeg discussed after a good tech meeting.
The discussion is around pre-processing footprints by identifying values in the footprint by checking if they have the same variable name. The common causes might be an iterator in a loop or a function such as the 'maximum(a)' illustrated above by @marijanbeg . An extension of this idea @marijanbeg and I discussed would be to annotate these values according to variable name and group these values in something like a collection. Then, if possible, further annotations on the footprint could be set to ignore values in the footprint already annotated.
The reason for not clearing or cleaning the footprint in pre-processing of these values is that these annotations might be useful in analysing the code further. I can already say that in my limited experience it would be beneficial to catch iterators this way so I can search for values that are recursively related. It's difficult at this stage for me to imagine how else it might be used at this moment but if is at all possible I think retaining the information would be better than discarding it.
Please let me know your thoughts.