You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current tests should be removed because they are too difficult to maintain and are hand written, so they probably don't cover enough cases. Tests should ideally be generated.
The difficulty of this is that the output of a program is the state of all the stacks. It is difficult (not possible?) to know what the expected output of a generated push program without running it, unless it isn't generated completely randomly. How can you generate a program in such a way that you know what its output should be?
Also, it is just as important to know what the output of a program should not be. In other words, if we are testing a random program we will have to determine what values should be expected on the stacks after execution. We will also have to check that no other values are on the stacks after execution. This is difficult to check for with programs that were generated with any degree of randomness in them.
The text was updated successfully, but these errors were encountered:
The design for new instruction set tests is complete, and can be seen on the test branch. The tests now take a tabular structure, and are much easier to write and maintain.
All that is left now is to implement these tests for the entire instruction set. Given that this is mostly mindless coding, I feel safe adding this issue to the Beta milestone.
Instruction unit tests have been re-designed again as of #64. Things now use the unittest library which means that the instruction set tests will be included in when you run python setup.py test.
In the future, another issue should be opened to address the large amounts of duplicated code in these tests, although I am not confident that issue will be easy to resolve.
Current tests should be removed because they are too difficult to maintain and are hand written, so they probably don't cover enough cases. Tests should ideally be generated.
The difficulty of this is that the output of a program is the state of all the stacks. It is difficult (not possible?) to know what the expected output of a generated push program without running it, unless it isn't generated completely randomly. How can you generate a program in such a way that you know what its output should be?
Also, it is just as important to know what the output of a program should not be. In other words, if we are testing a random program we will have to determine what values should be expected on the stacks after execution. We will also have to check that no other values are on the stacks after execution. This is difficult to check for with programs that were generated with any degree of randomness in them.
The text was updated successfully, but these errors were encountered: