Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push Instruction Set Tests #35

Closed
erp12 opened this issue Dec 15, 2016 · 2 comments
Closed

Push Instruction Set Tests #35

erp12 opened this issue Dec 15, 2016 · 2 comments

Comments

@erp12
Copy link
Owner

erp12 commented Dec 15, 2016

Current tests should be removed because they are too difficult to maintain and are hand written, so they probably don't cover enough cases. Tests should ideally be generated.

The difficulty of this is that the output of a program is the state of all the stacks. It is difficult (not possible?) to know what the expected output of a generated push program without running it, unless it isn't generated completely randomly. How can you generate a program in such a way that you know what its output should be?

Also, it is just as important to know what the output of a program should not be. In other words, if we are testing a random program we will have to determine what values should be expected on the stacks after execution. We will also have to check that no other values are on the stacks after execution. This is difficult to check for with programs that were generated with any degree of randomness in them.

@erp12 erp12 changed the title Come up with a way to generate tests of the push language. Push Instruction Set Tests Dec 27, 2016
@erp12 erp12 modified the milestone: Pysh Beta Dec 27, 2016
@erp12 erp12 added the tests label Dec 27, 2016
@erp12 erp12 added this to the Pysh Beta milestone Dec 28, 2016
@erp12
Copy link
Owner Author

erp12 commented Dec 28, 2016

The design for new instruction set tests is complete, and can be seen on the test branch. The tests now take a tabular structure, and are much easier to write and maintain.

All that is left now is to implement these tests for the entire instruction set. Given that this is mostly mindless coding, I feel safe adding this issue to the Beta milestone.

@erp12
Copy link
Owner Author

erp12 commented Jun 16, 2017

Instruction unit tests have been re-designed again as of #64. Things now use the unittest library which means that the instruction set tests will be included in when you run python setup.py test.

In the future, another issue should be opened to address the large amounts of duplicated code in these tests, although I am not confident that issue will be easy to resolve.

@erp12 erp12 closed this as completed Jun 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant