Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make tests agnostic to specific addresses #2

Open
thass0 opened this issue Aug 14, 2023 · 0 comments
Open

Make tests agnostic to specific addresses #2

thass0 opened this issue Aug 14, 2023 · 0 comments
Labels
help wanted Extra attention is needed

Comments

@thass0
Copy link
Owner

thass0 commented Aug 14, 2023

At this point in time, many unit (and some integration) tests rely on checking for specific addresses to validate the output of different parts of the debugger. This is problematic because if the addresses in one of the example binaries used in testing change just a bit, the tests that expect them will fail. Those failures will usually be false negatives, since it's just the binary that contains a different address after compiling again.

An obvious solution that comes to mind is to compile all the examples once and check them into version control. This way they would stay the same no matter what machine the tests
run on. This approach has the disadvantage that the range of different compilers and platforms tests is much more narrow. Specifically the tests will exclusively make assertions about the platform they were once compiled on.

In the integration tests, regular expressions might be used to check that there are addresses at the right positions. The downside of this is that any address will be accepted. Since all addresses passed around internally have their own address type anyways, the type system already enforces the presence of addresses. Hence, this solution decreases the value of the assertions drastically, too.

Lastly, we could assert that any given address must be in a specific (narrow) range. I've seen multiple tests fail because of off-by-one assertions where the new address was only slightly different.

This is definitely still an open problem and I'd be happy to discuss different options.

@thass0 thass0 added the help wanted Extra attention is needed label Aug 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant