Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
automatic tests #697
I have perhaps quite unusual feature request. I would like edb to have automatic tests.
I did some testing a while ago. I tested edb under valgrind, under address sanitizer and with debug mode of libstdc++. As the result I found a few bugs. I have fixes for some of them and I'm going send patches when they are ready.
My problem is that I believe that fixes without tests are fragile. Nothing prevent the same code to break in the future again during some refactoring. That's why I would like edb to get some automatic tests.
I can imagine two types of automatic tests in edb:
I believe we can do either 1 or 2 or both. At least for my use case either option is OK.
I think I can spend some time implementing these tests. But before I start I would like to ask your opinion.
At the moment I'm currently leaning toward implementing option 2.
I would love to have some tests be available. It's just not always easy to imagine how to implement them for an interactive GUI application such as edb...
I can definitely imagine a test that use the DebuggerCore plugin directly, uses it to manipulate a test binary, and looks at the results. That is certainly a "independent" unit which can/should be tested on its own, even without a GUI.
So as for the type of tests, I would say whatever is really doable and would be effective.
I am also curious what bugs you have found? Can you please report them so I am aware of them, even if you intend to fix them yourself?
In my case one bug is related to RunUntilRet which is implement in main binary, therefore testing DebuggerCore alone is not enough.
OK, then I would like to try the option 2. I tried to test a simple qt application, here what the test looks like:
I'll try to make a prototype for edb, so we can have something to discuss.
One problem is SIGABRT ("some debug event handlers weren't property removed") in DebugEventHandlers when edb is closed while RunUntilRet is running. Looks like RunUtilRet is not destroyed before DebugEventHandlers. I haven't investigated yet.
The others are valgrind reports of accessing uninitialized memory. Valgrind reports lots of issues, but most of them are manifestations of only few problems. I don't know how many distinct of them are there. I just decided to fix them one by one until valgrind output nothing. Also most of them are harmless (technically it's undefined behavior, but in practice they are harmless) as the uninitialized memory is accessed, something based on it is computed and then discarded. Nothing serious that's why I decided not to bother you until I have some patches.
I'll report what I found.