New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent unity from bailing on failed tests #616
Comments
Hi. Unity aborts the current test on a failure (when used by itself or with Ceedling). It's the "current test" portion that is important here. In either case, it should move on to the next test. My suspicion is that something about your structure on your target tests isn't quite right. Maybe all your tests are getting called from a single RUN_TEST() call? Maybe you don't have RUN_TEST() wrappers around your tests at all? Depending on your target, people also can run into situations where their tests run into memory problems, and Unity has no way to recover from it and so execution just stops. Are you using a generated test runner or are you creating your own? If the latter, it'd be good to see an example of what that looks like. Are you using the default RUN_TEST macro, or have you created your own? If the latter, it'd be good to see that as well. :) |
Sorry, providing more information: My test runner is created manually, it runs within an OS task: void Task00(...)
{
// <snip>
// UNITY_BEGIN() is called from the startup hooks
RUN_TEST(TestDopFun);
RUN_TEST(TestDopOwner);
RUN_TEST(TestPwmOwner);
RUN_TEST(TestPwmFun);
RUN_TEST(TestLpoOwner);
RUN_TEST(TestLpoFun);
UNITY_END();
__asm("debug");
} When everything is OK I'm seeing all results correctly (handled via the debugger interface):
If, say,
Only if I use I think the main gotcha is that my If I'm using |
Thanks. This is all good information. You're using the default RUN_TEST macro, I assume? I don't believe there should be a problem with calling UNITY_BEGIN() from a different task, so long as you're confident it gets called before the first test. It's not handling any of the abortframe management for setjmp, so the issue of it using a different stack shouldn't be an issue. I don't have a better guess at this point, unless you've customized RUN_TEST. You could consider initializing the actual communication from where you are now, and then assigning the |
My unity configuration is the following #define UNITY_OUTPUT_CHAR(a) UnityOutPut(a)
#define UNITY_OUTPUT_START() UnityOutInit()
#define UNITY_OUTPUT_FLUSH() UnityOutFlush()
#define UNITY_OUTPUT_COMPLETE() UnityOutClose()
#define UNITY_EXCLUDE_LIMITS_H
#define UNITY_EXCLUDE_STDDEF_H
#define UNITY_INCLUDE_PRINT_FORMATTED
#define UNITY_INT_WIDTH 32
#define UNITY_LONG_WIDTH 32
#define UNITY_POINTER_WIDTH 32
#define UNITY_EXCLUDE_FLOAT
#define UNITY_EXCLUDE_FLOAT_PRINT
// TODO: prevent Unity from BAILING on failures https://github.com/ThrowTheSwitch/Unity/issues/616
// #define UNITY_EXCLUDE_SETJMP_H So I'm really not customizing My current fear is that the |
It IS sounding like setjmp/longjmp itself is the problem. Optimistically, I'd say that if they include the setjmp library, that means they support it... but it's also one of those libraries that doesn't get the attention that the other standard libraries do. It sounds like you are in a multitasking environment, which complicates things slightly. It is important that all interactions with setjmp / longjmp happen from the same task (and therefore stack). From everything you've written, though, it sounds like you have that under control already. I'm writing this so (a) it might jog your memory of something that might be a little different and (b) anyone coming to this thread later understands it's important. ;) I think I've run out of helpful things to say at the moment. If you learn anything or have questions, dump it here. Good luck! I'll be interested to see what's going on. |
Yeah, it is definitely not ideal :D. The problem is that the execution of the #define TEST_PROTECT() (setjmp(Unity.AbortFrame) == 0)
#define TEST_ABORT() longjmp(Unity.AbortFrame, 1) I have my own clone of the I'm not really blocked since I'm wrapping |
Ah. this makes a lot of sense. Long-term, it probably makes sense to allow people to customize TEST_PROTECT and TEST_ABORT. I'm glad this isn't holding you back and that you've found an answer that makes sense! |
Yeah, for now, I'm leaving my integration as follows:
Then my output is fine: ../Application/TestOwner/src/Task00.c:57:TestDopFun:PASS
../Application/TestOwner/src/Task00.c:120:TestDopOwner:FAIL: Expected 0 Was 1
../Application/TestOwner/src/Task00.c:59:TestPwmOwner:PASS
../Application/TestOwner/src/Task00.c:60:TestPwmFun:PASS
../Application/TestOwner/src/Task00.c:61:TestLpoOwner:PASS
../Application/TestOwner/src/Task00.c:62:TestLpoFun:PASS
-----------------------
6 Tests 1 Failures 0 Ignored
FAIL Should I keep this issue open? |
Yes, let's keep it open. Memo to myself or whoever gets here first: Make |
Thanks, also for the superfast feedback! I'm very happy with |
Thanks! I appreciate the kind words. You've identified our big goal there! I'm glad to know we're near the target! |
@mvandervoord anything new about customizing In our case, we are using Unity for on target tests and use longjmp version of |
The merged fix for issue #695 should solve this issue as well :) |
I'm using unity on target and it works beautifully. However, when I have a test that fails, then all the other tests are skipped without any output due to the
UNITY_FAIL_AND_BAIL
macro in the assertion functions (leading toTEST_ABORT
).So far the only way to prevent this is to define
UNITY_EXCLUDE_SETJMP_H
in myunity_conf.h
file, which might have some impact on mocking. Is there no other way to do this? I've never seen this behaviour when using ceedling, there the test execution continues on failures.The text was updated successfully, but these errors were encountered: