Test dependency resolution

[meyercm]

This may not be an issue, but rather my misconception of how Ceedling should work.

Given 3 modules, a, b, and c, where a depends on b and b depends on c;

In test_a.c, I have to include not just a.h (makes sense), but also b.h, and c.h

The workaround is obvious, but I expect that I should just have to include a.h, and Ceedling should be able to identify and pull in all the dependencies recursively. In my larger integration tests, it's quite irritating, as I have a list of #include that is 30-40 lines long, and those tests break frequently when I modify modules that are 2-3 dependency links away and forget to update the integrations.

I've tried manipulating :use_test_preprocessor and :use_deep_dependencies, which seem like the obvious project.yml settings, with no success. Any advice?

[mvandervoord]

Chris:

First, I'll start with my real advice. It sounds like you should be using mocks. When you use a, you're going to have to pull in ALL the dependencies that a file requires, all the way down, unless you tell it otherwise. This becomes unnecessarily complex and it also makes it REALLY hard to test.

Instead, when you test a, you should be including mock_b so that you get a fake version of b to test. The fake version of b doesn't depend on anything else, except cmock, so your chain ends there. You are now fully in control of your b module, allowing you to return or verify ANYTHING you want, instead of just what you can make happen through the interface.

Back to your original question: This is why Ceedling isn't doing the job for you. It wants to give you control over what real and fake things you include in your tests (no matter if you are using CMock or not). Therefore, it's only going to link the files you specify.

I hope this helps. Happy testing!

Mark

[meyercm]

Mark,

Thanks for the advice about mocks. I think there is a balance to be had between mocking and letting collaborators actually participate in tests. Utility modules are the easiest to make a case for: I don't need to mock my LinkedList module, and doing so actually makes the test more brittle: what if I later decide that the CUT is going to use a different data structure internally?

In my unit tests, I generally draw the line for mocking with those modules that the CUT must call as part of it's responsibility: For my AT command parser, I'll mock my UART library, but I won't mock my string utility module, if that makes sense. When it comes to an integration test, and the whole point of the test is verifying the conversation between two/more collaborators, I'll mock only what must be mocked; in the microcontroller environment, I'll only mock the UART/I2C, or possibly mock the driver for external chips like an EEPROM.

In my current project, I have an integration test that doesn't have any code to run in the test case: the whole point of the test is to ensure that as many modules can link together as possible, and I've found bugs with this test (e.g. leaving the static modifier off of the same-named variable in two modules).

I hadn't considered the case of non-CMock includes (even though I have a test with a custom fake, which is exactly what you are talking about). That does make this problem stickier than I'd originally thought, but probably not insoluble given that we have Ruby at our disposal - perhaps include the specified files first, then include the files discovered during the dependency tree walk, and let #ifndef directives do the work of putting the mocks in first.

If you end up re-working the test runners to close other issues and you see a nice way to fold in a feature like this, I think it would be a nice improvement, but definitely falls in the nice-to-have category: I originally opened an issue because I thought I had the configuration wrong.

Thanks,

Chris

[mvandervoord]

Chris: I definitely agree. There is a place for integration tests and a place for interaction tests. Both are very valuable and knowing when to apply each is a great skill. I apologize if it sounded like I was talking down to you. The majority of the people asking questions on this list are fairly new to mocking and therefore don't understand where it works best, yet. I was speaking from that standpoint, though that was clearly not what you required. In any case, I really appreciate your ideas on how we could improve the test runner generator. I believe there are some other changes coming to this tool soon, and this might be a good place to implement some of your thoughts. Thanks! Mark

[rwerthman]

I'm working with legacy code now and trying to add tests. I ran into a situation where given 3 modules, a, b, and c, where a depends on c and b depends on c. Instead of a including c directly b was included in a. This created a deep dependency according to the ceedling documentation.

In test_a.c, I included mock_b.h. But, when running ceedling test:all I got undefined references for functions from c.h during the linking phase. To fix this, instead of using mock_b.h in test_a.c I had to use mock_c.h. This is legacy code, so I don't know why the developer included b.h in a.h but not c.h in a.h.

[tobyjwebb]

@mvandervoord

I've sent a PR (#378) which should solve this issue (disabled by default - you have to enable it explicitly in the project.yml file). At least, it's worked so far on our test repo.

:)

PS: Feedback welcome.