As an application developer I want to use CLI tests to test the interface of my program such that incompatible options or invalid inputs can be tested automatically.

[rrahn]

Acceptance criteria

• I can run make cli_test in the build directory of the application and it builds the tests
• there is a demo of the cli tests in the cli test directory that can be also used to automatically test the application template

Tasks

• Change cmake to fetch and build gtest within configuration step using FETCH_CONTENT
• Add a cli_test class that inherits from testing::test and adds support for configuring the application CLI parameter and calling the application.
• Add a cli test for the demo application inside of the application template

Definition of Done

• Implementation and design approved
• Unit tests pass
• Test coverage = 100%
• Microbenchmarks added and/or affected microbenchmarks < 5% performance drop
• API documentation added
• Tutorial/teaching material added
• Test suite compiles in less than 30 seconds (on travis)
• Changelog entry added

[rrahn]

What is the design of the test class? Which functionality is expected?
How to set the program to call?
How to set the program options?
How to control the input/output?

[rrahn]

Some example: https://github.com/joergi-w/app-template/tree/cli_test_class/test/cli

[joergi-w]

Task 1 (Change cmake to fetch and build gtest within configuration step using FETCH_CONTENT) is independent and not related to the User Story above. I created a separate issue for this: #47

[joergi-w]

The Definition of Done here makes no sense (API doc for CLI test?, we do not have coverage test yet, unit tests and microbenchmarks are independent). I will ignore many of them.

[rrahn]

The Definition of Done here makes no sense (API doc for CLI test?, we do not have coverage test yet, unit tests and microbenchmarks are independent). I will ignore many of them.

That won't always be the case. But you did it right to cross through what is not needed. It is merely a reminder on commonly agreed goals for a story. In the future we might want to differentiate between items with a DoD and without. But first let's gather some intel to make an informed decision.

[joergi-w]

This is finished since https://github.com/joergi-w/app-template/pull/5 and https://github.com/joergi-w/app-template/pull/6 are merged.