Skip to content

Latest commit

 

History

History
63 lines (45 loc) · 4.93 KB

TESTING.md

File metadata and controls

63 lines (45 loc) · 4.93 KB
Raw

Testing 'oneview' cookbook

The 'oneview' cookbook already provides a unit testing strategy, and linting tools to ensure its code style. This document will cover on how to execute and implement these tests.

Executing tests

The tests can be executed by rake tasks, guard watches, and by their original tools, like rspec, chefspec, rubocop and foodcritic.

Rake

All the test strategies and checks can be executed by the rake command. Please use rake -T to see a full list of commands.

Guard

Guard will watch the changes and execute unit tests and style checks. Just activate it to have real time test execution on every change.

Unit tests

All unit tests are inside the spec folder. You can execute them manually by using RSpec and specifying the test files, like rspec spec/path/to/my/tests.

For pure lib tests the projects use solely RSpec, however, for Chef custom resources tests we use a variation of RSpec, the ChefSpec. Both of them work the same way by issuing the rspec command.

Coverage

The unit tests issue a coverage report generated by SimpleCov for the pure lib files, and ChefSpec issues the report for the Chef custom resources (Since SimpleCov does not support it).

However, you must pay attention because SimpleCov's and ChefSpec's coverage report conflict with each other, i.e. as soon ChefSpec's coverage starts running, it will interrupt the SimpleCov's coverage status and it will generate the report at that point. So, because of this, running all pure ruby lib files before any Chef custom resource is extremely important, otherwise, the SimpleCov report would be incorrect.

The main difference between SimpleCov and ChefSpec coverage is that SimpleCov reports line coverage and ChefSpec shows custom resource touch coverage.

Style checking

Style checking is done by rubocop for all the custom resource providers and pure ruby lib files, and foodcritic to analyze testing recipes and examples.

Functional tests

Every resource must have an example for each action that shows how to use it. These examples can be used to perform functional and OneView integration tests but they need to be executed manually using chef-client.

Implementing tests

All code must have associated tests, be it the already implemented or newly submitted, and this section covers what tests need to be implemented.

Unit tests

The unit tests are required for each pure ruby lib methods and for each Chef custom resource action.

Minimum requisites

The necessary amount of testing is dictated by who is implementing and by who is accepting the code, however, there is a floor acceptance level that must be followed:

  1. All the pure ruby lib code must have unit tests hitting each line of them, i.e., the line coverage for this kind of code must be 100%.
  2. Each custom resource must have at least one unit test for each resource action.

Lib file tests

The pure ruby lib files are tested using common RSpec syntax, and they are located in spec/unit/libraries.

Custom resource tests

The custom resources are tested using the ChefSpec syntax, similar to RSpec, but they need some extra information to work:

  1. The actual tests are located in spec/unit/resource-like folders. Each resource has a folder, and, inside these folders, there is one test file for each custom resource action (or use case, if necessary).
  2. The fixtures are fake cookbooks bound to a hardware variant and API version that contain fixture recipes that uniquely execute one action for a specific resource. They are located in spec/fixtures.
  3. Each test file has at least one corresponding fixture recipe. (They may have more than one if testing new features from newer API versions)
  4. Each it clause must execute the fixture recipe, and that is why the code must create custom matchers to execute the Chef run and check if the correct action was triggered.

Creating the custom matchers

The custom matchers need to be created in the library/matchers.rb file. There you must add the action verb of your new action to the action list if it does not exist, and add the full action name to your resource in the matrix.

For instance, if your resource is the oneview_test_resource, and the new action should be called prepare_for_management you will need to:

  1. Add your action verb to the action verb list if it does not exist, in this case it is prepare.
  2. Add an entry for :oneview_test_resource in the oneview association hash.
  3. The new hash entry must be the key to a symbol list containing all the resource actions, so the symbol :prepare_for_management must be added to the :oneview_test_resource entry. After that your custom matcher will be automatically generated named prepare_oneview_test_resource_for_management. (Read the file documentation for more info).

Also the custom matchers need to be tested too. Simply add the new tests for the matchers created in the spec/unit/matchers_spec.rb as soon as they are implemented