Dynamic unit tests when checking components versions - Kitchen

[rshad]

Hi all!

In the actual approach, Wazuh's components versions are being hard-coded in the corresponding unit tests to the latest version on each release. But this way the unit tests will always fail if we want to test old releases.

To fix so, we need to make the unit tests as dynamic as possible, so we first detect which are the component's version and run the tests in correspond to it.

Tasks to Do:

• Study the different alternative solutions.

• Choose the proper one and add the corresponding configuration.

• Verify the changes.

• Note: This description may be updated in the future if needed.

Kr,

Rshad

[rshad]

Hi all!

Approach 1.

To do so and based on inspec/inspec#541 and http://www.hurryupandwait.io/blog/accessing-chef-node-attributes-from-kitchen-tests

we would add a new cookbook with a unique purpose and it's testing. Such cookbook consists of 1 simple recipe with the following task.

ruby_block "Save node attributes" do
  block do
    File.write("/tmp/kitchen_chef_node.json", node.to_json)
  end
end

This way we will save all the corresponding node's variables in a temporary file in the corresponding kitchen instance in /tmp/kitchen_chef_node.json.

Then and in order to load such file as a JSON object, in the testing script, we run

node = json("/tmp/kitchen_chef_node.json").params

[root@ef41351bd727 tmp]# cat kitchen_chef_node.json

{"name":"elastic-centos-kitchen-chef","chef_environment":"development","json_class":"Chef::Node","automatic":{"time":{"timezone":"UTC"},"dmi":{},"virtualization":{"systems":{"vbox":"guest","docker":"guest"},"system":"docker","role":"guest"},"kernel":{"name":"Linux","release":"4.15.0-54-generic","version":"#58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019","machine":"x86_64","processor":"x86_64","os":"GNU/Linux","modules":{"joydev":{"size":"24576","refcount":"0"},"xt_nat":{"size":"16384","refcount":"1"},"xt_tcpudp":{"size":"16384","refcount":"3"},"veth":{"size":"16384","refcount":"0"},"ipt_MASQUERADE":{"size":"16384","refcount":"2"},"nf_nat_masquerade_ipv4":{"size":"16384","refcount":"1"}, .....

---

Approach 2.

Maybe someone thinks that writing the attributes in a temporary file would be an unnecessary step and in some way, it could be avoided. As we know kitchen and from its configuration file kitchen.yml has complete access to the attributes of Chef nodes. And as we also know the command to run the unit tests is also specified in kitchen.yml by the directive verifier. This way we think about a new approach in which we adapt our test scripts to receive arguments, such as Wazuh version, Elasticsearch version, etc ... . Then it could be possible to pass the corresponding attributes values from inside kitchen.yml`.

PENDING FOR IMPLEMENTATION AND TESTING

Kr,

Rshad

[rshad]

Hi all!

We finally selected the Approach 1, and to get it implemented we followed:

1- Added the test cookbook we already described in the previous commit.

2- Include the testing cookbook in the runlist of each suite, as follows:

wazuh-qa/kitchen/wazuh-chef/manager/files/kitchen.yml

Lines 35 to 36 in 20397cd

	run_list:
	- recipe[test::test]

3- Define a Fixture function to save load the file where the attributes were saved.

wazuh-qa/kitchen/wazuh-chef/manager/test/conftest.py

Lines 12 to 14 in 20397cd

	@pytest.fixture
	def node():
	return json.loads(str(test_host.file('/tmp/kitchen_chef_node.json').content_string))

4- For each testing script, we added a function which gets the corresponding variable/s

manager case

wazuh-qa/kitchen/wazuh-chef/manager/test/base/test_wazuh_agent.py

Lines 7 to 11 in 20397cd

	wazuh_version = ""
	@pytest.mark.filterwarnings('ignore')
	def test_load_variables(host,node):
	wazuh_version = node['default']['wazuh-agent']['version']

Then we use it, as follows:

wazuh-qa/kitchen/wazuh-chef/manager/test/base/test_wazuh_agent.py

Line 17 in 20397cd

version = wazuh_version

Kr,

Rshad