Syscheck automated tests: 'check_*' option

[albertomn86]

Working branch
fim-integration-tests-checks

Description

This issue is part of the roadmap in #7.

The check_* tag is used to check the information specified with this attribute.

According to documentation, there are several attributes to enable or disable the required information:

Attribute	Description	Allowed values
check_all	All attributes with the prefix check_ will be activated.	yes, no
check_sum	Check the MD5, SHA-1 and SHA-256 hashes of the files.	yes, no
check_sha1sum	Check only the SHA-1 hash of the files.	yes, no
check_md5sum	Check only the MD5 hash of the files.	yes, no
check_sha256sum	Check only the SHA-256 hash of the files.	yes, no
check_size	Check the size of the files.	yes, no
check_owner	Check the owner of the files.On Windows, uid will always be 0.	yes, no
check_group	Check the group owner of the files/directories. Available for UNIX. On Windows, gid will always be 0 and the group name will be blank.	yes, no
check_perm	Check the permission of the files/directories. On Windows, a list of denied and allowed permissions will be given for each user or group. Only works on NTFS partitions on Windows systems.	yes, no
check_attrs	Check the attributes of the files. Available for Windows.	yes, no
check_mtime	Check the modification time of a file.	yes, no
check_inode	Check the file inode.Available for UNIX. On Windows, inode will always be 0.	yes, no

Subtasks

• Realtime
  • Add, modify and delete file without check_sum option enabled.
  • Add, modify and delete file without check_md5sum option enabled.
  • Add, modify and delete file without check_sha1sum option enabled.
  • Add, modify and delete file without check_sha256sum option enabled.
  • Add, modify and delete file without check_size option enabled.
  • Add, modify and delete file without check_owner option enabled.
  • Add, modify and delete file without check_group option enabled.
  • Add, modify and delete file without check_perm option enabled.
  • Add, modify and delete file without check_mtime option enabled.
  • Add, modify and delete file without check_inode option enabled.
  • Add, modify and delete file with only check_sum option enabled.
  • Add, modify and delete file with only check_md5sum option enabled.
  • Add, modify and delete file with only check_sha1sum option enabled.
  • Add, modify and delete file with only check_sha256sum option enabled.
  • Add, modify and delete file with only check_size option enabled.
  • Add, modify and delete file with only check_owner option enabled.
  • Add, modify and delete file with only check_group option enabled.
  • Add, modify and delete file with only check_perm option enabled.
  • Add, modify and delete file with only check_mtime option enabled.
  • Add, modify and delete file with only check_inode option enabled.
• Whodata
  • Add, modify and delete file without check_sum option enabled.
  • Add, modify and delete file without check_md5sum option enabled.
  • Add, modify and delete file without check_sha1sum option enabled.
  • Add, modify and delete file without check_sha256sum option enabled.
  • Add, modify and delete file without check_size option enabled.
  • Add, modify and delete file without check_owner option enabled.
  • Add, modify and delete file without check_group option enabled.
  • Add, modify and delete file without check_perm option enabled.
  • Add, modify and delete file without check_mtime option enabled.
  • Add, modify and delete file without check_inode option enabled.
  • Add, modify and delete file with only check_sum option enabled.
  • Add, modify and delete file with only check_md5sum option enabled.
  • Add, modify and delete file with only check_sha1sum option enabled.
  • Add, modify and delete file with only check_sha256sum option enabled.
  • Add, modify and delete file with only check_size option enabled.
  • Add, modify and delete file with only check_owner option enabled.
  • Add, modify and delete file with only check_group option enabled.
  • Add, modify and delete file with only check_perm option enabled.
  • Add, modify and delete file with only check_mtime option enabled.
  • Add, modify and delete file with only check_inode option enabled.
• Scheduled
  • Add, modify and delete file without check_sum option enabled.
  • Add, modify and delete file without check_md5sum option enabled.
  • Add, modify and delete file without check_sha1sum option enabled.
  • Add, modify and delete file without check_sha256sum option enabled.
  • Add, modify and delete file without check_size option enabled.
  • Add, modify and delete file without check_owner option enabled.
  • Add, modify and delete file without check_group option enabled.
  • Add, modify and delete file without check_perm option enabled.
  • Add, modify and delete file without check_mtime option enabled.
  • Add, modify and delete file without check_inode option enabled.
  • Add, modify and delete file with only check_sum option enabled.
  • Add, modify and delete file with only check_md5sum option enabled.
  • Add, modify and delete file with only check_sha1sum option enabled.
  • Add, modify and delete file with only check_sha256sum option enabled.
  • Add, modify and delete file with only check_size option enabled.
  • Add, modify and delete file with only check_owner option enabled.
  • Add, modify and delete file with only check_group option enabled.
  • Add, modify and delete file with only check_perm option enabled.
  • Add, modify and delete file with only check_mtime option enabled.
  • Add, modify and delete file with only check_inode option enabled.

[CarlosRS9]

Summary

All test with check_all option are currently running for scheduled, whodata and realtime mode. New tests without this option has been proposed and are under development.
Several optimization has been done to improve code quality and performance.

Pending

• Finish the new proposed tests.

[CarlosRS9]

Summary

New proposed tests for all options except inode has been implemented and are working as expected. Inode tests are still pending due to the dificulty of designing tests for such functionality.

More tests will be created to properly test the behaviour of syscheck changing the order of the parameters and their combinations.

An exhaustive analysis is being carried out to propose new tests.

Difficulties

• Desingn and implements tests to properly check the inode functionality.

Pending

• inode tests for Realtime, Whodata and Scheduled.
• More tests for the rest of the options to check the behaviour changing the order in which they appear.

[CarlosRS9]

Summary

A new set of tests has been implemented successfully. Some strange failures with the whodata part are happening. They looks like random errors. Everything is fine with Realtime and Scheduled part.
An exhaustive analysis over inode tests is being carried out to verify if its possible to test it.

Difficulties

• Design and implements tests to properly check the inode functionality.

Pending

• inode tests for Realtime, Whodata and Scheduled.
• Find the root cause of the random whodata failures and fix/report it.

[CarlosRS9]

Summary

inode tests were implemented and they are working as expected. All tests all running except for some whodata failures not related to the tests. Core Team are working on fixing those whodata issues.

Tests

This is the current status of the tests:

============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.2.1, py-1.8.0, pluggy-0.13.0
rootdir: /vagrant/wazuh-qa/test_wazuh, inifile: pytest.ini
collected 366 items

test_check_all.py ..........ssssssssssssssssss..................ssssssss [ 14%]
ssssssssss.......FFFFFFFFFFFssssssssssssssssssFFFFFFFF                   [ 29%]
test_check_others.py ..........ssssssssssssssssss..................sssss [ 43%]
sssssssssssss................FFssssssssssssssssss...F...F                [ 59%]
test_checksums.py .............sssssssssssssssssssssssss............FFFF [ 73%]
FFF......sssssssssssssssssssssssss............FFFFFFFFFFFFFsssssssssssss [ 93%]
ssssssssssss............                                                 [100%]
===== 43 failed, 140 passed, 183 skipped in 8565721.18s (99 days, 3:22:01) =====

Note that the failed tests are due to a under investigation issue with whodata and audit.