Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SCA Global Testing #97

Closed
90 of 94 tasks
Skeptor opened this issue Jul 24, 2019 · 23 comments
Closed
90 of 94 tasks

SCA Global Testing #97

Skeptor opened this issue Jul 24, 2019 · 23 comments
Assignees
@cristgl

Comments

@Skeptor
Copy link

Skeptor commented Jul 24, 2019 •
edited

SCA Memory test

Version Branch
3.10.0 3249-refactor-SCA-internal-logic-engine

These tests assess that there are no memory leaks in the PR: wazuh/wazuh#3286

It also reflects the results of static code analyzers and tests of the memory and CPU use.

  • Valgrind
    • Execute almost every part of the SCA code. This will be done with test policies.
    • Remove check between scans.
    • Remove policy file between scans.
    • Add check between scans.
    • Add policy with invalid YAML format.
    • Launch every policy (including test policies, specific, generic, applications, etc).
    • Run a considerable number of scans.
    • Run a scan with disordered checks.
    • Revise that in the previous tests the database has the correct information.
  • scan-build
    • Manager
    • Debian agent
  • mtrace
  • libleak. This tool only reports allocations as leaks if they haven't been released before a given timeout expires. It does not give any conclusive output, it won't be used for this test.
  • AddressSanitizer
    • Execute almost every part of the SCA code. This will be done with test policies.
    • Remove check between scans.
    • Remove policy file between scans.
    • Add check between scans.
    • Repeated checks.
    • Add policy with invalid YAML format.
    • Launch every policy (including test policies, specific, generic, applications, etc).
    • Run a scan with disordered checks.
    • Run a considerable number of scans.
    • Revise that in the previous tests the database has the correct information.
  • Dr. Memory on Windows
  • Dr. Memory on Linux
  • Leaks on MacOS
  • Tests developed at https://github.com/wazuh/wazuh-qa/tree/add-test-for-sca

Functional tests

LINUX

  • Fresh installation. Check the correct policies for each OS are installed at ruleset/sca. In case of the manager, check that it has every policy file with the .disabled suffix except for its specific files.
  • Try to run a policy whose requirements fail.
  • Set an absolute path for a policy out of the default directory.
  • Run scans for every available policy.
  • The subsequent scans just send differences and summaries.
  • If the policy file has changed, it sends the whole scan.
  • When restarting, the whole scan should be shown at the manager without any repeated alert.
  • Disable policies with .disabled or enable="no" at the configuration file.
  • Disable a policy file in a scan subsequent to the first one.
  • Enable a policy with the suffix .disabled, it shouldn't be run.
  • Enable a policy with <policy>cis_debian9_L1.yml.disabled</policy>, for example. It should be enabled.
  • Send an SCA configuration to the agents through agent.conf. Notice .disabled files should be disabled always except when they have been set with their complete name (including the suffix) at the configuration, i.e.<policy>cis_debian9_L1.yml.disabled</policy>.

Check first scan results:

  • The DB has been filled with all the information.
  • Alerts about new checks and summaries appear in the alerts.log and alerts.json.
  • The API returns information about policies and checks.
  • See alerts on the Kibana Discover and information about the whole scan in the Conf. Assessment tab for the agent.

Check subsequent scans restults:

  • The DB is updated.
  • Alert about the different state of a check and summary alert (when any difference)
  • When no differences between scans: Check that no alerts are triggered.
  • The Configuration Assessment tab is updated.

Check results when a policy is disabled:

  • The DB must purge the policy information and its checks.
  • No alerts about the disabled policy.
  • The Configuration Assessment tab is updated.

Check the configuration on demand:

  • API call.
  • Wazuh App.

Integrity:

  • Check the scan results are resend in a random interval between 5 seconds and sca.request_db_interval when the integrity check fails.
  • No alerts about a resend are shown.
  • Check the database is updated correctly after a recovery.

Upgrade:

  • Update from 3.9 to 3.10 and run an old policy file from 3.9 (considering that its information is stored at the database).
  • Check that old policies with old syntax are shown as outdated at the ossec.log file.
  • A warning message appears when Rootcheck is configured for policy monitoring.
  • Wazuh DB must upgrade agent databases including the new tables for Configuration Assessment.
  • Check the correct policies for each OS are installed at ruleset/sca.

WINDOWS

  • Fresh installation. Check the correct policies for each OS are installed at ruleset/sca. In case of the manager, check that it has every policy file with the .disabled suffix except for its specific files.
  • Try to run a policy whose requirements fail.
  • Set an absolute path for a policy out of the default directory.
  • Run scans for every available policy.
  • The subsequent scans just send differences and summaries.
  • If the policy file has changed, it sends the whole scan.
  • When restarting, the whole scan should be shown at the manager without any repeated alert.
  • Disable policies with .disabled or enable="no" at the configuration file.
  • Disable a policy file in a scan subsequent to the first one.
  • Enable a policy with the suffix .disabled, it shouldn't be run.
  • Enable a policy with <policy>cis_debian9_L1.yml.disabled</policy>, for example. It should be enabled.
  • Send an SCA configuration to the agents through agent.conf. Notice .disabled files should be disabled always except when they have been set with their complete name (including the suffix) at the configuration, i.e.<policy>cis_debian9_L1.yml.disabled</policy>.

Check first scan results:

  • The DB has been filled with all the information.
  • Alerts about new checks and summaries appear in the alerts.log and alerts.json.
  • The API returns information about policies and checks.
  • See alerts on the Kibana Discover and information about the whole scan in the Conf. Assessment tab for the agent.

Check subsequent scans restults:

  • The DB is updated.
  • Alert about the different state of a check and summary alert (when any difference)
  • When no differences between scans: Check that no alerts are triggered.
  • The Configuration Assessment tab is updated.

Check results when a policy is disabled:

  • The DB must purge the policy information and its checks.
  • No alerts about the disabled policy.
  • The Configuration Assessment tab is updated.

Check the configuration on demand:

  • API call.
  • Wazuh App.

Integrity:

  • Check the scan results are resend in a random interval between 5 seconds and sca.request_db_interval when the integrity check fails.
  • No alerts about a resend are shown.
  • Check the database is updated correctly after a recovery.

Upgrade:

  • Update from 3.9 to 3.10 and run an old policy file from 3.9 (considering that its information is stored at the database).
  • Check that old policies with old syntax are shown as outdated at the ossec.log file.
  • Wazuh DB must upgrade agent databases including the new tables for Configuration Assessment.
  • Check the correct policies for each OS are installed at ruleset/sca.
@cristgl cristgl self-assigned this Jul 25, 2019
@cristgl
Copy link
Contributor

cristgl commented Jul 25, 2019 •
edited

Valgrind reports

Executing SCA module with test policies.

OUTDATED. FIXED IN: #97 (comment)

wazuh-modulesd

==25992== HEAP SUMMARY:
==25992==     in use at exit: 656,444 bytes in 3,878 blocks
==25992==   total heap usage: 599,497 allocs, 595,619 frees, 781,553,673 bytes allocated
==25992== 
==25992== 272 bytes in 1 blocks are possibly lost in loss record 75 of 120
==25992==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25992==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==25992==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==25992==    by 0x5871227: allocate_stack (allocatestack.c:627)
==25992==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==25992==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==25992==    by 0x13C705: CreateThread (pthreads_op.c:62)
==25992==    by 0x112310: main (main.c:102)
==25992== 
==25992== 1,904 bytes in 7 blocks are possibly lost in loss record 96 of 120
==25992==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25992==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==25992==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==25992==    by 0x5871227: allocate_stack (allocatestack.c:627)
==25992==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==25992==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==25992==    by 0x11226F: main (main.c:95)
==25992== 
==25992== 2,021 bytes in 69 blocks are definitely lost in loss record 97 of 120
==25992==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25992==    by 0x5B259B9: strdup (strdup.c:42)
==25992==    by 0x16A01C: wm_sca_check_policy (wm_sca.c:786)
==25992==    by 0x1688E0: wm_sca_read_files (wm_sca.c:473)
==25992==    by 0x1682AB: wm_sca_start (wm_sca.c:352)
==25992==    by 0x167C6C: wm_sca_main (wm_sca.c:233)
==25992==    by 0x58706DA: start_thread (pthread_create.c:463)
==25992==    by 0x5BA988E: clone (clone.S:95)
==25992== 
==25992== 4,104 bytes in 1 blocks are possibly lost in loss record 104 of 120
==25992==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25992==    by 0x50FFDAA: sqlite3MemMalloc (sqlite3.c:20790)
==25992==    by 0x51007A2: mallocWithAlarm (sqlite3.c:24464)
==25992==    by 0x510084B: sqlite3Malloc (sqlite3.c:24494)
==25992==    by 0x510EC36: pcache1Alloc (sqlite3.c:45336)
==25992==    by 0x510EF24: sqlite3PageMalloc (sqlite3.c:45479)
==25992==    by 0x511F55F: allocateTempSpace (sqlite3.c:61560)
==25992==    by 0x51218CA: btreeCursor (sqlite3.c:63234)
==25992==    by 0x5121A3C: sqlite3BtreeCursor (sqlite3.c:63276)
==25992==    by 0x513D44F: sqlite3VdbeExec (sqlite3.c:81807)
==25992==    by 0x5136036: sqlite3Step (sqlite3.c:76693)
==25992==    by 0x51361E4: sqlite3_step (sqlite3.c:76754)
==25992== 
==25992== 16,226 bytes in 680 blocks are definitely lost in loss record 113 of 120
==25992==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==25992==    by 0x5B259B9: strdup (strdup.c:42)
==25992==    by 0x16A18E: wm_sca_check_policy (wm_sca.c:794)
==25992==    by 0x1688E0: wm_sca_read_files (wm_sca.c:473)
==25992==    by 0x1682AB: wm_sca_start (wm_sca.c:352)
==25992==    by 0x167C6C: wm_sca_main (wm_sca.c:233)
==25992==    by 0x58706DA: start_thread (pthread_create.c:463)
==25992==    by 0x5BA988E: clone (clone.S:95)
==25992== 
==25992== LEAK SUMMARY:
==25992==    definitely lost: 18,247 bytes in 749 blocks
==25992==    indirectly lost: 0 bytes in 0 blocks
==25992==      possibly lost: 6,280 bytes in 9 blocks
==25992==    still reachable: 631,917 bytes in 3,120 blocks
==25992==                       of which reachable via heuristic:
==25992==                         length64           : 223,896 bytes in 124 blocks
==25992==         suppressed: 0 bytes in 0 blocks
==25992== Reachable blocks (those to which a pointer was found) are not shown.
==25992== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==25992== 
==25992== For counts of detected and suppressed errors, rerun with: -v
==25992== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0)

wazuh-db

==26310== HEAP SUMMARY:
==26310==     in use at exit: 8,616 bytes in 3 blocks
==26310==   total heap usage: 6,820 allocs, 6,817 frees, 3,779,460 bytes allocated
==26310== 
==26310== LEAK SUMMARY:
==26310==    definitely lost: 0 bytes in 0 blocks
==26310==    indirectly lost: 0 bytes in 0 blocks
==26310==      possibly lost: 272 bytes in 1 blocks
==26310==    still reachable: 8,344 bytes in 2 blocks
==26310==         suppressed: 0 bytes in 0 blocks
==26310== Rerun with --leak-check=full to see details of leaked memory
==26310== 
==26310== For counts of detected and suppressed errors, rerun with: -v
==26310== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

ossec-analysisd

==3244== HEAP SUMMARY:
==3244==     in use at exit: 12,184,373 bytes in 57,542 blocks
==3244==   total heap usage: 404,722 allocs, 347,180 frees, 435,580,154 bytes allocated
==3244== 
==3244== 12 bytes in 2 blocks are definitely lost in loss record 50 of 510
==3244==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x5B259B9: strdup (strdup.c:42)
==3244==    by 0x171DBA: Read_Rules (rules-config.c:202)
==3244==    by 0x1632A9: read_main_elements (config.c:95)
==3244==    by 0x163F6A: ReadConfig (config.c:245)
==3244==    by 0x11A7CC: GlobalConf (config.c:99)
==3244==    by 0x1314C7: main (analysisd.c:382)
==3244== 
==3244== 12 bytes in 2 blocks are definitely lost in loss record 51 of 510
==3244==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x5B259B9: strdup (strdup.c:42)
==3244==    by 0x1722B7: Read_Rules (rules-config.c:232)
==3244==    by 0x1632A9: read_main_elements (config.c:95)
==3244==    by 0x163F6A: ReadConfig (config.c:245)
==3244==    by 0x11A7CC: GlobalConf (config.c:99)
==3244==    by 0x1314C7: main (analysisd.c:382)
==3244== 
==3244== 22 bytes in 1 blocks are definitely lost in loss record 99 of 510
==3244==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x5B259B9: strdup (strdup.c:42)
==3244==    by 0x1716F1: Read_Rules (rules-config.c:171)
==3244==    by 0x1632A9: read_main_elements (config.c:95)
==3244==    by 0x163F6A: ReadConfig (config.c:245)
==3244==    by 0x11A7CC: GlobalConf (config.c:99)
==3244==    by 0x1314C7: main (analysisd.c:382)
==3244== 
==3244== 24 bytes in 2 blocks are definitely lost in loss record 111 of 510
==3244==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x5B259B9: strdup (strdup.c:42)
==3244==    by 0x171FD5: Read_Rules (rules-config.c:216)
==3244==    by 0x1632A9: read_main_elements (config.c:95)
==3244==    by 0x163F6A: ReadConfig (config.c:245)
==3244==    by 0x11A7CC: GlobalConf (config.c:99)
==3244==    by 0x1314C7: main (analysisd.c:382)
==3244== 
==3244== 30 bytes in 2 blocks are definitely lost in loss record 119 of 510
==3244==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x5B259B9: strdup (strdup.c:42)
==3244==    by 0x171AD8: Read_Rules (rules-config.c:186)
==3244==    by 0x1632A9: read_main_elements (config.c:95)
==3244==    by 0x163F6A: ReadConfig (config.c:245)
==3244==    by 0x11A7CC: GlobalConf (config.c:99)
==3244==    by 0x1314C7: main (analysisd.c:382)
==3244== 
==3244== 33 bytes in 1 blocks are definitely lost in loss record 123 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x122CDE: loadmemory (rules.c:1485)
==3244==    by 0x11D14D: Rules_OP_ReadRules (rules.c:475)
==3244==    by 0x131A85: main (analysisd.c:570)
==3244== 
==3244== 104 bytes in 1 blocks are definitely lost in loss record 217 of 510
==3244==    at 0x4C31D2F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x122E19: loadmemory (rules.c:1507)
==3244==    by 0x11D079: Rules_OP_ReadRules (rules.c:464)
==3244==    by 0x131A85: main (analysisd.c:570)
==3244== 
==3244== 148 (48 direct, 100 indirect) bytes in 1 blocks are definitely lost in loss record 251 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x122698: Rules_OP_ReadRules (rules.c:1321)
==3244==    by 0x131A85: main (analysisd.c:570)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 293 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x131FDD: main (analysisd.c:705)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 294 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x154096: SecurityConfigurationAssessmentInit (security_configuration_assessment.c:72)
==3244==    by 0x13205C: OS_ReadMSG (analysisd.c:747)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 295 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13261E: OS_ReadMSG (analysisd.c:887)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 296 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13263D: OS_ReadMSG (analysisd.c:890)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 297 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13265C: OS_ReadMSG (analysisd.c:893)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 298 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13267B: OS_ReadMSG (analysisd.c:896)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 299 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13269A: OS_ReadMSG (analysisd.c:899)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 300 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x1326B9: OS_ReadMSG (analysisd.c:902)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 301 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x1326D8: OS_ReadMSG (analysisd.c:905)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 272 bytes in 1 blocks are possibly lost in loss record 302 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x1328A0: OS_ReadMSG (analysisd.c:948)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 2,546 (472 direct, 2,074 indirect) bytes in 1 blocks are definitely lost in loss record 387 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x122FCF: zerorulemember (rules.c:1547)
==3244==    by 0x11C543: Rules_OP_ReadRules (rules.c:299)
==3244==    by 0x131A85: main (analysisd.c:570)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 400 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x132700: OS_ReadMSG (analysisd.c:909)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 401 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x132734: OS_ReadMSG (analysisd.c:914)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 402 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x132768: OS_ReadMSG (analysisd.c:919)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 403 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13279C: OS_ReadMSG (analysisd.c:924)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 404 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x1327D0: OS_ReadMSG (analysisd.c:929)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 405 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x132804: OS_ReadMSG (analysisd.c:934)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 406 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x13283B: OS_ReadMSG (analysisd.c:939)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== 3,264 bytes in 12 blocks are possibly lost in loss record 407 of 510
==3244==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3244==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==3244==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==3244==    by 0x5871227: allocate_stack (allocatestack.c:627)
==3244==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==3244==    by 0x1DEE08: CreateThreadJoinable (pthreads_op.c:47)
==3244==    by 0x1DEEAF: CreateThread (pthreads_op.c:62)
==3244==    by 0x132872: OS_ReadMSG (analysisd.c:944)
==3244==    by 0x131FF8: main (analysisd.c:708)
==3244== 
==3244== LEAK SUMMARY:
==3244==    definitely lost: 757 bytes in 13 blocks
==3244==    indirectly lost: 2,174 bytes in 7 blocks
==3244==      possibly lost: 28,832 bytes in 106 blocks
==3244==    still reachable: 12,152,610 bytes in 57,416 blocks
==3244==         suppressed: 0 bytes in 0 blocks
==3244== Reachable blocks (those to which a pointer was found) are not shown.
==3244== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3244== 
==3244== For counts of detected and suppressed errors, rerun with: -v
==3244== ERROR SUMMARY: 27 errors from 27 contexts (suppressed: 0 from 0)

@Skeptor
Copy link
Author

Skeptor commented Jul 25, 2019 •
edited

There was an error introduced merging 3.10 in the branch. The commit wazuh/wazuh@2b6f8f6 solves the error.

This is the new Valgrind report of wazuh-modulesd:

==15340== LEAK SUMMARY:
==15340==    definitely lost: 0 bytes in 0 blocks
==15340==    indirectly lost: 0 bytes in 0 blocks
==15340==      possibly lost: 6,280 bytes in 9 blocks
==15340==    still reachable: 1,407,980 bytes in 20,418 blocks
==15340==                       of which reachable via heuristic:
==15340==                         length64           : 223,848 bytes in 121 blocks
==15340==         suppressed: 0 bytes in 0 blocks
==15340== Reachable blocks (those to which a pointer was found) are not shown.
==15340== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==15340==

@Skeptor
Copy link
Author

Skeptor commented Jul 25, 2019 •
edited

Scan build

Done building server

scan-build: Removing directory '/tmp/scan-build-2019-07-25-154745-22303-1' because it contains no reports.
scan-build: No bugs found.

Done building agent

scan-build: Removing directory '/tmp/scan-build-2019-07-25-163919-9908-1' because it contains no reports.
scan-build: No bugs found.

@cristgl
Copy link
Contributor

cristgl commented Jul 25, 2019 •
edited

When launching a scan with an invalid YAML file, Valgrind reports the next summary:

OUTDATED. FIXED IN #97 (comment)

==14626== HEAP SUMMARY:
==14626==     in use at exit: 635,764 bytes in 2,659 blocks
==14626==   total heap usage: 1,382,226 allocs, 1,379,567 frees, 1,569,103,953 bytes allocated
==14626== 
==14626== 89 bytes in 1 blocks are possibly lost in loss record 53 of 126
==14626==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x4012C94: allocate_dtv_entry (dl-tls.c:582)
==14626==    by 0x4012C94: allocate_and_init (dl-tls.c:607)
==14626==    by 0x4012C94: tls_get_addr_tail (dl-tls.c:787)
==14626==    by 0x4019A27: __tls_get_addr (tls_get_addr.S:55)
==14626==    by 0xDAF8FF1: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0xDAFEA35: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0xDAF94B2: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0xDB0E48E: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0xDB0E8CB: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0xDAF3B56: _nss_systemd_getpwuid_r (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==14626==    by 0x5B6BD34: getpwuid_r@@GLIBC_2.2.5 (getXXbyYY_r.c:315)
==14626==    by 0x5B6B497: getpwuid (getXXbyYY.c:135)
==14626==    by 0x5234729: user_from_uid (pwcache.c:42)
==14626== 
==14626== 272 bytes in 1 blocks are possibly lost in loss record 76 of 126
==14626==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==14626==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==14626==    by 0x5871227: allocate_stack (allocatestack.c:627)
==14626==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==14626==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==14626==    by 0x13C705: CreateThread (pthreads_op.c:62)
==14626==    by 0x167C31: wm_sca_main (wm_sca.c:220)
==14626==    by 0x58706DA: start_thread (pthread_create.c:463)
==14626==    by 0x5BA988E: clone (clone.S:95)
==14626== 
==14626== 272 bytes in 1 blocks are possibly lost in loss record 77 of 126
==14626==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==14626==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==14626==    by 0x5871227: allocate_stack (allocatestack.c:627)
==14626==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==14626==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==14626==    by 0x13C705: CreateThread (pthreads_op.c:62)
==14626==    by 0x112310: main (main.c:102)
==14626== 
==14626== 1,904 bytes in 7 blocks are possibly lost in loss record 104 of 126
==14626==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==14626==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==14626==    by 0x5871227: allocate_stack (allocatestack.c:627)
==14626==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==14626==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==14626==    by 0x11226F: main (main.c:95)
==14626== 
==14626== 3,072 bytes in 2 blocks are definitely lost in loss record 109 of 126
==14626==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x51A5D88: yaml_malloc (in /var/ossec/lib/libwazuhext.so)
==14626==    by 0x51C21B5: yaml_parser_load (in /var/ossec/lib/libwazuhext.so)
==14626==    by 0x195585: yaml_parse_file (yaml2json.c:46)
==14626==    by 0x16877B: wm_sca_read_files (wm_sca.c:454)
==14626==    by 0x1682AB: wm_sca_start (wm_sca.c:352)
==14626==    by 0x167C6C: wm_sca_main (wm_sca.c:233)
==14626==    by 0x58706DA: start_thread (pthread_create.c:463)
==14626==    by 0x5BA988E: clone (clone.S:95)
==14626== 
==14626== 4,104 bytes in 1 blocks are possibly lost in loss record 114 of 126
==14626==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==14626==    by 0x50FFDAA: sqlite3MemMalloc (sqlite3.c:20790)
==14626==    by 0x51007A2: mallocWithAlarm (sqlite3.c:24464)
==14626==    by 0x510084B: sqlite3Malloc (sqlite3.c:24494)
==14626==    by 0x510EC36: pcache1Alloc (sqlite3.c:45336)
==14626==    by 0x510EF24: sqlite3PageMalloc (sqlite3.c:45479)
==14626==    by 0x511F55F: allocateTempSpace (sqlite3.c:61560)
==14626==    by 0x51218CA: btreeCursor (sqlite3.c:63234)
==14626==    by 0x5121A3C: sqlite3BtreeCursor (sqlite3.c:63276)
==14626==    by 0x513D44F: sqlite3VdbeExec (sqlite3.c:81807)
==14626==    by 0x5136036: sqlite3Step (sqlite3.c:76693)
==14626==    by 0x51361E4: sqlite3_step (sqlite3.c:76754)
==14626== 
==14626== LEAK SUMMARY:
==14626==    definitely lost: 3,072 bytes in 2 blocks
==14626==    indirectly lost: 0 bytes in 0 blocks
==14626==      possibly lost: 6,641 bytes in 11 blocks
==14626==    still reachable: 626,051 bytes in 2,646 blocks
==14626==                       of which reachable via heuristic:
==14626==                         length64           : 223,848 bytes in 121 blocks
==14626==         suppressed: 0 bytes in 0 blocks
==14626== Reachable blocks (those to which a pointer was found) are not shown.
==14626== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==14626== 
==14626== For counts of detected and suppressed errors, rerun with: -v
==14626== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0)

@cristgl
Copy link
Contributor

cristgl commented Jul 25, 2019 •
edited

Fixed memory leak when parsing an invalid YAML file. Solved in this commit.

==27920== HEAP SUMMARY:
==27920==     in use at exit: 666,599 bytes in 3,911 blocks
==27920==   total heap usage: 866,406 allocs, 862,495 frees, 860,048,923 bytes allocated
==27920== 
==27920== 89 bytes in 1 blocks are possibly lost in loss record 53 of 124
==27920==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27920==    by 0x4012C94: allocate_dtv_entry (dl-tls.c:582)
==27920==    by 0x4012C94: allocate_and_init (dl-tls.c:607)
==27920==    by 0x4012C94: tls_get_addr_tail (dl-tls.c:787)
==27920==    by 0x4019A27: __tls_get_addr (tls_get_addr.S:55)
==27920==    by 0xDAF8FF1: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0xDAFEA35: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0xDAF94B2: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0xDB0E48E: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0xDB0E8CB: ??? (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0xDAF3B56: _nss_systemd_getpwuid_r (in /lib/x86_64-linux-gnu/libnss_systemd.so.2)
==27920==    by 0x5B6BD34: getpwuid_r@@GLIBC_2.2.5 (getXXbyYY_r.c:315)
==27920==    by 0x5B6B497: getpwuid (getXXbyYY.c:135)
==27920==    by 0x5234729: user_from_uid (pwcache.c:42)
==27920== 
==27920== 272 bytes in 1 blocks are possibly lost in loss record 77 of 124
==27920==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27920==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==27920==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==27920==    by 0x5871227: allocate_stack (allocatestack.c:627)
==27920==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==27920==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==27920==    by 0x13C705: CreateThread (pthreads_op.c:62)
==27920==    by 0x112310: main (main.c:102)
==27920== 
==27920== 2,176 bytes in 8 blocks are possibly lost in loss record 103 of 124
==27920==    at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27920==    by 0x40134A6: allocate_dtv (dl-tls.c:286)
==27920==    by 0x40134A6: _dl_allocate_tls (dl-tls.c:530)
==27920==    by 0x5871227: allocate_stack (allocatestack.c:627)
==27920==    by 0x5871227: pthread_create@@GLIBC_2.2.5 (pthread_create.c:644)
==27920==    by 0x13C65E: CreateThreadJoinable (pthreads_op.c:47)
==27920==    by 0x11226F: main (main.c:95)
==27920== 
==27920== 4,104 bytes in 1 blocks are possibly lost in loss record 109 of 124
==27920==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27920==    by 0x50FFDAA: sqlite3MemMalloc (sqlite3.c:20790)
==27920==    by 0x51007A2: mallocWithAlarm (sqlite3.c:24464)
==27920==    by 0x510084B: sqlite3Malloc (sqlite3.c:24494)
==27920==    by 0x510EC36: pcache1Alloc (sqlite3.c:45336)
==27920==    by 0x510EF24: sqlite3PageMalloc (sqlite3.c:45479)
==27920==    by 0x511F55F: allocateTempSpace (sqlite3.c:61560)
==27920==    by 0x51218CA: btreeCursor (sqlite3.c:63234)
==27920==    by 0x5121A3C: sqlite3BtreeCursor (sqlite3.c:63276)
==27920==    by 0x513D44F: sqlite3VdbeExec (sqlite3.c:81807)
==27920==    by 0x5136036: sqlite3Step (sqlite3.c:76693)
==27920==    by 0x51361E4: sqlite3_step (sqlite3.c:76754)
==27920== 
==27920== LEAK SUMMARY:
==27920==    definitely lost: 0 bytes in 0 blocks
==27920==    indirectly lost: 0 bytes in 0 blocks
==27920==      possibly lost: 6,641 bytes in 11 blocks
==27920==    still reachable: 659,958 bytes in 3,900 blocks
==27920==                       of which reachable via heuristic:
==27920==                         length64           : 223,848 bytes in 121 blocks
==27920==         suppressed: 0 bytes in 0 blocks
==27920== Reachable blocks (those to which a pointer was found) are not shown.
==27920== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==27920== 
==27920== For counts of detected and suppressed errors, rerun with: -v
==27920== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)

@cristgl
Copy link
Contributor

cristgl commented Jul 29, 2019 •
edited

OUTDATED. FIXED IN wazuh/wazuh@bc1222d

There was a problem when trying to check the integrity from file sca_win_registry_test_suite.yml.

SCA module:

results: passed:passed:passed:passed:passed:passed:passed:failed:failed:failed:failed:failed:failed:failed::
hash: 8e5a1b9f0f2b4c2730531413cec8a0d4f0ae1e68d2eb7721c5858203343daddc

DB:

results: passed:passed:passed:failed:failed:failed:passed:passed:passed:passed:failed:failed:failed:failed::
hash: b7263a6adefa09232beb380b52b655dd10be78fb6f6f2c05ac7bb50db116dd74

The database orders by ID the results obtained, but the agent doesn't. This leads to an infinite loop of integrity checksum failures.

For example, if the YAML policy has the IDs 1,0,2,3,4, and the results are failed:passed:passed:passed:failed, the database will store them ordered by ID as 0,1,2,3,4, passed:failed:passed:passed:failed.

@cristgl
Copy link
Contributor

cristgl commented Jul 29, 2019

Dr. Memory report:

Dr. Memory version 2.1.0 build 1 built on Mar 18 2019 00:02:08
Windows version: WinVer=63;Rel=;Build=9600;Edition=Professional
Dr. Memory results for pid 4728: "ossec-agent.exe"
Application cmdline: ".\ossec-agent.exe start"
Recorded 117 suppression(s) from default C:\Program Files (x86)\Dr. Memory\bin\suppress-default.txt

@cristgl
Copy link
Contributor

cristgl commented Jul 31, 2019 •
edited

Adding a check which runs the command yes:

- id: 300107
   title: INVALID -- Non executable file (will trigger leak reports due to execvp failing)
   condition: any
   rules:
     - c:yes ram -> r:ram

Generates an error of memory allocation:

==3290==ERROR: AddressSanitizer failed to allocate 0x20000 (131072) bytes at address 6320005d0000 (errno: 12)
==3290==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cc:184 "(("unable to mmap" && 0)) != (0)" (0x0, 0x0)
==3290==WARNING: readlink("/proc/self/exe") failed with errno 2, some stack frames may not be symbolized
    #0 0x7f6b5bfbf631  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa0631)
    #1 0x7f6b5bfc45e3  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa55e3)
    #2 0x7f6b5bfcc1df  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xad1df)
    #3 0x7f6b5bf4334b  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x2434b)
    #4 0x7f6b5bf43897  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x24897)
    #5 0x7f6b5bf438f5  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x248f5)
    #6 0x7f6b5bf4274b  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x2374b)
    #7 0x7f6b5bfb75d2  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x985d2)
    #8 0x7f6b5b9b9b34  (/var/ossec/bin/../lib/libwazuhext.so+0x2c0b34)
    #9 0x7f6b5b9ba52c  (/var/ossec/bin/../lib/libwazuhext.so+0x2c152c)
    #10 0x7f6b5b9ba5d5  (/var/ossec/bin/../lib/libwazuhext.so+0x2c15d5)
    #11 0x7f6b5b9c882d  (/var/ossec/bin/../lib/libwazuhext.so+0x2cf82d)
    #12 0x7f6b5b9c8b5a  (/var/ossec/bin/../lib/libwazuhext.so+0x2cfb5a)
    #13 0x7f6b5b9c95e5  (/var/ossec/bin/../lib/libwazuhext.so+0x2d05e5)
    #14 0x7f6b5b9c975f  (/var/ossec/bin/../lib/libwazuhext.so+0x2d075f)
    #15 0x7f6b5b9c978e  (/var/ossec/bin/../lib/libwazuhext.so+0x2d078e)
    #16 0x7f6b5b9c7b05  (/var/ossec/bin/../lib/libwazuhext.so+0x2ceb05)
    #17 0x7f6b5b9cf1d5  (/var/ossec/bin/../lib/libwazuhext.so+0x2d61d5)
    #18 0x7f6b5b9cf6b4  (/var/ossec/bin/../lib/libwazuhext.so+0x2d66b4)
    #19 0x7f6b5b9d838d  (/var/ossec/bin/../lib/libwazuhext.so+0x2df38d)
    #20 0x7f6b5b9d99fa  (/var/ossec/bin/../lib/libwazuhext.so+0x2e09fa)
    #21 0x7f6b5b9da191  (/var/ossec/bin/../lib/libwazuhext.so+0x2e1191)
    #22 0x7f6b5b9f7390  (/var/ossec/bin/../lib/libwazuhext.so+0x2fe390)
    #23 0x7f6b5b9efdcd  (/var/ossec/bin/../lib/libwazuhext.so+0x2f6dcd)
    #24 0x7f6b5b9eff78  (/var/ossec/bin/../lib/libwazuhext.so+0x2f6f78)
    #25 0x4173c5  (/proc/self/exe+0x4173c5)
    #26 0x417103  (/proc/self/exe+0x417103)
    #27 0x42b020  (/proc/self/exe+0x42b020)
    #28 0x41e7b3  (/proc/self/exe+0x41e7b3)
    #29 0x409030  (/proc/self/exe+0x409030)
    #30 0x7f6b5b4e36b9  (/lib/x86_64-linux-gnu/libpthread.so.0+0x76b9)
    #31 0x7f6b5b21941c  (/lib/x86_64-linux-gnu/libc.so.6+0x10741c)

The timeout doesn't work for this command as it generates an output and the timeout checks whether a command does show output at a certain threshold.

@cristgl
Copy link
Contributor

cristgl commented Aug 2, 2019

Update from 3.9 to 3.10 and run an SCA scan:

Databases are correctly purged, they do not generate any error. With this update, the configuration is still the same, this means that it will show this kind of messages when it has been updated.

2019/08/02 06:26:35 sca: WARNING: Policy file not found: '/var/ossec/ruleset/sca/cis_debian_linux_rcl.yml'. Skipping it.
2019/08/02 06:26:35 sca: WARNING: Policy file not found: '/var/ossec/ruleset/sca/system_audit_rcl.yml'. Skipping it.
2019/08/02 06:26:35 sca: WARNING: Policy file not found: '/var/ossec/ruleset/sca/system_audit_ssh.yml'. Skipping it.
2019/08/02 06:26:35 sca: WARNING: Policy file not found: '/var/ossec/ruleset/sca/system_audit_pw.yml'. Skipping it.

@cristgl
Copy link
Contributor

cristgl commented Aug 2, 2019 •
edited

Agent remote configuration

OUTDATED. FIXED IN #97 (comment)

With an agent having its default SCA configuration:

  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>12h</interval>
    <skip_nfs>yes</skip_nfs>
  </sca>

With the next agent.conf set:

<agent_config>
  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>2m</interval>
    <skip_nfs>yes</skip_nfs>

    <policies>
      <policy enable="no">cis_debian9_L2.yml</policy>
    </policies>
  </sca>
</agent_config>

We obtain the next result:

2019/08/02 11:51:34 sca: INFO: Adding policy file 'cis_debian9_L2.yml' by default.
2019/08/02 11:51:34 sca: INFO: Adding policy file 'cis_debian9_L1.yml' by default.
2019/08/02 11:51:34 sca: INFO: Adding policy file 'cis_debian9_L2.yml' by default.
2019/08/02 11:51:34 sca: INFO: Adding policy file 'cis_debian9_L1.yml' by default.

And after finishing the evaluation of the first two files:

2019/08/02 11:51:40 sca: WARNING: Found duplicated policy ID: cis_debian9_L2. File 'cis_debian9_L2.yml' contains the same ID.
2019/08/02 11:51:40 sca: WARNING: Error found while validating policy file: '/var/ossec/ruleset/sca/cis_debian9_L2.yml'. Skipping it.
2019/08/02 11:51:40 sca: WARNING: Found duplicated policy ID: cis_debian9_L1. File 'cis_debian9_L1.yml' contains the same ID.
2019/08/02 11:51:40 sca: WARNING: Error found while validating policy file: '/var/ossec/ruleset/sca/cis_debian9_L1.yml'. Skipping it.

Also, notice that we have set the policy cis_debian9_L2.yml file to disabled in the remote configuration, but enable instead of enabled, but it does not show any configuration warning or error.
If we set the same configuration but with enabled, the module reads the ossec.conf first, runs the scan and then the remote configuration, which only enables the cis_debian9_L1.yml policy, and shows it as duplicated.

2019/08/05 08:57:28 sca: INFO: Evaluation finished for policy 'cis_debian9_L1.yml'
2019/08/05 08:57:28 sca: INFO: Starting evaluation of policy: 'cis_debian9_L2.yml'
2019/08/05 08:57:30 ossec-logcollector: INFO: Agent is now online. Process unlocked, continuing...
2019/08/05 08:57:31 sca: INFO: Evaluation finished for policy 'cis_debian9_L2.yml'
2019/08/05 08:57:31 sca: WARNING: Found duplicated policy ID: cis_debian9_L1. File 'cis_debian9_L1.yml' contains the same ID.
2019/08/05 08:57:31 sca: WARNING: Error found while validating policy file: '/var/ossec/ruleset/sca/cis_debian9_L1.yml'. Skipping it.

@Skeptor
Copy link
Author

Skeptor commented Aug 5, 2019 •
edited

The behaviour of SCA when a policy is disabled

The Wazuh manager removes the information of a disabled policy when a new scan is received.
This feature works fine while SCA is running.
In the case that the disabled policy is the only monitored policy the SCA module stop working (Expected behaviour). If SCA stops running the manager never receive an event with the new policies. In this case, the information isn't purged.
It will be purged when the manager receives a new SCA event with the policies.

@cristgl
Copy link
Contributor

cristgl commented Aug 5, 2019 •
edited

Checking the correct policies for Windows are installed at ruleset/sca

Every Windows has only sca_win_audit.yml at ruleset/sca, without considering the Windows version. For example, A Windows Server 2012 does not have a cis_win2012r2_memberL1.yml file.

@cristgl
Copy link
Contributor

cristgl commented Aug 6, 2019 •
edited

Agent remote configuration

The problem was that the array of policies to be scanned was adding every file included in the ossec.conf file. Checking if the policy has already been added fixes the issue.

Testing

  • Adding more than one SCA block to the configuration.
  • Sending an SCA remote configuration to an agent.
  • Enabling a policy and disabling it in the remote configuration block. The policy should be disabled.
  • Disabling a policy and enabling it in the remote configuration block. The policy should be enabled.
  • Adding more than once the same policy file. It should be scanned once.
  • Valgrind does not report any error.
==17687== 
==17687== HEAP SUMMARY:
==17687==     in use at exit: 776,126 bytes in 7,087 blocks
==17687==   total heap usage: 157,465 allocs, 150,378 frees, 112,502,675 bytes allocated
==17687== 
==17687== LEAK SUMMARY:
==17687==    definitely lost: 0 bytes in 0 blocks
==17687==    indirectly lost: 0 bytes in 0 blocks
==17687==      possibly lost: 6,280 bytes in 9 blocks
==17687==    still reachable: 769,846 bytes in 7,078 blocks
==17687==                       of which reachable via heuristic:
==17687==                         length64           : 223,896 bytes in 124 blocks
==17687==         suppressed: 0 bytes in 0 blocks
==17687== Rerun with --leak-check=full to see details of leaked memory
==17687== 
==17687== For counts of detected and suppressed errors, rerun with: -v
==17687== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Log sample

Configuration blocks:

  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>60s</interval>
    <skip_nfs>yes</skip_nfs>
    <policies>
        <policy enabled="no">cis_debian9_L2.yml</policy>
    </policies>
  </sca>

  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>12h</interval>
    <skip_nfs>yes</skip_nfs>
    <policies>
      <policy enabled="yes">cis_debian9_L2.yml</policy>
      <policy enabled="yes">cis_debian9_L1.yml</policy>
    </policies>
  </sca>

ossec.log output:

2019/08/06 11:43:44 sca: INFO: Adding policy file 'cis_debian9_L2.yml' by default.
2019/08/06 11:43:44 sca: INFO: Adding policy file 'cis_debian9_L1.yml' by default.
2019/08/06 11:43:44 sca: INFO: Disabling policy 'cis_debian9_L2.yml' by configuration.
2019/08/06 11:43:45 ossec-csyslogd: INFO: Remote syslog server not configured. Clean exit.
2019/08/06 11:43:45 ossec-dbd: INFO: Database not configured. Clean exit.
2019/08/06 11:43:45 ossec-integratord: INFO: Remote integrations not configured. Clean exit.
2019/08/06 11:43:45 ossec-agentlessd: INFO: Not configured. Exiting.
2019/08/06 11:43:45 ossec-authd: INFO: Started (pid: 22677).
2019/08/06 11:43:45 ossec-authd: INFO: Accepting connections on port 1515. No password required.
2019/08/06 11:43:45 ossec-authd: INFO: Setting network timeout to 1.000000 sec.
2019/08/06 11:43:45 wazuh-db: INFO: Started (pid: 22686).
2019/08/06 11:43:45 ossec-execd: INFO: Started (pid: 22704).
2019/08/06 11:43:45 ossec-syscheckd: WARNING: The check_unixaudit option is deprecated in favor of the SCA module.
2019/08/06 11:43:45 rootcheck: ERROR: System audit file not configured.
2019/08/06 11:43:45 ossec-remoted: INFO: Started (pid: 22726). Listening on port 1514/UDP (secure).
2019/08/06 11:43:45 ossec-monitord: INFO: Started (pid: 22738).
2019/08/06 11:43:45 sca: INFO: Adding policy file 'cis_debian9_L2.yml' by default.
2019/08/06 11:43:45 sca: INFO: Adding policy file 'cis_debian9_L1.yml' by default.
2019/08/06 11:43:45 sca: INFO: Disabling policy 'cis_debian9_L2.yml' by configuration.
2019/08/06 11:43:45 wazuh-modulesd: INFO: Process started.
2019/08/06 11:43:45 wazuh-modulesd:oscap: INFO: Module disabled. Exiting...
2019/08/06 11:43:45 wazuh-modulesd:ciscat: INFO: Module disabled. Exiting...
2019/08/06 11:43:45 wazuh-modulesd:osquery: INFO: Module disabled. Exiting...
2019/08/06 11:43:45 wazuh-modulesd:syscollector: INFO: Module disabled. Exiting...
2019/08/06 11:43:45 wazuh-modulesd:database: INFO: Module started.
2019/08/06 11:43:45 wazuh-modulesd:download: INFO: Module started
2019/08/06 11:43:45 sca: INFO: Module started.
2019/08/06 11:43:45 ossec-analysisd: INFO: Total rules enabled: '3395'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/mtab'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/hosts.deny'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/mail/statistics'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/random-seed'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/random.seed'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/adjtime'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/httpd/logs'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/utmpx'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/wtmpx'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/cups/certs'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/dumpdates'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/etc/svc/volatile'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/sys/kernel/security'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/sys/kernel/debug'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '/dev/core'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '^/proc'
2019/08/06 11:43:45 ossec-analysisd: INFO: Ignoring file: '.log$|.swp$'
2019/08/06 11:43:45 ossec-analysisd: INFO: Started (pid: 22712).
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/audit/audit.log'.
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/ossec/logs/active-responses.log'.
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/auth.log'.
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/syslog'.
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/dpkg.log'.
2019/08/06 11:43:46 ossec-logcollector: INFO: (1950): Analyzing file: '/var/log/kern.log'.
2019/08/06 11:43:46 ossec-logcollector: INFO: Monitoring output of command(360): df -P
2019/08/06 11:43:46 ossec-logcollector: INFO: Monitoring full output of command(360): netstat -tulpn | sed 's/\([[:alnum:]]\+\)\ \+[[:digit:]]\+\ \+[[:digit:]]\+\ \+\(.*\):\([[:digit:]]*\)\ \+\([0-9\.\:\*]\+\).\+\ \([[:digit:]]*\/[[:alnum:]\-]*\).*/\1 \2 == \3 == \4 \5/' | sort -k 4 -g | sed 's/ == \(.*\) ==/:\1/' | sed 1,2d
2019/08/06 11:43:46 ossec-logcollector: INFO: Monitoring full output of command(360): last -n 20
2019/08/06 11:43:46 ossec-logcollector: INFO: Started (pid: 22732).
2019/08/06 11:43:46 ossec-remoted: INFO: (4111): Maximum number of agents allowed: '14000'.
2019/08/06 11:43:46 ossec-remoted: INFO: (1410): Reading authentication keys file.
2019/08/06 11:43:46 sca: INFO: Starting Security Configuration Assessment scan.
2019/08/06 11:43:46 sca: INFO: Starting evaluation of policy: 'cis_debian9_L2.yml'
2019/08/06 11:43:49 ossec-syscheckd: INFO: Started (pid: 22718).
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/etc', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/usr/bin', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/usr/sbin', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/bin', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/sbin', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6003): Monitoring directory: '/boot', with options 'perm | size | owner | group | md5sum | sha1sum | sha256sum | mtime | inode'.
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mtab'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/hosts.deny'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/mail/statistics'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random-seed'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/random.seed'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/adjtime'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/httpd/logs'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/utmpx'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/wtmpx'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/cups/certs'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/dumpdates'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/etc/svc/volatile'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/sys/kernel/security'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/sys/kernel/debug'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6206): Ignore 'file' entry '/dev/core'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6207): Ignore 'file' sregex '^/proc'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6207): Ignore 'file' sregex '.log$|.swp$'
2019/08/06 11:43:49 ossec-syscheckd: INFO: (6004): No diff for file: '/etc/ssl/private.key'
2019/08/06 11:43:49 rootcheck: INFO: Started (pid: 22718).
2019/08/06 11:43:49 sca: INFO: Evaluation finished for policy 'cis_debian9_L2.yml'
2019/08/06 11:43:49 sca: INFO: Starting evaluation of policy: 'cis_debian9_L1.yml'
2019/08/06 11:43:52 sca: INFO: Evaluation finished for policy 'cis_debian9_L1.yml'
2019/08/06 11:43:53 sca: INFO: Security Configuration Assessment scan finished. Duration: 7 seconds.
2019/08/06 11:44:04 ossec-syscheckd: INFO: (6010): File integrity monitoring scan frequency: 43200 seconds
2019/08/06 11:44:04 ossec-syscheckd: INFO: (6008): File integrity monitoring scan started.
2019/08/06 11:44:04 rootcheck: INFO: Starting rootcheck scan.

@Skeptor Skeptor changed the title SCA Memory Test SCA Global Testing Aug 6, 2019
@JuantAldea
Copy link
Contributor

sca-10-scans-osx

Usage of memory on MacOS for 10-scan run. Longer testing is needed to evaluate what's going on at the end.

@Skeptor
Copy link
Author

Skeptor commented Aug 8, 2019 •
edited

Integrity testing

Delete a check from the DB

Server side:

sqlite> select * from sca_check where id="3000";
1284019443|3000|cis_debian9_L1|Ensure mounting of freevxfs filesystems is disabled|The freevxfs filesystem type is a free version of the Veritas type filesystem. This is the primary filesystem type for HP-UX operating systems.|Removing support for unneeded filesystem types reduces the local attack surface of the system. If this filesystem type is not needed, disable it.|Edit or create a file in the /etc/modprobe.d/ directory ending in .conf Example: vim /etc/modprobe.d/freevxfs.conf and add the following line: install freevxfs /bin/true Run the following command to unload the freevxfs module: # rmmod freevxfs|||||/sbin/modprobe -n -v freevxfs||failed||

sqlite> delete from sca_check where id="3000";

Agent side at next scan:
2019/08/08 07:30:21 sca: INFO: Integration checksum failed for policy 'cis_debian9_L1.yml'. Resending scan results in 109 seconds.

Server side after receiving the scan:

sqlite> select * from sca_check where id="3000";
912760343|3000|cis_debian9_L1|Ensure mounting of freevxfs filesystems is disabled|The freevxfs filesystem type is a free version of the Veritas type filesystem. This is the primary filesystem type for HP-UX operating systems.|Removing support for unneeded filesystem types reduces the local attack surface of the system. If this filesystem type is not needed, disable it.|Edit or create a file in the /etc/modprobe.d/ directory ending in .conf Example: vim /etc/modprobe.d/freevxfs.conf and add the following line: install freevxfs /bin/true Run the following command to unload the freevxfs module: # rmmod freevxfs|||||/sbin/modprobe -n -v freevxfs||failed||

Changing a check from a policy

Server side:

sqlite> select * from sca_check where id="3001";
912760343|3001|cis_debian9_L1|Ensure mounting of jffs2 filesystems is disabled|The jffs2 (journaling flash filesystem 2) filesystem type is a log-structured filesystem used in flash memory devices.|Removing support for unneeded filesystem types reduces the local attack surface of the system. If this filesystem type is not needed, disable it.|Edit or create a file in the /etc/modprobe.d/ directory ending in .conf Example: vim /etc/modprobe.d/jffs2.conf and add the following line: install jffs2 /bin/true Run the following command to unload the jffs2 module: # rmmod jffs2|||||/sbin/modprobe -n -v jffs2||failed||

Agent side:
Change the check from the scanned policy to write a test check:

- id: 3001
    title: "Change the content of a check"
    description: "The check must be updated in the server."
    rationale: "The SCA integrity must handle this."
    remediation: "Resend the scan"
    compliance:
      - cis: ["1.1.1.2"]
      - cis_csc: ["5.1"]
    condition: none
    rules:
      - 'c:echo -n integrity_test -> r:passed'

Logs

Agent side:

2019/08/08 07:49:08 sca[6060] wm_sca.c:2855 at wm_sca_dump_db_thread(): DEBUG: Sending first scan results for policy 'cis_debian9_L1.yml'
2019/08/08 07:49:08 sca[6060] wm_sca.c:2863 at wm_sca_dump_db_thread(): DEBUG: Dumping results to SCA DB for policy 'cis_debian9_L1.yml' (Policy index: 1)
2019/08/08 07:49:14 sca[6060] wm_sca.c:2892 at wm_sca_dump_db_thread(): DEBUG: Sending end of dump control event.
2019/08/08 07:49:16 sca[6060] wm_sca.c:2911 at wm_sca_dump_db_thread(): DEBUG: Finished dumping scan results to SCA DB for policy 'cis_debian9_L1' (1) (1)

Server side:
2019/08/08 07:49:06 ossec-analysisd: INFO: Policy 'cis_debian9_L1' outdated in agent '005'. Latest scan requested.

New check in the server:

sqlite> select * from sca_check where id="3001";
2128865376|3001|cis_debian9_L1|Change the content of a check|The check must be updated in the server.|The SCA integrity must handle this.|Resend the scan|||||echo -n integrity_test||passed||

@cristgl
Copy link
Contributor

cristgl commented Aug 8, 2019 •
edited

Using leaks to detect memory leaks on MacOS

Changing the notify_time to 60 seconds and disabling every module but SCA. Scanning cis_apple_macOS_10.13.yml and policy file tests.
After running at least 15 scans, the summary of them is the next one, with 0 leaks.

# leaks 72527 --fullContent --fullStack
Process:         wazuh-modulesd [72527]
Path:            /private/var/ossec/bin/wazuh-modulesd
Load Address:    0x10ef4a000
Identifier:      wazuh-modulesd
Version:         ???
Code Type:       X86-64
Parent Process:  sh [49958]

Date/Time:       2019-08-08 09:52:09.203 +0200
Launch Time:     2019-08-08 09:51:10.371 +0200
OS Version:      Mac OS X 10.13.5 (17F77)
Report Version:  7
Analysis Tool:   /usr/bin/leaks

Physical footprint:         2032K
Physical footprint (peak):  2032K
----

leaks Report Version: 3.0, multi-line stacks
Process 72527: 2189 nodes malloced for 251 KB
Process 72527: 0 leaks for 0 total leaked bytes.

After the 60 seconds from the notify_time have passed, leaks reports this:

# leaks 72527 --fullContent --fullStack
Process:         wazuh-modulesd [72527]
Path:            /private/var/ossec/bin/wazuh-modulesd
Load Address:    0x10ef4a000
Identifier:      wazuh-modulesd
Version:         ???
Code Type:       X86-64
Parent Process:  sh [49958]

Date/Time:       2019-08-08 09:52:10.030 +0200
Launch Time:     2019-08-08 09:51:10.371 +0200
OS Version:      Mac OS X 10.13.5 (17F77)
Report Version:  7
Analysis Tool:   /usr/bin/leaks

Physical footprint:         2044K
Physical footprint (peak):  2044K
----

leaks Report Version: 3.0, multi-line stacks
Process 72527: 2302 nodes malloced for 253 KB
Process 72527: 113 leaks for 1808 total leaked bytes.

Same results after running several subsequent scans.

@JuantAldea
Copy link
Contributor

15 minutes of SCA running under masiff. Executing one scan per minute of the Debian & testing policies.
massiff-15min-linux

@cristgl
Copy link
Contributor

cristgl commented Aug 8, 2019 •
edited

Windows testing

Disabling a policy
A policy being disabled means that its information is not included in the database. Disabling a policy between two scans with the suffix .disabled does not remove this data.

Setting a complete path to a policy different than the one by default
With the next configuration:

  <sca>
    <enabled>yes</enabled>
    <scan_on_start>yes</scan_on_start>
    <interval>1m</interval>
    <skip_nfs>yes</skip_nfs>

    <policies>
	  <policy>C:\Program Files (x86)\ossec-agent\ruleset\sca_win_registry_test_suite.yml</policy>
    </policies>
  </sca>

Shows the next message, however the policy is correctly scanned.

 2019/08/08 11:31:30 sca: ERROR: Unable to calculate SHA256 for file 'ruleset\sca/C:\Program Files (x86)\ossec-agent\ruleset\sca_win_registry_test_suite.yml'
2019/08/08 11:31:47 sca: INFO: Starting evaluation of policy: 'C:\Program Files (x86)\ossec-agent\ruleset\sca_win_registry_test_suite.yml'
2019/08/08 11:31:51 sca: INFO: Evaluation finished for policy 'C:\Program Files (x86)\ossec-agent\ruleset\sca_win_registry_test_suite.yml'

It adds the directory by default, ruleset\sca, to the complete path.

@Skeptor
Copy link
Author

Skeptor commented Aug 9, 2019

Extended testing on disabling a policy on Windows while the agent is running

sqlite> select distinct policy_id from sca_check;
cis_win10_enterprise_L1
cis_win10_enterprise_L2
directories_test_suite
sca_condition_test_suite
sca_files_test_suite
sca_negation_test_suite
sca_processes_test_suite
sca_win_audit
win_registry_test_suite

sca_files_test_suite.yml -> sca_files_test_suite.yml.disable

Until the agent is restarted the agent logs a Warning: WARNING: Policy file not found: 'ruleset\sca\sca_files_test_suite.yml'. Skipping it.
But the table isn't deleted from the DB.

When the agent is restarted and the first SCA scan is send the DB purges that policy:

sqlite> select distinct policy_id from sca_check;
cis_win10_enterprise_L1
cis_win10_enterprise_L2
directories_test_suite
sca_condition_test_suite
sca_negation_test_suite
sca_processes_test_suite
sca_win_audit
win_registry_test_suite

@Skeptor
Copy link
Author

Skeptor commented Aug 9, 2019 •
edited

Memory Use of Windows Agent

Powershell script to get the Memory on real time:
Get-Process -Name ossec-agent | Select-Object Name,@{Name='WorkingSet';Expression={($_.WorkingSet)}}

Configuration:
All modules except SCA are disabled.
Notify time: 10000
SCA with default policy: sca_win_audit and a frequency of 1 minute.
X axis: time (seconds)
Y axis: Memory (KB)

image

@Skeptor
Copy link
Author

Skeptor commented Aug 9, 2019 •
edited

Script: while (1){ Get-Process -Name ossec-agent | Select-Object Name,@{Name='WorkingSet';Expression={($_.WorkingSet)}}; sleep 4;}

Configuration:
All modules except SCA are disabled.
Notify time: 10000
SCA policies:

cis_win10_enterprise_L1
cis_win10_enterprise_L2
directories_test_suite
sca_condition_test_suite
sca_files_test_suite
sca_negation_test_suite
sca_processes_test_suite
sca_win_audit
win_registry_test_suite
C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_files.yml  (Custom policy based on file test created for this test)

Frequency: 1 minute.
X axis: time (seconds)
Y axis: Memory (KB)

Changes during execution:
Policy appear/disappear

 while(1){mv "C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_files.yml" "C:\Program Files (x8
6)\ossec-agent\ruleset\sca\sca_files.yml.disable"; sleep 60; mv "C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_file
s.yml.disable" "C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_files.yml"; sleep 60;}

Half time IDs repeated and the other half valid policy (Check repeated ID and integrity)

while(1){(Get-Content -path "C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_repeated_global_
ids_1.yml" -Raw) -replace '300','330'; sleep 60; (Get-Content -path "C:\Program Files (x86)\ossec-agent\ruleset\sca\sca_
repeated_global_ids_1.yml" -Raw) -replace '330','300'; sleep 60;}

sca_allw10_policies

@cristgl
Copy link
Contributor

cristgl commented Aug 14, 2019 •
edited

SCA actions with events and alerts generated

Action Events Alerts
First scan Every check + summary Every check + summary
Agent restart Summary None
Check changes its status Changed check + summary Changed check + summary
Agent stop, change check, agent start Every check + summary Changed check + summary
Policy file change (change condition) Every check + summary Changed check + summary + every check + summary
Policy file change (add line) Every check + summary Every check + summary
Integrity fail (event lost) Every check + summary The event lost (in the next scan)

@chemamartinez
Copy link
Contributor

The SCA refactor has been merged, so I am closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cristgl cristgl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@JuantAldea @Skeptor @cristgl @chemamartinez