Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 4.3.6 - Release Candidate 1 - E2E UX tests - Configuration assessment #14268

Closed
1 task done
jnasselle opened this issue Jul 15, 2022 · 2 comments
Closed
1 task done
Assignees
Labels
feed/sca Security Configuration Assessment policies related issues module/sca Security Configuration Assessment module release test/4.3.6 Issues related to testing for 4.3.6 type/test/manual

Comments

@jnasselle
Copy link
Member

jnasselle commented Jul 15, 2022

The following issue aims to run the specified test for the current release candidate, report the results, and open new issues for any encountered errors.

Modules tests information

Main release candidate issue #14188
Main E2E UX test issue #14260
Version 4.3.6
Release candidate # RC1
Tag v4.3.6-rc1
Previous modules tests issue #12889

Installation procedure

Description

Validate issues reported from last E2E test

#13030
#12347

Validate changes from v4.3.0 to 4.3.6 (only if applies)

#13893
#13905
#13781
#13950

Validate documentation consistency

https://documentation-dev.wazuh.com/4.3.6-rc/user-manual/capabilities/sec-config-assessment/index.html

Validate use case

https://documentation-dev.wazuh.com/4.3.6-rc/user-manual/capabilities/sec-config-assessment/use-case.html

Check Wazuh Dasboard SCA scan results

Test report procedure

All test results must have one of the following statuses:

🟢 Test went as expected.
🔴 Test fails and must be addressed.
🟡 Test meet the goal but some improvements must be addressed for UX

Any failing test must be properly addressed with a new issue, detailing the error and the possible cause.

An extended report of the test results can be attached as a ZIP or TXT file. Please attach any documents, screenshots, or tables to the issue update with the results. This report can be used by the auditors to dig deeper into any possible failures and details.

Conclusions

All tests have been executed and the results can be found in the issue updates.

Status Test Failure type Notes
🟢 Validate documentation consistency Functional
🟡 Validate issues reported from last E2E test Functional Issue #12347 is not still addressed since v4.3.0
🟢 Validate issues reported from last E2E test Functional Issue #13030 solved
🟡 Validate changes from v4.3.0 to 4.3.6 Functional #13949 can be improved. Reopened
🟡 Validate documentation consistency Documentation wazuh/wazuh-documentation#5478
🔴 Validate documentation consistency Functional SCA checks IDs are not unique #14346
🟡 Validate use case Documentation wazuh/wazuh-documentation#5478

Auditors validation

The definition of done for this one is the validation of the conclusions and the test results from all auditors.

All checks from below must be accepted in order to close this issue.

@jnasselle jnasselle added module/sca Security Configuration Assessment module feed/sca Security Configuration Assessment policies related issues type/test/manual release test/4.3.6 Issues related to testing for 4.3.6 labels Jul 15, 2022
@jftuduri
Copy link
Member

jftuduri commented Jul 15, 2022

Wazuh Manager stack on Ubuntu 22.04 🟢

  • Indexer installation 🟢
root@jammy:/home/vagrant#  curl -k -u admin:admin https://192.168.20.27:9200/_cat/nodes?v
ip            heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.20.27           19          95   0    0.09    0.07     0.08 dimr      *      node-1
  • Server installation 🟢
root@jammy:/home/vagrant# systemctl status wazuh-manager
● wazuh-manager.service - Wazuh manager
     Loaded: loaded (/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2022-07-15 14:58:21 -03; 53min ago
      Tasks: 175 (limit: 2240)
     Memory: 251.7M
        CPU: 1min 11.371s
     CGroup: /system.slice/wazuh-manager.service
             ├─76087 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
             ├─76126 /var/ossec/bin/wazuh-authd
             ├─76142 /var/ossec/bin/wazuh-db
             ├─76156 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
             ├─76159 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
             ├─76171 /var/ossec/bin/wazuh-execd
             ├─76185 /var/ossec/bin/wazuh-analysisd
             ├─76282 /var/ossec/bin/wazuh-syscheckd
             ├─76295 /var/ossec/bin/wazuh-remoted
             ├─76327 /var/ossec/bin/wazuh-logcollector
             ├─76350 /var/ossec/bin/wazuh-monitord
             └─76372 /var/ossec/bin/wazuh-modulesd

Jul 15 14:58:19 jammy env[76034]: Started wazuh-modulesd...
Jul 15 14:58:21 jammy env[76034]: Completed.
Jul 15 14:58:21 jammy systemd[1]: Started Wazuh manager.
  • Filebeat status: 🟢
root@jammy:/home/vagrant#  filebeat test output
elasticsearch: https://192.168.20.27:9200...
  parse url... OK
  connection...
    parse host... OK
    dns lookup... OK
    addresses: 192.168.20.27
    dial up... OK
  TLS...
    security: server's certificate chain verification is enabled
    handshake... OK
    TLS version: TLSv1.3
    dial up... OK
  talk to server... OK
  version: 7.10.2
  • Dashboard 🟢
    dashboard

Wazuh Agent on Ubuntu 22.04 🟢

  • Agent v4.3.6 installation 🟢
    agent4 3 6

Changes from v4.3.0 to 4.3.6 #13893 🟢

- Test that an SCA policy check that passed on the first scan generates an alert at a later time when the condition check fails.:green_circle:
  • Policy check used: Id: 28577, Title: Ensure FTP Server is not installed.
  • First scan: FTP server is not installed, check passed:
** Alert 1657921910.755745: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 18:51:50 (jammy-agent) any->sca
< 22.04 LTS.: Ensure FTP Server is not installed.'
{"type":"check","id":1643541447,"policy":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ub>
sca.type: check
sca.scan_id: 1643541447
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.check.id: 28557
sca.check.title: Ensure FTP Server is not installed.
sca.check.description: The File Transfer Protocol (FTP) provides networked computers with the ability to transfe>
sca.check.rationale: FTP does not protect the confidentiality of data or authentication credentials. It is recom>
sca.check.remediation: Run the following command to remove vsftpd: # apt purge vsftpd
sca.check.compliance.cis: 2.1.9
sca.check.compliance.cis_csc: 9.2
sca.check.compliance.pci_dss: 2.2.2
sca.check.compliance.nist_800_53: CM.1
sca.check.compliance.tsc: CC5.2
sca.check.command: ["dpkg -s vsftpd"]
sca.check.result: passed

Screenshot from 2022-07-15 19-24-34

  • FTP server is installed on agent.
  • Second scan: FTP server is installed, check fails, alert is generated:
** Alert 1657924319.1214375: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 19:31:59 (jammy-agent) any->sca
Rule: 19011 (level 9) -> 'CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04>
{"type":"check","id":1016283238,"policy":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ub>
sca.type: check
sca.scan_id: 1016283238
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.check.id: 28557
sca.check.title: Ensure FTP Server is not installed.
sca.check.description: The File Transfer Protocol (FTP) provides networked computers with the ability to transfe>
sca.check.rationale: FTP does not protect the confidentiality of data or authentication credentials. It is recom>
sca.check.remediation: Run the following command to remove vsftpd: # apt purge vsftpd
sca.check.compliance.cis: 2.1.9
sca.check.compliance.cis_csc: 9.2
sca.check.compliance.pci_dss: 2.2.2
sca.check.compliance.nist_800_53: CM.1
sca.check.compliance.tsc: CC5.2
sca.check.command: ["dpkg -s vsftpd"]
sca.check.result: failed
sca.check.previous_result: passed

** Alert 1657924323.1216744: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 19:32:03 (jammy-agent) any->sca
Rule: 19004 (level 7) -> 'SCA summary: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubunt>
{"type":"summary","scan_id":1016283238,"name":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark f>
sca.type: summary
sca.scan_id: 1016283238
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.description: This document provides prescriptive guidance for establishing a secure configuration posture fo>
sca.policy_id: cis_ubuntu22-04
sca.passed: 66
sca.failed: 109
sca.invalid: 16
sca.total_checks: 191
sca.score: 37
sca.file: cis_ubuntu22-04.yml

Screenshot from 2022-07-15 19-34-05

- In addition, several policies were checked changing their state and verifying the status on the dashboard.:green_circle: - Policies used: 28500, 28501, 28502, 28503, 28504, 28505, 28506, 28513, 28525, 28528, 28539, 28540, 28545, 28555, 28557, 28564, 28569, 28571, 28600, 28631, 28635.

Use case: Getting an alert when a check changes its result value (link) 🟢

  • First scan alert:
** Alert 1657918961.475117: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 18:02:41 (jammy-agent) any->sca
Rule: 19008 (level 3) -> 'CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04>
{"type":"check","id":1427994619,"policy":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ub>
sca.type: check
sca.scan_id: 1427994619
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.check.id: 28655
sca.check.title: Ensure SSH root login is disabled.
sca.check.description: The PermitRootLogin parameter specifies if the root user can log in using ssh.
sca.check.rationale: Disallowing root logins over SSH requires system admins to authenticate using their own ind>
sca.check.remediation: Edit /etc/ssh/sshd_config or a file in /ssh/sshd_config.d/ ending in .conf to set the par>
sca.check.compliance.cis: 5.3.10
sca.check.compliance.cis_csc: 4.3
sca.check.compliance.pci_dss: 4.1
sca.check.compliance.hipaa: 164.312.a.2.IV,164.312.e.1,164.312.e.2.I,164.312.e.2.II
sca.check.compliance.nist_800_53: SC.8
sca.check.compliance.tsc: CC6.7
sca.check.command: ["sshd -T"]
sca.check.result: passed
  • First scan summary:
** Alert 1657918969.588191: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 18:02:49 (jammy-agent) any->sca
Rule: 19004 (level 7) -> 'SCA summary: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubunt>
{"type":"summary","scan_id":1427994619,"name":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark f>
sca.type: summary
sca.scan_id: 1427994619
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.description: This document provides prescriptive guidance for establishing a secure configuration posture fo>
sca.policy_id: cis_ubuntu22-04
sca.passed: 68
sca.failed: 107
sca.invalid: 16
sca.total_checks: 191
sca.score: 38
sca.file: cis_ubuntu22-04.yml
  • Alert on dashboard:
    Screenshot from 2022-07-15 18-06-36

  • Scan after enabling PermitRootLogin:

** Alert 1657919499.591261: - sca,gdpr_IV_35.7.d,pci_dss_2.2,nist_800_53_CM.1,tsc_CC7.1,tsc_CC7.2,
2022 Jul 15 18:11:39 (jammy-agent) any->sca
Rule: 19011 (level 9) -> 'CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04>
{"type":"check","id":443760860,"policy":"CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubu>
sca.type: check
sca.scan_id: 443760860
sca.policy: CIS benchmark for Ubuntu Linux 22.04 LTS based on CIS benchmark for Ubuntu Linux 22.04 LTS.
sca.check.id: 28655
sca.check.title: Ensure SSH root login is disabled.
sca.check.description: The PermitRootLogin parameter specifies if the root user can log in using ssh.
sca.check.rationale: Disallowing root logins over SSH requires system admins to authenticate using their own ind>
sca.check.remediation: Edit /etc/ssh/sshd_config or a file in /ssh/sshd_config.d/ ending in .conf to set the par>
sca.check.compliance.cis: 5.3.10
sca.check.compliance.cis_csc: 4.3
sca.check.compliance.pci_dss: 4.1
sca.check.compliance.hipaa: 164.312.a.2.IV,164.312.e.1,164.312.e.2.I,164.312.e.2.II
sca.check.compliance.nist_800_53: SC.8
sca.check.compliance.tsc: CC6.7
sca.check.command: ["sshd -T"]
sca.check.result: failed
sca.check.previous_result: passed
  • Alert on dashboard:
    Screenshot from 2022-07-15 18-12-17

Wazuh Dashboard SCA scan results 🟢

  • No problems were found exploring the SCA Dashboard.

Screenshot from 2022-07-18 17-34-06

Screenshot 2022-07-18 at 17-34-19 Wazuh - Wazuh

@sebasfalcone
Copy link
Member

sebasfalcone commented Jul 16, 2022

Server install

🟢 Wazuh install on Amazon linux 2

For this tests the wazuh_install.sh script was used, it was modified to download RC 4.3.6 from the URL packages-dev.wazuh.com/pre-release.

  • 🟢 Very clean and descriptive console logs.
  • 🟢 No issue on installer from fresh amazon linux 2

🟢 Wazuh Manager

🟢wazuh-manager.service - Wazuh manager
   Loaded: loaded (/usr/lib/systemd/system/wazuh-manager.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2022-07-16 19:29:02 UTC; 20min ago
   Process: 7119 ExecStart=/usr/bin/env /var/ossec/bin/wazuh-control start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/wazuh-manager.service
           ├─7428 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─7469 /var/ossec/bin/wazuh-authd
           ├─7486 /var/ossec/bin/wazuh-db
           ├─7511 /var/ossec/bin/wazuh-execd
           ├─7525 /var/ossec/bin/wazuh-analysisd
           ├─7527 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─7530 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh-apid.py
           ├─7544 /var/ossec/bin/wazuh-syscheckd
           ├─7561 /var/ossec/bin/wazuh-remoted
           ├─7641 /var/ossec/bin/wazuh-logcollector
           ├─7663 /var/ossec/bin/wazuh-monitord
           └─7685 /var/ossec/bin/wazuh-modulesd

🟢 Wazuh indexer

🟢wazuh-indexer.service - Wazuh-indexer
   Loaded: loaded (/usr/lib/systemd/system/wazuh-indexer.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2022-07-16 19:29:27 UTC; 20min ago
     Docs: https://documentation.wazuh.com
 Main PID: 7117 (java)
   CGroup: /system.slice/wazuh-indexer.service
           └─7117 /usr/share/wazuh-indexer/jdk/bin/java -Xshare:auto -Dopensearch.networkaddress.cache.ttl=60 -Dopensearch.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headl...

🟢 Wazuh dashboard

🟢wazuh-dashboard.service - wazuh-dashboard
   Loaded: loaded (/etc/systemd/system/wazuh-dashboard.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat 2022-07-16 19:28:28 UTC; 22min ago
 Main PID: 6609 (node)
   CGroup: /system.slice/wazuh-dashboard.service
           └─6609 /usr/share/wazuh-dashboard/bin/../node/bin/node --no-warnings --max-http-header-size=65536 --unhandled-rejections=warn /usr/share/wazuh-dashboard/bin/../src/cli/dist -c /etc/wazuh-dashb...

Agent install

🟢Wazuh agent installation on linux
  • 🟢 No issue on installer from fresh amazon linux 2

For this test we used the one liner command provided by the wazuh dasboard WUI:

  • It was necessary to modify the command because the URL provided for the version 4.3.6 is located on the pre-release state:
    - Provided:
    sudo WAZUH_MANAGER='wazuhmanager.com' WAZUH_AGENT_GROUP='default' yum install https://packages.wazuh.com/4.x/yum/wazuh-agent-4.3.6-1.x86_64.rpm
    - Used:
    sudo WAZUH_MANAGER='wazuhmanager.com' WAZUH_AGENT_GROUP='default' yum install https://packages-dev.wazuh.com/pre-release/yum/wazuh-agent-4.3.6-1.x86_64.rpm

SCA

Requisites

🟢 Defaults and temples are properly set.
[user1@amazonlinux wazuh-install-files]$ sudo tail -f /var/ossec/logs/ossec.log
2022/07/18 13:26:03 sca: INFO: Starting Security Configuration Assessment scan.
2022/07/18 13:26:03 wazuh-modulesd:task-manager: INFO: (8200): Module Task Manager started.
2022/07/18 13:26:03 sca: INFO: Starting evaluation of policy: '/var/ossec/ruleset/sca/cis_amazon_linux_2.yml'
2022/07/18 13:26:04 wazuh-modulesd:syscollector: INFO: Module started.
2022/07/18 13:26:04 wazuh-modulesd:syscollector: INFO: Starting evaluation.
2022/07/18 13:26:05 wazuh-modulesd:syscollector: INFO: Evaluation finished.
2022/07/18 13:26:13 wazuh-syscheckd: INFO: (6009): File integrity monitoring scan ended.
2022/07/18 16:26:29 sca: INFO: Evaluation finished for policy '/var/ossec/ruleset/sca/cis_amazon_linux_2.yml'
2022/07/18 16:26:29 sca: INFO: Security Configuration Assessment scan finished. Duration: 10826 seconds.
2022/07/18 16:26:55 rootcheck: INFO: Ending rootcheck scan.

Previous release tests

🟢 Ensure sticky bit is set on all world-writable directories.
  • Remove sticky bits from the /tmp directory: # chmod -t /tmp
  • Check expected SCA alert is generated:
    image.png
🟢 Ensure iptables are flushed with nftables.
  • Install nftables: # yum install nftables
  • flush iptables and ip6tables rules:
# iptables -F
# ip6tables -F
  • Check that control pass:
    image.png
🟢 Policy doesn't create the /dev/null file.
  • After multiple scans, no /dev/null file was created.
[root@linuxagent user1]# ls /dev/nul*
/dev/null
🔴 Ensure XD/NX support is enabled.
  • Following the tests described in the issue #3014. I've discover that one of the commands executed on the check for the SCA rule 20529 is probably faulty.

  • Command executed:

sh -c "journalctl | grep "protection: " | tail -1"
  • if for some reason, some command logged on journal the string "protection: " then this scan will fail:

  • Steps to reproduce:

    • Output before "corrupting" journal:
     # journalctl | grep "protection: " | tail -1
     Jul 18 14:59:17 localhost kernel: NX (Execute Disable) protection: active
    
    • Output after corruption:
     $ sudo sh -c "journalctl | grep "protection: " | tail -1"
     Jul 18 15:10:56 linuxagent sudo[35079]:    user1 : TTY=pts/0 ; PWD=/home/user1 ; USER=root ; COMMAND=/bin/sh -c journalctl | grep protection:  | tail -1
     	
     # journalctl | grep "protection: " | tail -1
     Jul 18 15:10:56 linuxagent sudo[35079]:    user1 : TTY=pts/0 ; PWD=/home/user1 ; USER=root ; COMMAND=/bin/sh -c journalctl | grep protection:  | tail -1
    
     # journalctl | grep "protection: " | tail -2
     Jul 18 14:59:17 localhost kernel: NX (Execute Disable) protection: active
     Jul 18 15:00:30 linuxagent sudo[7923]:    user1 : TTY=pts/0 ; PWD=/home/user1 ; USER=root ; COMMAND=/bin/sh -c journalctl | grep protection:  | tail -1
    
    • The outcome is that the check failes, despites the NX protection being active.

Proposed solutions

  • We can add an extra layer of filtering by using flags on the journalctl command. Given the nature of the XD/NX, we propose the following flags:
    # journalctl -k --boot
    • -k because is a kernel message.
    • --boot because the message is generated at boot time.
  • With these two flags in place and the grep that was used before we solved the problem addessed before.

New tests

🟢 Ensure journald is configured to write logfiles to persistent disk.
  • ID: 20629
  • Initial status: failed
  • Status after Remediation: passed
🟢 Ensure SSH PermitEmptyPasswords is disabled.
  • ID 20651
  • Initial status: failed
  • Status after Remediation: passed
🟡 Ensure iptables packages are installed.
  • ID: 20598

  • Initial status: failed

  • Status after Remediation: failed

    • This policie is a bundle of three audits described on the CIS benchmark:
      1. 3.5.3.1.1 Ensure iptables packages are installed
      2. 3.5.3.1.2 Ensure nftables is not installed with iptables
      3. 3.5.3.1.3 Ensure firewalld is either not installed or masked with iptables.
    • Remediation only suggest:

    # yum install iptables iptables-services

    • We should add the remediation associated with B and C, this is:
     # yum remove nftables
     # yum remove firewalld
    
🟢 Ensure noexec option set on /dev/shm partition.
  • ID: 20508
  • Initial status: failed
  • Status after remediation: passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feed/sca Security Configuration Assessment policies related issues module/sca Security Configuration Assessment module release test/4.3.6 Issues related to testing for 4.3.6 type/test/manual
Projects
No open projects
Status: Done
Development

No branches or pull requests

5 participants