Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vulnerability Detection is not working as expected in Demo environment #23322

Closed
Rebits opened this issue May 7, 2024 · 9 comments
Closed

Vulnerability Detection is not working as expected in Demo environment #23322

Rebits opened this issue May 7, 2024 · 9 comments

Comments

@Rebits
Copy link
Member

Rebits commented May 7, 2024

Test information

Main release stage issue #23246
Version 4.8.0
Release candidate RC 1
Tag https://github.com/wazuh/wazuh-qa/tree/v4.8.0-rc1

Description

This issue addresses concerns regarding vulnerability detection in the Demo environment. It has been observed that vulnerability detection is not functioning correctly for Debian agents, and there are unexpected behaviors when toggling the vulnerability detection feature. Further investigation is necessary to identify the root causes of these issues.

Problem Statement

  • Vulnerability detection is failing to report vulnerabilities for Debian agents.
  • Enabling and disabling vulnerability detection is leading to unexpected behavior, potentially removing already detected RHEL vulnerabilities.

Test Requirements

Environment

Access to the Demo environment is necessary. Credentials can be obtained from the devops team .

Configuration

The environment should have enabled the wazuh-modulesd debug mode. This will provide more detailed logs for analysis.

Tests Cases

T1: Basic Vulnerability Detection

Description: Verify if vulnerability detection triggers vulnerabilities for all agents of the Demo Environment, having the vulnerability detection module enabled when agents were registered.
Initial conditions: Initially, vulnerability detection is enabled, and all agents are registered while this module remains active.

ID Description Status
T1.1 Basic vulnerability detection for Debian agent 🟢
T1.2 Basic vulnerability detection for RHEL agent 🟢
T1.3 Basic vulnerability detection for centOS agent 🟢
T1.4 Basic vulnerability detection for Windows Server agent 🟢

T2: Assess Vulnerability Detection Re-Scan Behavior

Description: Verify if re-scan mechanism works as expected when a vulnerability detection is enabled once agents have been already registered.

Initial conditions: At the outset, vulnerability detection remains disabled, and all agents are registered while this module remains in its deactivated state.

ID Description Status
T2.1 Vulnerability detection for Debian agent when re-scan is triggered 🔴
T2.2 Vulnerability detection for RHEL agent when re-scan is triggered 🔴

Conclusion 🔴

Multiple issues were detected in the demo environment

@Rebits Rebits self-assigned this May 7, 2024
@Rebits Rebits changed the title Vulnearbility Detection is not working as expected in Demo environment Vulnerability Detection is not working as expected in Demo environment May 7, 2024
@Rebits
Copy link
Member Author

Rebits commented May 7, 2024

T1: Basic Vulnerability Detection

ID Description Status
T1.1 Basic vulnerability detection for Debian agent 🟢
T1.2 Basic vulnerability detection for RHEL agent 🟢
T1.3 Basic vulnerability detection for centOS agent 🟢
T1.4 Basic vulnerability detection for Windows Server agent 🟢

Description

Initially, the Demo environment comprised a RHEL and a Debian agent. Vulnerability scans on these agents did not detect any vulnerabilities. However, the remaining agents exhibited triggered vulnerabilities.

image

Note

No vulnerabilities were detected for RHEL agent

Upon assessing the current status of the manager, it was observed that removing and re-registering the agents resolved the issue.

image

It's likely that the reported errors are specifically related to the re-scan mechanism. This will be further investigated in the T2 cases.


After a few hours, after restraint manager and worker several times for other tests cases, vulnerabilities were detected for RHEL but not for Debian agent:

image

@Rebits
Copy link
Member Author

Rebits commented May 7, 2024

T2: Assess Vulnerability Detection Re-Scan Behavior 🔴

ID Description Status
T2.1 Vulnerability detection for Debian agent when re-scan is triggered 🔴
T2.2 Vulnerability detection for RHEL agent when re-scan is triggered 🔴

Description

In an environment where all agents are linked and Vulnerability Detection is actively running, our next steps involve disabling the Vulnerability Detection module. We'll initiate this process via the dashboard, starting with the worker nodes and then proceeding to the master node.

Subsequently, we'll undertake the crucial task of backing up agents's client keys and then removing them from both Debian and RHEL agents. Before removal, we'll ensure to rename them within the configuration settings as 'RHEL-4' and 'Debian-4' respectively.

Once the client keys are secured and renamed, we'll proceed to register the agents in the environment with the Vulnerability Detection disabled. Following this, as the agents reconnect, we'll re-enable the Vulnerability Detection module seamlessly

Results

After enabling the vulnerability detector module again, after 1:30 hours, vulnerabilities of new agents appear:

Debian

debian4

RHEL

rhel4

However several issues were detected:

@Rebits
Copy link
Member Author

Rebits commented May 8, 2024

No existing agent's vulnerabilities found in index 🔴

Description

After registering new agents in the environment, we observed the following list of agents registered:

image

However, upon removing the default wazuh.cluster.name filter, vulnerabilities were detected for a non-existing Ubuntu agent with wazuh.cluster.name value set to 2:

image

The exact registration time of this old agent cannot be provided due to the previous use of a Demo instance. However, it has been more than 1 day since registration.

Steps to Reproduce

No steps to reproduce can be provided for now due this agent was already present in the environment

Expected Behavior

No vulnerabilities should be detected for non-existing agents.

@Rebits
Copy link
Member Author

Rebits commented May 8, 2024

Vulnerabilities of some agents disappear suddely 🔴

Note

After researching this issue, it seems directly related to #23322 (comment)

Description

No vulnerabilities were detected in Debian and RHEL9 agents during testing. These agents were initially deployed in the environment and replicating the same conditions with new agents proved unsuccessful, suggesting a unique setup at the environment's inception.

Various scenarios were tested, including:

  • Restarting the master node concurrently with agent registration.
  • Restarting the worker node concurrently with agent registration.
  • Changing the manager before syscollector packages are dispatched to the original manager.

Despite these efforts, the absence of vulnerabilities remains consistent, hinting at specific conditions present at the environment's outset

After toggling the vulnerability detection module on both the master and worker nodes, please allow approximately 30 minutes for the expected vulnerabilities to manifest.

IMAGE

Few minutes later, vulnerabilities were automatically deleted
image

Steps to Reproduce

No steps to reproduce can be provided for now due this agent was already present in the environment

Expected behaviour

Vulnerability scan should detect vulnerabilities for all agents without the need of disabling and enabling vulnerability detection module.

@Rebits
Copy link
Member Author

Rebits commented May 8, 2024

Index unstable when vulnerability detection module is disabled and enabled again 🔴

Description

The vulnerabilities index seems to be unstable after enabling and disabling the vulnerability detection module in master and worker nodes.
Vulnerabilities of the environment are being removed completely and generated again several times, leading into some agents without any vulnerability (Check #23322 (comment) for more information). In this case the agent 013, created during the test end with no vulnerabilities detected:

image

Video

vuln_dissapear.webm

Steps to reproduce

  • Deploy an environment of a few agents and a cluster of indexer and manager
  • Enable vulnerability detection and wait until vulnerability scan finished to scan all agents
  • Disable vulnerability detection module in both managers
  • Register a new agent into one of the manager
  • Enable vulnerability detection again
  • Check that the vulnerabilities index first does not detect the vulnerabilities of the new agents, and them remove all the vulnerabilities. A few minutes later, vulnerabilities are regenerated and removed again.

Expected behavior

Vulnerability index should be stable in case of enabling and disabling the vulnerability detection module. It is expected the removal of the index, although this should be produced only one time, generating vulnerabilities for all the agents.

@Rebits
Copy link
Member Author

Rebits commented May 8, 2024

Inconsistent wazuh.cluster.name filter 🔴

Description

The default setting for the vulnerability detection dashboard incorporates a filter labeled wazuh.cluster.name set to the value "wazuh1". Initially, it appears this filter is not intended for modification. However, it is possible to deactivate this filter and apply negation by accessing the "Change all filters" menu.

image

Video

filter_inconsistent.webm

Steps to reproduce

  • Go to Vulnerability Detection > Inventory
  • Using "Change all filters" negate the wazuh.cluster.name filter
  • Go to Vulnerability Deteciton > Dashboard
  • Check that dashboard is not working as expected. In addition, cluster filter now is not negated.

@Rebits
Copy link
Member Author

Rebits commented May 8, 2024

Debian agent includes Windows agent syscheck events 🔴

Description

Difficult to replicate error appear in the Debian endpoint menu. In the FIM recent events menu appear some Windows agent events

debian_bad_dashboard

Steps to reproduce

No steps to reproduce can be provided because this issue was impossible to reproduce

@santipadilla
Copy link
Member

LGTM

1 similar comment
@juliamagan
Copy link
Member

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

3 participants