Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'test_providers/test_providers_multiple_providers' tests in 4.2 #1560

Closed
4 tasks done
Tracked by #1531
damarisg opened this issue Jul 6, 2021 · 3 comments
Closed
4 tasks done
Tracked by #1531

Comments

@damarisg
Copy link
Member

damarisg commented Jul 6, 2021

Issue information

Related issue
Closes #1531

When it comes to starting with the test fix, there is some information that can help you:

Module: Vulnerability Detector
Support Team:  "binary beasts"
Target: Manager   
OS: Linux

I add some information obtained while researched the logs that failed.

Case:

Type Description
Test Path test_providers/test_providers_multiple_providers
Consistent no
Test Execution 1/3 executions failed
Cases Fails 15
Summary - It should to do research with more details because in the logs doesn't show any details.
- Also, check it there are a problem when is executed for 3 time on Vagrant.

In order to finish this issue the following tasks should be fulfilled:

  • Research of fails.
  • Apply Fix
  • Full Green/ Full Yellow in test_providers/test_providers_multiple_providers for 3 times.
  • Merge to 1531-full-yellow-vuln-det
@damarisg damarisg changed the title Fix test_providers/test_providers_multiple_providers tests in 4.2 Fix 'test_providers/test_providers_multiple_providers' tests in 4.2 Jul 6, 2021
@pereyra-m
Copy link
Member

pereyra-m commented Jul 13, 2021

Results for test_providers/test_providers_multiple_providers

Test Executions Date By Status
centos_providers_multiple_providers_1.log 2021-07-13 Matias 🔴
centos_providers_multiple_providers_2.log 2021-07-13 Matias 🔴
centos_providers_multiple_providers_3.log 2021-07-13 Matias 🔴

The same package and QA branch were tested on Jenkins, and they all passed, so the issue may be local

This doesn't seem to be a FileMonitor issue, because the log the test needs appears after the test ends.
Considering that these tests depend on the internet connection speed to generate the Downloading feed ... log, the timeout was increased in the local tests. It is reasonable to think that the hosts in the cloud have a faster connection.

Here we have the results of the tests after the timeout increase (VULN_DETECTOR_GLOBAL_TIMEOUT + 100):

Test Executions Date By Status
centos_providers_multiple_providers_timeout_1.log 2021-07-13 Matias 🟢
centos_providers_multiple_providers_timeout_2.log 2021-07-13 Matias 🟢
centos_providers_multiple_providers_timeout_3.log 2021-07-13 Matias 🟢

Update

It was confirmed that disabling the unnecessary modules (sca, syscollector and rootcheck) without increasing the timeout doesn't make the tests pass

Test Executions Date By Status
test_providers_multiple_providers_modules_off_1.log 2021-07-14 Matias 🔴
test_providers_multiple_providers_modules_off_2.log 2021-07-14 Matias 🔴
test_providers_multiple_providers_modules_off_3.log 2021-07-14 Matias 🔴

Tests with modules off and timeout extended

Test Executions Date By Status
test_providers_multiple_providers_modules_off_timeout_1.log 2021-07-14 Matias 🟢
test_providers_multiple_providers_modules_off_timeout_2.log 2021-07-14 Matias 🟢
test_providers_multiple_providers_modules_off_timeout_3.log 2021-07-14 Matias 🟢

@damarisg
Copy link
Member Author

Execution Results with the changes applied after merging to 1531-full-yellow-vuln-det

Test Executions Date By Status
ResultsMultipleProvider1.log 2021-07-16 Seyla 🟡
ResultsMultipleProvider2.log 2021-07-16 Seyla 🟡
ResultsMultipleProvider3.log 2021-07-16 Seyla 🟢
Reference Status
🟢 Pass without warnings
🟡 Pass with Warnings

@mdengra
Copy link
Contributor

mdengra commented Aug 18, 2021

I reopen this issue to fix a DeprecationWarning that appears when running this test:

test_vulnerability_detector/test_providers/test_providers_multiple_providers.py:73
  /root/wazuh-qa/tests/integration/test_vulnerability_detector/test_providers/test_providers_multiple_providers.py:73: DeprecationWarning: invalid escape sequence \<.
    """

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants