Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 'test_general_settings/test_general_settings_run_on_start' tests in 4.2 #1558

Closed
4 tasks done
Tracked by #1531
damarisg opened this issue Jul 6, 2021 · 5 comments
Closed
4 tasks done
Tracked by #1531

Comments

@damarisg
Copy link
Member

damarisg commented Jul 6, 2021

Issue information

Related issue PR Created
Closes #1531 #1603

When it comes to starting with the test fix, there is some information that can help you:

Module: Vulnerability Detector
Support Team:  "binary beasts"
Target: Manager   
OS: Linux

I add some information obtained while researched the logs that failed.

Case:

Type Description
Test Path test_general_settings/test_general_settings_run_on_start
Consistent no
Test Execution 1/3 executions failed
Cases Fails 2
Summary It should to do research with more details because in the logs doesn't show any details.

In order to finish this issue the following tasks should be fulfilled:

  • Research of fails.
  • Apply Fix
  • Full Green/ Full Yellow in test_general_settings/test_general_settings_run_on_start for 3 times.
  • Merge to 1531-full-yellow-vuln-det
@damarisg damarisg changed the title Fix test_general_settings/test_general_settings_run_on_start tests in 4.2 Fix 'test_general_settings/test_general_settings_run_on_start' tests in 4.2 Jul 6, 2021
@mdengra
Copy link
Contributor

mdengra commented Jul 13, 2021

@damarisg
Copy link
Member Author

This issue requires more research because when we execute it, we keep getting fails.

Test Executions Date By Status
test_general_settings_run_on_start_local_r1.log 2021-07-13 Miguel 🔴
test_general_settings_run_on_start_local_r2.log 2021-07-13 Miguel 🔴
test_general_settings_run_on_start_local_r3.log 2021-07-13 Miguel 🔴
Results1_RunOnStart.log 2021-07-13 Seyla 🔴
Results2_RunOnStart.log 2021-07-13 Seyla 🔴
Results3_RunOnStart.log 2021-07-13 Seyla 🔴

@pereyra-m
Copy link
Member

pereyra-m commented Jul 13, 2021

This test was covered in this comment, were all the folder was run.
All the local tests were green:

Test Executions Date By Status
centos1.log 2021-07-12 Matias 🟢
centos2.log 2021-07-12 Matias 🟢
centos3.log 2021-07-12 Matias 🟢

But if the test is run alone, it fails

Test Executions Date By Status
centos_test_general_settings_run_on_start_1.log 2021-07-13 Matias 🔴
centos_test_general_settings_run_on_start_2.log 2021-07-13 Matias 🔴
centos_test_general_settings_run_on_start_3.log 2021-07-13 Matias 🔴

@pereyra-m
Copy link
Member

It was found that the file test_general_settings/data/feeds/custom_nvd_feed.json may not be properly formatted.
This tests fails because the Vulnerability scan finished log line isn't found, the scan is aborted instead

ERROR: (5582): Unavailable vulnerabilities at the NVD database. The scan is aborted.

But if we replace that feed with the one located in test_vulnerability_detector/test_scan_results/data/custom_nvd_feed.json, the NVD vulnerabilities are inserted and the scan ends as expected.

Test Executions Date By Status
test_general_settings_run_on_start_modules_off_new_nvd_1.log 2021-07-14 Matias 🟢
test_general_settings_run_on_start_modules_off_new_nvd_2.log 2021-07-14 Matias 🟢
test_general_settings_run_on_start_modules_off_new_nvd_3.log 2021-07-14 Matias 🟢

This also explains why the test runs successfully together with the whole test_general_settings folder, the NVD vulnerabilities are inserted in a previous test.

@damarisg
Copy link
Member Author

Execution Results with the changes applied on 1531-full-yellow-vuln-det

Test Executions Date By Status
ResultsRunOnStart1.log 2021-07-16 Seyla 🟢
ResultsRunOnStart2.log 2021-07-16 Seyla 🟢
ResultsRunOnStart3.log 2021-07-16 Seyla 🟡
Reference Status
🟢 Pass without warnings
🟡 Pass with Warnings

Merged to 1531-full-yellow-vuln-det

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants