Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the extraction point of the package name in the OVALs #3245

Merged
merged 7 commits into from May 17, 2019

Conversation

Projects
None yet
3 participants
@crolopez
Copy link
Member

commented May 6, 2019

This PR modifies the way in which the vulnerable package name is extracted from the OVAL of Ubuntu, solving #3229.

It is not a good practice to extract fields from XML strings when we have other OVAL sections where we can find that information more accurately. For this reason, the same change has been made for the Debian feeds.

How to reproduce it

Configure the Ubuntu 16 or 18 feed as follows and restart the manager.

<feed name="ubuntu-18">
    <disabled>no</disabled>
    <update_interval>1h</update_interval>
</feed>

A flooding message as the following should be seen:

2019/05/03 14:56:56 wazuh-modulesd:vulnerability-detector: ERROR: (5404): The package name could not be obtained.

Testing

The best way to verify the fix is to generate alerts about the feeds whose update has been broken.

  • Ubuntu 18 (Bionic) alerts.
  • Ubuntu 16 (Xenial) alerts.

The Debian feed update flow has also been modified.

  • Debian 9 (Stretch) alerts.
  • Debian 8 (Jessie) alerts.
  • Debian 7 (Wheezy) alerts.

The the vulnerability indexing can be verified with the following command in the manager side:

sqlite3 /var/ossec/queue/vulnerabilities/cve.db "SELECT * FROM VULNERABILITIES;"  | grep -v tst
  • Valgrind test.

@crolopez crolopez force-pushed the 3229-fix-vuln-canonical branch from 66fb016 to bdaf0d8 May 6, 2019

@crolopez crolopez marked this pull request as ready for review May 7, 2019

@albertomn86

This comment has been minimized.

Copy link
Member

commented May 7, 2019

2019/05/07 13:20:50 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Ubuntu Bionic database update...
2019/05/07 13:21:57 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Ubuntu Xenial database update...
2019/05/07 13:26:06 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Stretch database update...
2019/05/07 13:26:26 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Jessie database update...
2019/05/07 13:26:40 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Wheezy database update...
2019/05/07 13:26:42 wazuh-modulesd:vulnerability-detector: INFO: (5494): The update of the feeds ended successfully.
2019/05/07 13:26:42 wazuh-modulesd:vulnerability-detector: INFO: (5452): Starting vulnerability scanning.
2019/05/07 13:26:42 wazuh-modulesd:vulnerability-detector: INFO: (5453): Vulnerability scanning finished.
@albertomn86

This comment has been minimized.

Copy link
Member

commented May 7, 2019

# sqlite3 /var/ossec/queue/vulnerabilities/cve.db "SELECT OS, COUNT(*) FROM VULNERABILITIES GROUP BY OS;"

BIONIC|77432
JESSIE|21514
STRETCH|19580
WHEEZY|834
XENIAL|186806

@albertomn86 albertomn86 requested a review from snaow May 7, 2019

@albertomn86

This comment has been minimized.

Copy link
Member

commented May 8, 2019

2019/05/08 18:29:16 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Ubuntu Bionic database update...
2019/05/08 18:30:26 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Ubuntu Xenial database update...
2019/05/08 18:35:31 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Stretch database update...
2019/05/08 18:35:44 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Jessie database update...
2019/05/08 18:35:57 wazuh-modulesd:vulnerability-detector: INFO: (5461): Starting Debian Wheezy database update...
2019/05/08 18:35:58 wazuh-modulesd:vulnerability-detector: INFO: (5494): The update of the feeds finished successfully.
2019/05/08 18:35:58 wazuh-modulesd:vulnerability-detector: INFO: (5452): Starting vulnerability scanning.
2019/05/08 18:35:59 wazuh-modulesd:vulnerability-detector: INFO: (5453): Vulnerability scanning finished.
@albertomn86

This comment has been minimized.

Copy link
Member

commented May 8, 2019

# sqlite3 /var/ossec/queue/vulnerabilities/cve.db "SELECT OS, COUNT(*) FROM VULNERABILITIES GROUP BY OS;"

BIONIC|77590
JESSIE|21517
STRETCH|19585
WHEEZY|834
XENIAL|187097
@albertomn86

This comment has been minimized.

Copy link
Member

commented May 8, 2019

Valgrind report:

==3757== HEAP SUMMARY:
==3757==     in use at exit: 380,874 bytes in 181 blocks
==3757==   total heap usage: 34,007 allocs, 33,826 frees, 27,993,911 bytes allocated
==3757== 
==3757== 288 bytes in 1 blocks are possibly lost in loss record 88 of 106
==3757==    at 0x4839775: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3757==    by 0x4013726: allocate_dtv (dl-tls.c:286)
==3757==    by 0x4013726: _dl_allocate_tls (dl-tls.c:532)
==3757==    by 0x4E87D9A: allocate_stack (allocatestack.c:621)
==3757==    by 0x4E87D9A: pthread_create@@GLIBC_2.2.5 (pthread_create.c:669)
==3757==    by 0x13EC82: CreateThreadJoinable (pthreads_op.c:47)
==3757==    by 0x13ED29: CreateThread (pthreads_op.c:62)
==3757==    by 0x11281D: main (main.c:102)
==3757== 
==3757== 1,152 bytes in 4 blocks are possibly lost in loss record 97 of 106
==3757==    at 0x4839775: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3757==    by 0x4013726: allocate_dtv (dl-tls.c:286)
==3757==    by 0x4013726: _dl_allocate_tls (dl-tls.c:532)
==3757==    by 0x4E87D9A: allocate_stack (allocatestack.c:621)
==3757==    by 0x4E87D9A: pthread_create@@GLIBC_2.2.5 (pthread_create.c:669)
==3757==    by 0x13EC82: CreateThreadJoinable (pthreads_op.c:47)
==3757==    by 0x11277C: main (main.c:95)
==3757== 
==3757== 4,104 bytes in 1 blocks are possibly lost in loss record 101 of 106
==3757==    at 0x483774F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3757==    by 0x4AFA434: sqlite3MemMalloc (sqlite3.c:20790)
==3757==    by 0x4AFAE2C: mallocWithAlarm (sqlite3.c:24464)
==3757==    by 0x4AFAED5: sqlite3Malloc (sqlite3.c:24494)
==3757==    by 0x4B0928A: pcache1Alloc (sqlite3.c:45336)
==3757==    by 0x4B09578: sqlite3PageMalloc (sqlite3.c:45479)
==3757==    by 0x4B19B8A: allocateTempSpace (sqlite3.c:61560)
==3757==    by 0x4B1BEF8: btreeCursor (sqlite3.c:63234)
==3757==    by 0x4B1C06A: sqlite3BtreeCursor (sqlite3.c:63276)
==3757==    by 0x4B37979: sqlite3VdbeExec (sqlite3.c:81807)
==3757==    by 0x4B305CC: sqlite3Step (sqlite3.c:76693)
==3757==    by 0x4B3077A: sqlite3_step (sqlite3.c:76754)
==3757==    by 0x11CEDF: wdb_step (wdb.c:366)
==3757==    by 0x11F3CC: wdb_update_agent_name (wdb_agent.c:127)
==3757==    by 0x116125: wm_sync_manager (wm_database.c:228)
==3757==    by 0x115DA9: wm_database_main (wm_database.c:128)
==3757==    by 0x4E87163: start_thread (pthread_create.c:486)
==3757==    by 0x4FBADEE: clone (clone.S:95)
==3757== 
==3757== LEAK SUMMARY:
==3757==    definitely lost: 0 bytes in 0 blocks
==3757==    indirectly lost: 0 bytes in 0 blocks
==3757==      possibly lost: 5,544 bytes in 6 blocks
==3757==    still reachable: 375,330 bytes in 175 blocks
==3757==                       of which reachable via heuristic:
==3757==                         length64           : 223,872 bytes in 122 blocks
==3757==         suppressed: 0 bytes in 0 blocks
==3757== Reachable blocks (those to which a pointer was found) are not shown.
==3757== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==3757== 
==3757== For counts of detected and suppressed errors, rerun with: -v
==3757== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

crolopez added some commits May 10, 2019

Prevent the insertion of duplicate vulnerabilities
When the Red Hat API connection fails, there is a possibility that the next page 
to download contains an already indexed vulnerability.
Convert CURL error message to debug
This avoids warnings flooding in Vulnerability Detector
Remove useless vulnerability checks from the DB
Vulnerability Detector does not use these checks until #3283
@crolopez

This comment has been minimized.

Copy link
Member Author

commented May 15, 2019

OS Indexed vulnerabilities in database
BIONIC 52126
JESSIE 21543
RHEL5 14507
RHEL6 18432
RHEL7 12851
STRETCH 19614
WHEEZY 834
XENIAL 138326
PRECISE 8269
TRUSTY 10019
OS Vagrant box Box version Reported vulnerabilities
CentOS 5 bento/centos-5.11 v201812.27.0 14
CentOS 6 centos/6 v1902.01 2
CentOS 7 generic/centos7 1.8.58 10
Ubuntu 12.04 bento/ubuntu-12.04 v2.3.5 1
Ubuntu 14.04 ubuntu/trusty64 v20190429.0.1 0
Ubuntu 16.04 generic/ubuntu1604 1.9.12 10
Ubuntu 18.04 generic/ubuntu1804 1.9.12 17
Amazon Linux 2017 mvbcoding/awslinux v2017.03.0.20170401 180
Amazon Linux 2 wasilak/amazon-linux-2 v0.2 6
Debian 7 koalephant/debian7-amd64 2.0.0 0
Debian 8 generic/debian8 1.9.12 0
Debian 9 generic/debian9 1.9.6 0
@crolopez

This comment has been minimized.

Copy link
Member Author

commented May 16, 2019

These are some of the vulnerabilities extracted from the Amazon Linux 2017 machine where the branch has been tested:

CVE-2017-10346

  • Package: java-1.7.0-openjdk
  • Version: 1:1.7.0.151-2.6.11.0.74.amzn1
  • Check: less than 1:1.7.0.161-2.6.12.0.el7_4

We can find the patch that resolves the following vulnerability on this page.

imagen

In the patch we can see the version that fixes the vulnerability.

imagen

CVE-2017-15804

  • Package: glibc
  • Version: 2.17-196.172.amzn1
  • Check: less than 2.17-222.el7

We can find the patch that resolves the following vulnerability on this page.

imagen

In the patch we can see the version that fixes the vulnerability.

imagen

CVE-2017-3736

  • Package: openssl
  • Version: 1:1.0.1k-15.99.amzn1
  • Check: less than 1:1.0.2k-12.el7

We can find the patch that resolves the following vulnerability on this page.

imagen

In the patch we can see the version that fixes the vulnerability.

imagen

CVE-2014-0209

  • Package: libXfont
  • Version: 1.4.5-5.12.amzn1
  • Check: less than 1.4.7-2.el7_0

We can find the patch that resolves the following vulnerability on this page.

imagen

In the patch we can see the version that fixes the vulnerability.

imagen

CVE-2014-9636

  • Package: unzip
  • Version: 6.0-4.10.amzn1
  • Check: less than 6.0-15.el7

We can find the patch that resolves the following vulnerability on this page.

imagen

In the patch we can see the version that fixes the vulnerability.

imagen

The alerts of those vulnerabilities include related patches in the vulnerability.advisories field.

@crolopez

This comment has been minimized.

Copy link
Member Author

commented May 16, 2019

Results in a real AWS environment

OS Version Reported vulnerabilities
Amazon Linux AMI 2018.03.0.20190514 x86_64 HVM 88
Amazon Linux 2 AMI 2.0.20190508 x86_64 HVM gp2 1
@crolopez

This comment has been minimized.

Copy link
Member Author

commented May 16, 2019

The following alert is one of those extracted from Amazon Linux AMI from the previous test.

   {
        "timestamp":"2019-05-16T13:48:40.190+0000",
        "rule":{
            "level":10,
            "description":"libXfont: integer overflow of allocations in font metadata file parsing",
            "id":"23505",
            "firedtimes":1,
            "mail":false,
            "groups":[
                "vulnerability-detector"
            ],
            "gdpr":[
                "IV_35.7.d"
            ]
        },
        "agent":{
            "id":"000",
            "name":"ip-172-0-1-226"
        },
        "manager":{
            "name":"ip-172-0-1-226"
        },
        "id":"1558014520.144351",
        "decoder":{
            "name":"json"
        },
        "data":{
            "vulnerability":{
                "cve":"CVE-2014-0209",
                "title":"libXfont: integer overflow of allocations in font metadata file parsing",
                "severity":"High",
                "published":"2014-05-13T00:00:00+00:00",
                "state":"Fixed",
                "cvss":{
                    "cvss_score":"6.900000",
                    "cvss_scoring_vector":"AV:L/AC:M/Au:N/C:C/I:C/A:C"
                },
                "package":{
                    "name":"libXfont",
                    "version":"1.4.5-5.12.amzn1",
                    "condition":"less than 1.4.7-2.el7_0"
                },
                "advisories":"RHSA-2014:1893,RHSA-2014:1870",
                "cwe_reference":"CWE-190",
                "bugzilla_reference":"https://bugzilla.redhat.com/show_bug.cgi?id=1096593",
                "reference":"https://access.redhat.com/security/cve/CVE-2014-0209"
            }
        },
        "location":"vulnerability-detector"
    }

If we go to the vulnerability ticket, we can see the patch that fixes it:

imagen

And the package updated:

imagen

@snaow

snaow approved these changes May 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.