Skip to content

@kotakanbe kotakanbe released this Aug 27, 2018 · 135 commits to master since this release

Clever vulnerability detection of non-OS-packages

https://vuls.io/docs/en/usage-scan-non-os-packages.html

Before v0.5.0, Vuls can detect only by exact match of CPE.
After v0.5.0, Vuls supports NVD JSON Feed
NVD JSON Feed has detailed affected version information.

        "cpe" : [ {
          "vulnerable" : true,
          "cpe22Uri" : "cpe:/a:ibm:security_guardium",
          "cpe23Uri" : "cpe:2.3:a:ibm:security_guardium:*:*:*:*:*:*:*:*",
          "versionEndIncluding" : "8.2"
        }, {

Using this information, Vuls can detect vulnerabilities of non-OS-packages more cleverly than before, because vuls compares versions of specified CPEs in config.toml and the version in the feed.

related ( #599 )


Significant improvement in scanning

Support knqyf263/gost (go-security-tracker)

New version Vuls 0.5.0 now possible to detect vulnerabilities that patches have not been published from distributors using new datasource named gost.

RedHat API

before

One Line Summary
================
c74     Total: 38 (High:12 Medium:18 Low:7 ?:1) 36/36 Fixed     708 installed, 288 updatable

after (with gost)

One Line Summary
================
c74     Total: 459 (High:48 Medium:326 Low:84 ?:1)      36/457 Fixed    708 installed, 288 updatable

Debian Security Tracker

before

One Line Summary
================
deb8    Total: 105 (High:22 Medium:39 Low:9 ?:35)       55/103 Fixed    513 installed, 43 updatable

after (with gost)

One Line Summary
================
deb8    Total: 524 (High:56 Medium:148 Low:21 ?:299)    53/522 Fixed    512 installed, 43 updatable


HTTP Server mode - One-Liner Scanning

Vuls can scan vulnerabilities with one-liner like below.

$ curl -X POST -H "Content-Type: text/plain" \
    -H "X-Vuls-OS-Family: centos" \
    -H "X-Vuls-OS-Release: 6.9" \
    -H "X-Vuls-Kernel-Release: 2.6.32-696.30.1.el6.x86_64" \
    --data-binary "`rpm -qa --queryformat "%{NAME} %{EPOCHNUM} %{VERSION} %{RELEASE} %{ARCH}\n"`" \
    http://localhost:5515/vuls

For detials, see the Doc


Enable to define scan mode for each servers in config.tom

#510 #669

fast scan with internet access

  • config.toml
[servers]

[servers.localhost]
host         = "192.168.100.111" # or "127.0.0.1"
port         = "22"
scanMode     = ["fast"]

fast scan without internet access

  • config.toml
[servers]

[servers.localhost]
host         = "192.168.100.111" # or "127.0.0.1"
port         = "22"
scanMode     = ["fast", "offline"]

fast-root scan

fast-root scan with internet access

  • config.toml
[servers]

[servers.localhost]
host         = "192.168.100.111" # or "127.0.0.1"
port         = "22"
scanMode     = ["fast-root"]

fast-root scan without internet access

  • config.toml
[servers]

[servers.localhost]
host         = "192.168.100.111" # or "127.0.0.1"
port         = "22"
scanMode     = ["fast-root", "offline"]

deep scan

deep scan with internet access

  • config.toml
[servers]

[servers.localhost]
host         = "192.168.100.111" # or "127.0.0.1"
port         = "22"
scanMode     = ["deep"]

Reports

The following items were added

  • Display CWE name
  • OWASP TOP 10
  • Affected Packages, Processes
  • Mitigation

TUI

screen shot 2018-07-24 at 16 59 28


report -format-full-text

screen shot 2018-07-25 at 22 26 50


report -format-list

screen shot 2018-07-24 at 23 21 43


Slack Notification

screen shot 2018-07-18 at 21 24 40


Stride Notification

https://vuls.io/docs/en/usage-report.html#example-send-scan-results-to-stride
#624
2018-03-31 14 52 57


Charwork Notification

#634
https://vuls.io/docs/en/usage-report.html#example-send-scan-results-to-chatwork

2018-04-25 17 35 42


go-cve-dictionary list subcommand

kotakanbe/go-cve-dictionary#80

screen shot 2018-08-23 at 14 43 27


Change the format of ScanResult

Model has changed.
https://github.com/future-architect/vuls/tree/master/models


Change the format of config.toml

$ ./vuls discover 127.0.0.1/32
# Create config.toml using below and then ./vuls -config=/path/to/config.toml

[cveDict]
type = "sqlite3"
path = "/path/to/cve.sqlite3"
#url = ""

[ovalDict]
type = "sqlite3"
path = "/path/to/oval.sqlite3"
#url = ""

[gost]
type = "sqlite3"
path = "/path/to/gost.sqlite3"
#url = ""

# https://vuls.io/docs/en/usage-settings.html#slack-section
#[slack]
#hookURL      = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz"
##legacyToken = "xoxp-11111111111-222222222222-3333333333"
#channel      = "#channel-name"
##channel     = "${servername}"
#iconEmoji    = ":ghost:"
#authUser     = "username"
#notifyUsers  = ["@username"]

# https://vuls.io/docs/en/usage-settings.html#email-section
#[email]
#smtpAddr      = "smtp.example.com"
#smtpPort      = "587"
#user          = "username"
#password      = "password"
#from          = "from@example.com"
#to            = ["to@example.com"]
#cc            = ["cc@example.com"]
#subjectPrefix = "[vuls]"

# https://vuls.io/docs/en/usage-settings.html#http-section
#[http]
#url = "http://localhost:11234"

# https://vuls.io/docs/en/usage-settings.html#syslog-section
#[syslog]
#protocol    = "tcp"
#host        = "localhost"
#port        = "514"
#tag         = "vuls"
#facility    = "local0"
#severity    = "alert"
#verbose     = false

# https://vuls.io/docs/en/usage-report.html#example-put-results-in-s3-bucket
#[aws]
#profile                = "default"
#region                 = "ap-northeast-1"
#s3Bucket               = "vuls"
#s3ResultsDir           = "/path/to/result"
#s3ServerSideEncryption = "AES256"

# https://vuls.io/docs/en/usage-report.html#example-put-results-in-azure-blob-storage<Paste>
#[azure]
#accountName   = "default"
#accountKey    = "xxxxxxxxxxxxxx"
#containerName = "vuls"

# https://vuls.io/docs/en/usage-settings.html#stride-section
#[stride]
#hookURL   = "xxxxxxxxxxxxxxx"
#authToken = "xxxxxxxxxxxxxx"

# https://vuls.io/docs/en/usage-settings.html#hipchat-section
#[hipchat]
#room      = "vuls"
#authToken = "xxxxxxxxxxxxxx"

# https://vuls.io/docs/en/usage-settings.html#chatwork-section
#[chatwork]
#room     = "xxxxxxxxxxx"
#apiToken = "xxxxxxxxxxxxxxxxxx"

# https://vuls.io/docs/en/usage-settings.html#default-section
[default]
#port               = "22"
#user               = "username"
#keyPath            = "/home/username/.ssh/id_rsa"
#scanMode           = ["fast", "fast-root", "deep", "offline"]
#cpeNames = [
#  "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
#]
#owaspDCXMLPath     = "/tmp/dependency-check-report.xml"
#ignoreCves         = ["CVE-2014-6271"]
#containerType      = "docker" #or "lxd" or "lxc" default: docker
#containersIncluded = ["${running}"]
#containersExcluded = ["container_name_a"]

# https://vuls.io/docs/en/usage-settings.html#servers-section
[servers]

[servers.127-0-0-1]
host                = "127.0.0.1"
#port               = "22"
#user               = "root"
#keyPath            = "/home/username/.ssh/id_rsa"
#scanMode           = ["fast", "fast-root", "deep", "offline"]
#type               = "pseudo"
#memo               = "DB Server"
#cpeNames            = [ "cpe:/a:rubyonrails:ruby_on_rails:4.2.1" ]
#owaspDCXMLPath     = "/path/to/dependency-check-report.xml"
#ignoreCves         = ["CVE-2014-0160"]
#containerType      = "docker" #or "lxd" or "lxc" default: docker
#containersIncluded = ["${running}"]
#containersExcluded = ["container_name_a"]

#[servers.127-0-0-1.containers.container_name_a]
#cpeNames        = [ "cpe:/a:rubyonrails:ruby_on_rails:4.2.1" ]
#owaspDCXMLPath = "/path/to/dependency-check-report.xml"
#ignoreCves     = ["CVE-2014-0160"]

#[servers.127-0-0-1.optional]
#key = "value1"

Enable to define CpeNames for each container.

#668

To detect the vulnerability of Ruby on Rails v4.2.1 on specific container, cpeNames needs to be set in the servers>containers section.
The following is an example of running Ruby on Rails v4.2.1 on dockerA.

[servers]

[servers.172-31-4-82]
host         = "172.31.4.82"
user        = "ec2-user"
keyPath     = "/home/username/.ssh/id_rsa"
containerType = "docker"
containersIncluded = ["${running}"]

[servers.172-31-4-82.containers.dockerA]
cpeNames = [
    "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
]

Enable to define a path of OWASP Dependency Check XML for each container.

#667

The following is an example of how to specify a XML of OWASP DC to the specific container.

[servers]

[servers.172-31-4-82]
host         = "172.31.4.82"
user        = "ec2-user"
keyPath     = "/home/username/.ssh/id_rsa"
containerType = "docker"
containersIncluded = ["${running}"]

[servers.172-31-4-82.containers.dockerA]
owaspDCXMLPath = "/tmp/dependency-check-report.xml"

Enable to define ignoreCVEs for each container.

#666

[servers]

[servers.172-31-4-82]
host         = "172.31.4.82"
user        = "ec2-user"
keyPath     = "/home/username/.ssh/id_rsa"
containerType = "docker"
containersIncluded = ["${running}"]

[servers.172-31-4-82.containers.dockerA]
ignoreCves = ["CVE-2016-6314"]

Add ignorePkgsRegexp for each host, container in config.toml

#665

Format

[servers.192-168-11-6]
host         = "192.168.11.6"
ignorePkgsRegexp = ["^kernel.*"]
containersIncluded = ["${running}"]
ContainersExcluded= ["container_name_a"]

[servers.192-168-11-6.containers.container_name_a]
ignorePkgsRegexp = ["^vim.*"]

  • Ignore vulnerabilities in packages matching to patterns defined by ignorePkgs in config.toml
  • Can be defined for both host and container
  • Can be described with regular expressions
  • Match with pkgname

optional field

before

[servers.172-31-4-82]
host         = "172.31.4.82"
optional = [
    ["key", "value"],
    ["key2", "value2"],
]

after

[servers.172-31-4-82]
host         = "172.31.4.82"
[servers.172-31-4-82.Optional]
key = "value"
key2 = "value2"

(fast-root and deep) Detect processes affected by next update using yum-ps #482

Issues yum ps to detect processed affected by next software update on RedHat, CentOS, Amazon , Oracle.
AffectedProcs key under Packages will be added by this P/R.

  • result.json
{
  "Packages" : {
    "wpa_supplicant": {
      ...,
      "AffectedProcs": [
        {
          "PID": "638",
          "ProcName": "wpa_supplicant",
        }
      ]
    },
  },
}

(fast-root and deep) Detect need-restarting-processes for Redhat, CentOS, Ubuntu and Debian


Add -uuid option to report subcommand

When Servername was changed, Vulsrepo etc could not look back in the past. So, I added a unique UUID that will not be changed. If you report with this flag on, Vuls generates UUIDs automatically.
This Auto-generated UUID will be added to the config.toml automatically and result JSON.
The previous config.toml will be renamed to config.toml.bak.

Add memo field to Server section

When reporting with the -uuid flag on, config.toml is automatically recreated as described above. So I added the memo field because the TOML comment disappears. For memo about the server, please use this field, not TOML comment.


How to install

How to upgrade

Changelog

44fa2c5 v0.5.0 (no backwards compatibility) (#478)
d785fc2 Lint (#700)
ea800e0 fix(report): generate report even if some scan-err-jsons are included #685 (#686)
fe582ac Change GitHub templates
330edb3 change copyright (#677)
212fec7 Remove old Dockerfile (#684)
24d7021 Refactor Dockerfile (#683)
e3a01ff fix(report): database is locked with SQLite3 backend #681 (#682)
81f2ba8 fix(report): record not found on reporting with OVAL #679 (#680)
9e9370b refactor(suse): add testcase for detectSUSE (#675)
ced6114 pull request to add SLES variant OS SLES_SAP support (#672)
3144faa feat(syslog): add all CVSS scores/vectors (#664)
8960c67 fix(report): use CVSS score not calculated from severity preferentially (#663)
f8ca924 Add title to syslog (#662)
399a087 feat(scan): add -ssh-config option #417 (#660)
92f36ca Add missing ca-certificates, needed for slack webhook (#657)
3dcc582 Move to alpine based docker images (#643)
0977996 Fix(reporting): NotFixedYet of SourcePackage in OVAL match on Debian and Ubuntu (#656)
9cc7877 fix(configtest): Only warning when reboot-notifier is not installed on Debian (#654)
f653ca9 Don't check reboot-notifier package for debian containers (#642)
6f9fd91 Send logs via syslog when no CVE-IDs found (#646)
cb1aec4 Add scanned_at into syslog report (#641)
7cebaf8 Use servername for SSH ControlPath filename (#640)
241c943 fix(tui): show CVSS severity on TUI for Ubuntu (#638)
d5d88d8 Refactor stride (#637)
cf9d260 Update README.md (#631)
308a93d misspell (#632)
d6a7e65 [refactor]make fmt
e0a5c5d refactoring : hipchat (#635)
314f775 Chatwork support (#634)
7a16441 Stride support (#624)
5076326 Fix Amazon Linux 2 scanning (#630)
ce56261 fix(redhat): fix detection method of changelog scan (#628)
baa0e89 fix: a bug of diff logic when multiple oval defs found for a certain CVE-ID and same updated_at (#627)
1d49c0e fix(scan): fix RHEL 5 (#626)
08755e4 fix(fmt): fix gofmt warn (#625)
bb12d9d Add diff to TUI (#620)
fd1429f Fix diff logic (#619)
d3c421a inform new release on diff option (#614)
0c919da fix: change ControlPath to .vuls of SSH option (#618)
9afbf12 feat: Add -vvv option to scan cmd (#617)
50b105c fix: SSH session multiplexing (#616)
028508c fix link nvd on hipchat (#613)
f0137a3 feat: Display pkg information to slack notification #611 (#612)
e6d3a17 fix: validation for reporting (#610)
86ba551 fix: remove a validation of hipchat (#609)
26418be hipchat support (#593)
092a19b fix: bug of report -diff option (#607)
6d33985 fix: support CentOS cloud image (#606)
b08969a Support a reporting via Syslog (#604)

Assets 4
You can’t perform that action at this time.