Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(server): Apply filters for IgnoreCves and PkgsRegexps on server mode #1270

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

marklofff
Copy link

@marklofff marklofff commented Jul 8, 2021

What did you implement:

Closes #1267

The default settings should be applied to vuls server scans as well.

I applied the filters for ignoreCves and PkgsRegexps to the result from server mode.

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

How to reproduce this?

To get the same kind of results, simply run the vuls server command with the same config on listed below on run the same POST command.

  • Tested with the listed config below
[default]
ignorePkgsRegexp = [
  "^openssh"
]

ignoreCves = [
  "CVE-2004-0230"
]

[servers.via-server]
host = "localhost"
port = "local"
  • Commands that were used to test
dpkg-query -W -f='${binary:Package},${db:Status-Abbrev},${Version},${Source},${source:Version}\n' |
curl -sS -o - -X POST \
  -H "content-type: text/plain" -H 'X-Vuls-OS-Family: debian' \
  -H "X-Vuls-Server-Name: via-server" \
  -H "X-Vuls-OS-Release: $(cat /etc/debian_version)" \
  -H "X-Vuls-Kernel-Release: $(uname -r)" \
  -H "X-Vuls-Kernel-Version: $(uname -a | awk '{print $7}')" \
  http://127.0.0.1:5515/vuls \
  --data-binary @- | jq . > via-server-without-openssh.json

Then, Checked the result by myself.

Results for ignoreCves

  • before
❯ grep CVE-2004-0230 via-server.json
      "CVE-2004-0230": {
        "cveID": "CVE-2004-0230",
            "cveID": "CVE-2004-0230",
            "sourceLink": "https://security-tracker.debian.org/tracker/CVE-2004-0230",
            "cveID": "CVE-2004-0230",
                "link": "http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2004-0230",
                "link": "http://nvd.nist.gov/nvd.cfm?cvename=CVE-2004-0230",
            "cveID": "CVE-2004-0230",
            "sourceLink": "https://nvd.nist.gov/vuln/detail/CVE-2004-0230",
  • after
❯ grep CVE-2004-0230 via-server.json
empty

Results for PkgsRegexps

CVE-2019-16905 is one of the cves that includes openssh

  • before
❯ grep CVE-2019-16905 via-server-without-opnessh.json
      "CVE-2019-16905": {
        "cveID": "CVE-2019-16905",
            "cveID": "CVE-2019-16905",
            "sourceLink": "https://security-tracker.debian.org/tracker/CVE-2019-16905",
            "cveID": "CVE-2019-16905",
                "link": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-16905",
                "link": "https://nvd.nist.gov/vuln/detail/CVE-2019-16905",
            "cveID": "CVE-2019-16905",
            "sourceLink": "https://nvd.nist.gov/vuln/detail/CVE-2019-16905",
  • after
❯ grep CVE-2019-16905 via-server-without-opnessh.json
empty

Checklist:

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES

References

Copy link
Collaborator

@MaineK00n MaineK00n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the template for the config.toml generated by the discover command. There is no example of ignorePkgsRegexp here. I would like to have it added.

vuls/subcmds/discover.go

Lines 80 to 238 in f0b3a8b

# https://vuls.io/docs/en/config.toml.html#database-section
[cveDict]
#type = ["sqlite3", "mysql", "postgres", "redis", "http" ]
#sqlite3Path = "/path/to/cve.sqlite3"
#url = ""
[ovalDict]
#type = ["sqlite3", "mysql", "postgres", "redis", "http" ]
#sqlite3Path = "/path/to/oval.sqlite3"
#url = ""
[gost]
#type = ["sqlite3", "mysql", "postgres", "redis", "http" ]
#sqlite3Path = "/path/to/gost.sqlite3"
#url = ""
[exploit]
#type = ["sqlite3", "mysql", "postgres", "redis", "http" ]
#sqlite3Path = "/path/to/go-exploitdb.sqlite3"
#url = ""
[metasploit]
#type = ["sqlite3", "mysql", "postgres", "redis", "http" ]
#sqlite3Path = "/path/to/go-msfdb.sqlite3"
#url = ""
# https://vuls.io/docs/en/config.toml.html#slack-section
#[slack]
#hookURL = "https://hooks.slack.com/services/abc123/defghijklmnopqrstuvwxyz"
##legacyToken = "xoxp-11111111111-222222222222-3333333333"
#channel = "#channel-name"
##channel = "${servername}"
#iconEmoji = ":ghost:"
#authUser = "username"
#notifyUsers = ["@username"]
# https://vuls.io/docs/en/config.toml.html#email-section
#[email]
#smtpAddr = "smtp.example.com"
#smtpPort = "587"
#user = "username"
#password = "password"
#from = "from@example.com"
#to = ["to@example.com"]
#cc = ["cc@example.com"]
#subjectPrefix = "[vuls]"
# https://vuls.io/docs/en/config.toml.html#http-section
#[http]
#url = "http://localhost:11234"
# https://vuls.io/docs/en/config.toml.html#syslog-section
#[syslog]
#protocol = "tcp"
#host = "localhost"
#port = "514"
#tag = "vuls"
#facility = "local0"
#severity = "alert"
#verbose = false
# https://vuls.io/docs/en/usage-report.html#example-put-results-in-s3-bucket
#[aws]
#profile = "default"
#region = "ap-northeast-1"
#s3Bucket = "vuls"
#s3ResultsDir = "/path/to/result"
#s3ServerSideEncryption = "AES256"
# https://vuls.io/docs/en/usage-report.html#example-put-results-in-azure-blob-storage<Paste>
#[azure]
#accountName = "default"
#accountKey = "xxxxxxxxxxxxxx"
#containerName = "vuls"
# https://vuls.io/docs/en/config.toml.html#chatwork-section
#[chatwork]
#room = "xxxxxxxxxxx"
#apiToken = "xxxxxxxxxxxxxxxxxx"
# https://vuls.io/docs/en/config.toml.html#googlechat-section
#[googlechat]
#webHookURL = "https://chat.googleapis.com/v1/spaces/xxxxxxxxxx/messages?key=yyyyyyyyyy&token=zzzzzzzzzz%3D"
#skipIfNoCve = false
#serverNameRegexp = "^(\\[Reboot Required\\] )?((spam|ham).*|.*(egg)$)" # include spamonigiri, hamburger, boiledegg
#serverNameRegexp = "^(\\[Reboot Required\\] )?(?:(spam|ham).*|.*(?:egg)$)" # exclude spamonigiri, hamburger, boiledegg
# https://vuls.io/docs/en/config.toml.html#telegram-section
#[telegram]
#chatID = "xxxxxxxxxxx"
#token = "xxxxxxxxxxxxxxxxxx"
#[wpscan]
#token = "xxxxxxxxxxx"
#detectInactive = false
# https://vuls.io/docs/en/config.toml.html#default-section
[default]
#port = "22"
#user = "username"
#keyPath = "/home/username/.ssh/id_rsa"
#scanMode = ["fast", "fast-root", "deep", "offline"]
#scanModules = ["ospkg", "wordpress", "lockfile", "port"]
#cpeNames = [
# "cpe:/a:rubyonrails:ruby_on_rails:4.2.1",
#]
#owaspDCXMLPath = "/tmp/dependency-check-report.xml"
#ignoreCves = ["CVE-2014-6271"]
#containersOnly = false
#containerType = "docker" #or "lxd" or "lxc" default: docker
#containersIncluded = ["${running}"]
#containersExcluded = ["container_name_a"]
# https://vuls.io/docs/en/config.toml.html#servers-section
[servers]
{{- $names:= .Names}}
{{range $i, $ip := .IPs}}
[servers.{{index $names $i}}]
host = "{{$ip}}"
#port = "22"
#user = "root"
#sshConfigPath = "/home/username/.ssh/config"
#keyPath = "/home/username/.ssh/id_rsa"
#scanMode = ["fast", "fast-root", "deep", "offline"]
#scanModules = ["ospkg", "wordpress", "lockfile", "port"]
#type = "pseudo"
#memo = "DB Server"
#cpeNames = [ "cpe:/a:rubyonrails:ruby_on_rails:4.2.1" ]
#owaspDCXMLPath = "/path/to/dependency-check-report.xml"
#ignoreCves = ["CVE-2014-0160"]
#containersOnly = false
#containerType = "docker" #or "lxd" or "lxc" default: docker
#containersIncluded = ["${running}"]
#containersExcluded = ["container_name_a"]
#[servers.{{index $names $i}}.containers.container_name_a]
#cpeNames = [ "cpe:/a:rubyonrails:ruby_on_rails:4.2.1" ]
#owaspDCXMLPath = "/path/to/dependency-check-report.xml"
#ignoreCves = ["CVE-2014-0160"]
#[servers.{{index $names $i}}.githubs."owner/repo"]
#token = "yourToken"
#ignoreGitHubDismissed = false
#[servers.{{index $names $i}}.wordpress]
#cmdPath = "/usr/local/bin/wp"
#osUser = "wordpress"
#docRoot = "/path/to/DocumentRoot/"
#[servers.{{index $names $i}}.portscan]
#scannerBinPath = "/usr/bin/nmap"
#hasPrivileged = true
#scanTechniques = ["sS"]
#sourcePort = "65535"
#[servers.{{index $names $i}}.optional]
#key = "value1"
{{end}}

There is little description of ignorePkgsRegexp even in vulsdoc/vuls. In particular, there is no explanation in the config.toml section. I would like to see the explanation added here as well.

Here are the links to the parts that I need clarification on. However, I suspect that these are not all the parts that need to be explained, so please add explanations where you think they are needed.
https://github.com/vulsdoc/vuls/blob/47f6cedb39ef59f823e06ecd5cd1b98d8d91272f/docs/configtoml.md
https://github.com/vulsdoc/vuls/blob/47f6cedb39ef59f823e06ecd5cd1b98d8d91272f/docs/usage-automatic-discovery.md

Copy link
Collaborator

@MaineK00n MaineK00n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I look at the options for server mode, I see -ignore-unscored-cves and -ignore-unfixed. Can't you also support this?

$ vuls server -h
...
  -ignore-unfixed
    	Don't show the unfixed CVEs
  -ignore-unscored-cves
    	Don't Server the unscored CVEs
...

The report seems to be handled as follows.

r.ScannedCves = r.ScannedCves.FilterUnfixed(config.Conf.IgnoreUnfixed)

vuls/detector/detector.go

Lines 156 to 159 in f0b3a8b

// IgnoreUnscored
if config.Conf.IgnoreUnscoredCves {
r.ScannedCves = r.ScannedCves.FindScoredVulns()
}

@marklofff
Copy link
Author

Can't you also support this?

what exactly do you want me to implement here?

@MaineK00n
Copy link
Collaborator

Can't you also support this?

what exactly do you want me to implement here?

I am sorry that my English is not good enough to convey my intentions.
The following is an additional task.
If you don't want to be included in this PR, or don't feel the need to do it, please express your intention and I will do it for you.
Please don't hesitate to tell me.

There are three tasks that I would like you to do in this PR.

The first is to add an example of ignorePkgsRegexp to the discover command.
The second is to add a description of ignorePkgsRegexp in the section describing config.toml of vulsdoc/vuls.
Third, -ignore-unscored-cves and -ignore-unfixed are available as options of the server command, but they don't work because they are not implemented in the server command. This may be the same as ignorePkgsRegexp.

@marklofff
Copy link
Author

Thanks for the reply! <3

I am sorry that my English is not good enough to convey my intentions.

Your English is totally fine!! i just wanted more context which you added for me! thanks!

If you don't want to be included in this PR, or don't feel the need to do it, please express your intention and I will do it for you.

I would love to work on this so ill try to fix the -ignore-unscored-cves and -ignore-unfixed options to work.
Thanksss!!

@marklofff marklofff requested a review from MaineK00n July 30, 2021 05:32
Copy link
Collaborator

@MaineK00n MaineK00n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First, thanks for the PR to vulsdoc/vuls.

I saw commit:93729c6e73fef787c2d349e1ee1a404a32824d6c.
Thanks for the implementation of -ignore-unscored-cves and -ignore-unfixed.
I commented on the implementation questions.
Also, please write how you verified this feature and the results in the PR section How Has This Been Tested?.

The first is to add an example of ignorePkgsRegexp to the discover command.

Please add #ignorePkgsRegexp = ["^kernel", "^python"] below the link line.

Comment on lines -73 to +80
f.BoolVar(&config.Conf.IgnoreUnscoredCves, "ignore-unscored-cves", false,
"Don't Server the unscored CVEs")

f.BoolVar(&config.Conf.IgnoreUnfixed, "ignore-unfixed", false,
f.BoolVar(&p.ignoreUnfixed, "ignore-unfixed", false,
"Don't show the unfixed CVEs")

f.BoolVar(&p.ignoreUnscoredCves, "ignore-unscored-cves", false,
"Don't show the unscored CVEs")

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you add it to serverCmd to manage option states instead of config.Conf.IgnoreUnfixed and config.Conf.IgnoreUnscoredCves.
https://github.com/qwexvf/vuls/blob/93729c6e73fef787c2d349e1ee1a404a32824d6c/config/config.go#L77-L78

r.ScannedCves = r.ScannedCves.FilterIgnorePkgs(ignorePkgsRegexps)

// IgnoreUnfixed
r.ScannedCves = r.ScannedCves.FilterUnfixed(h.IgnoreUnfixed)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it possible to use config.Conf.IgnoreUnfixed?

r.ScannedCves = r.ScannedCves.FilterUnfixed(h.IgnoreUnfixed)

// IgnoreUnscoredCves
if h.IgnoreUnscoredCves {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it possible to use config.Conf.IgnoreUnscoredCves?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

server mode doesn't use default section ignoreCves/ignorePkgsRegexp
2 participants