Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(scanner/redhat): make cache before detect dnf modules #1812

Merged
merged 1 commit into from Dec 15, 2023

Conversation

wadda0714
Copy link
Contributor

@wadda0714 wadda0714 commented Dec 14, 2023

What did you implement:

When scanning for redhat, there was an error in the execution part of the following command.

dnf --nogpgcheck --cacheonly --color=never --quiet module list --enabled

The cacheonly flag was specified for this command, so that module information was retrieved by referencing the cache.
The scan was failing because the dnf --nogpgcheck --cacheonly --color=never --quiet module list --enabled was executed before the yum makecache command to create the cache was executed.
This problem does not occur if the cache is created before the scan.

[root@ip-192-168-0-193 vuls-saas]# yum clean all
Updating Subscription Management repositories.
25 files removed
[root@ip-192-168-0-193 vuls-saas]# ./vuls scan
[Dec 14 06:00:47]  INFO [localhost] vuls-v0.24.4-build-b9db541
[Dec 14 06:00:47]  INFO [localhost] Start scanning
[Dec 14 06:00:47]  INFO [localhost] config: /opt/vuls-saas/config.toml
[Dec 14 06:00:47]  INFO [localhost] Validating config...
[Dec 14 06:00:47]  INFO [localhost] Detecting Server/Container OS...
[Dec 14 06:00:47]  INFO [localhost] Detecting OS of servers...
[Dec 14 06:00:47]  INFO [localhost] (1/1) Detected: ip-192-168-0-193_ap-northeast-1_compute_internal: redhat 9.3
[Dec 14 06:00:47]  INFO [localhost] Detecting OS of containers...
[Dec 14 06:00:47]  INFO [localhost] Checking Scan Modes...
[Dec 14 06:00:47]  INFO [localhost] Detecting Platforms...
[Dec 14 06:00:48]  INFO [localhost] (1/1) ip-192-168-0-193_ap-northeast-1_compute_internal is running on aws
[Dec 14 06:00:48]  INFO [ip-192-168-0-193_ap-northeast-1_compute_internal] Scanning OS pkg in fast-root mode
[Dec 14 06:00:49] ERROR [localhost] Error on ip-192-168-0-193_ap-northeast-1_compute_internal, err: [Failed to detect installed dnf modules:
    github.com/future-architect/vuls/scanner.(*redhatBase).scanPackages
        /home/runner/work/futurevuls-backend/futurevuls-backend/scanner-src/scanner/redhatbase.go:424
  - Failed to dnf module list: execResult: servername:
      cmd: dnf --nogpgcheck --cacheonly --color=never --quiet module list --enabled
      exitstatus: 1
      stdout:
      stderr: Error: Cache-only enabled but no cache for 'rhel-9-appstream-rhui-rpms'

      err: exit status 1:
    github.com/future-architect/vuls/scanner.(*redhatBase).detectEnabledDnfModules
        /home/runner/work/futurevuls-backend/futurevuls-backend/scanner-src/scanner/redhatbase.go:949]


Scan Summary
================
ip-192-168-0-193_ap-northeast-1_compute_internal        Error           Use configtest subcommand or scan with --debug to view the details


[Dec 14 06:00:49] ERROR [localhost] Failed to scan: Failed to scan. err:
    github.com/future-architect/vuls/scanner.Scanner.Scan
        /home/runner/work/futurevuls-backend/futurevuls-backend/scanner-src/scanner/scanner.go:110
  - An error occurred on [ip-192-168-0-193_ap-northeast-1_compute_internal]

So I changed the dnf --nogpgcheck --cacheonly --color=never --quiet module list --enabled to be executed after yum makecache.

Note that if you run the scan offline, yum makecache will fail, so the scan will also fail.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

[root@ip-192-168-0-193 vuls-saas]# yum clean all
Updating Subscription Management repositories.
25 files removed
[root@ip-192-168-0-193 vuls-saas]# ./vuls_test scan
[Dec 14 06:20:19]  INFO [localhost] vuls-`make build` or `make install` will show the version-
[Dec 14 06:20:19]  INFO [localhost] Start scanning
[Dec 14 06:20:19]  INFO [localhost] config: /opt/vuls-saas/config.toml
[Dec 14 06:20:19]  INFO [localhost] Validating config...
[Dec 14 06:20:19]  INFO [localhost] Detecting Server/Container OS...
[Dec 14 06:20:19]  INFO [localhost] Detecting OS of servers...
[Dec 14 06:20:19]  INFO [localhost] (1/1) Detected: ip-192-168-0-193_ap-northeast-1_compute_internal: redhat 9.3
[Dec 14 06:20:19]  INFO [localhost] Detecting OS of containers...
[Dec 14 06:20:19]  INFO [localhost] Checking Scan Modes...
[Dec 14 06:20:19]  INFO [localhost] Detecting Platforms...
[Dec 14 06:20:19]  INFO [localhost] (1/1) ip-192-168-0-193_ap-northeast-1_compute_internal is running on aws
[Dec 14 06:20:19]  INFO [ip-192-168-0-193_ap-northeast-1_compute_internal] Scanning OS pkg in fast-root mode
[Dec 14 06:20:43]  INFO [ip-192-168-0-193_ap-northeast-1_compute_internal] Scanning listen port...
[Dec 14 06:20:43]  INFO [ip-192-168-0-193_ap-northeast-1_compute_internal] Using Port Scanner: Vuls built-in Scanner


Scan Summary
================
ip-192-168-0-193_ap-northeast-1_compute_internal        redhat9.3       430 installed, 21 updatable





To view the detail, vuls tui is useful.
To send a report, run vuls report -h.

Checklist:

You don't have to satisfy all of the following.

  • Write tests
  • Write documentation
  • Check that there aren't other open pull requests for the same issue/feature
  • Format your source code by make fmt
  • Pass the test by make test
  • Provide verification config / commands
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES

Reference

scanner/redhatbase.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@MaineK00n MaineK00n left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix this bug, shouldn't it be implemented to create a cache before checking the DNF module?

@MaineK00n
Copy link
Collaborator

MaineK00n commented Dec 14, 2023

I would write like this

diff --git a/scanner/redhatbase.go b/scanner/redhatbase.go
index 47afa67..fb2aa8d 100644
--- a/scanner/redhatbase.go
+++ b/scanner/redhatbase.go
@@ -420,10 +420,6 @@ func (o *redhatBase) scanPackages() (err error) {
 		return xerrors.Errorf("Failed to scan installed packages: %w", err)
 	}
 
-	if o.EnabledDnfModules, err = o.detectEnabledDnfModules(); err != nil {
-		return xerrors.Errorf("Failed to detect installed dnf modules: %w", err)
-	}
-
 	fn := func(pkgName string) execResult { return o.exec(fmt.Sprintf("rpm -q --last %s", pkgName), noSudo) }
 	o.Kernel.RebootRequired, err = o.rebootRequired(fn)
 	if err != nil {
@@ -433,23 +429,37 @@ func (o *redhatBase) scanPackages() (err error) {
 		// Only warning this error
 	}
 
-	if o.getServerInfo().Mode.IsOffline() {
+	if o.getServerInfo().Mode.IsOffline() || (o.Distro.Family == constant.RedHat && o.getServerInfo().Mode.IsFast()) {
 		return nil
-	} else if o.Distro.Family == constant.RedHat {
-		if o.getServerInfo().Mode.IsFast() {
-			return nil
-		}
 	}
 
-	updatable, err := o.scanUpdatablePackages()
+	updatables, modules, err := func() (models.Packages, []string, error) {
+		if err = o.yumMakeCache(); err != nil {
+			return nil, nil, xerrors.Errorf("Failed to `yum makecache`: %w", err)
+		}
+
+		modules, err := o.detectEnabledDnfModules()
+		if err != nil {
+			return nil, nil, xerrors.Errorf("Failed to detect installed dnf modules: %w", err)
+		}
+
+		updatables, err := o.scanUpdatablePackages()
+		if err != nil {
+			return nil, nil, xerrors.Errorf("Failed to scan updatable packages: %w", err)
+		}
+
+		return updatables, modules, nil
+	}()
 	if err != nil {
-		err = xerrors.Errorf("Failed to scan updatable packages: %w", err)
+		// Only warning this error
+		err = xerrors.Errorf("Failed to scan updatable packages and dnf modules: %w", err)
 		o.log.Warnf("err: %+v", err)
 		o.warns = append(o.warns, err)
-		// Only warning this error
-	} else {
-		o.Packages.MergeNewVersion(updatable)
+		return nil
 	}
+	o.Packages.MergeNewVersion(updatables)
+	o.EnabledDnfModules = modules
+
 	return nil
 }
 
@@ -645,10 +655,6 @@ func (o *redhatBase) yumMakeCache() error {
 }
 
 func (o *redhatBase) scanUpdatablePackages() (models.Packages, error) {
-	if err := o.yumMakeCache(); err != nil {
-		return nil, xerrors.Errorf("Failed to `yum makecache`: %w", err)
-	}
-
 	isDnf := o.exec(util.PrependProxyEnv(`repoquery --version | grep dnf`), o.sudo.repoquery()).isSuccess()
 	cmd := `repoquery --all --pkgnarrow=updates --qf='%{NAME} %{EPOCH} %{VERSION} %{RELEASE} %{REPO}'`
 	if isDnf {

@wadda0714 wadda0714 force-pushed the wada/fix_RHEL branch 2 times, most recently from cc9c610 to 4e01d55 Compare December 15, 2023 02:25
@wadda0714
Copy link
Contributor Author

I would write like this

diff --git a/scanner/redhatbase.go b/scanner/redhatbase.go
index 47afa67..fb2aa8d 100644
--- a/scanner/redhatbase.go
+++ b/scanner/redhatbase.go
@@ -420,10 +420,6 @@ func (o *redhatBase) scanPackages() (err error) {
 		return xerrors.Errorf("Failed to scan installed packages: %w", err)
 	}
 
-	if o.EnabledDnfModules, err = o.detectEnabledDnfModules(); err != nil {
-		return xerrors.Errorf("Failed to detect installed dnf modules: %w", err)
-	}
-
 	fn := func(pkgName string) execResult { return o.exec(fmt.Sprintf("rpm -q --last %s", pkgName), noSudo) }
 	o.Kernel.RebootRequired, err = o.rebootRequired(fn)
 	if err != nil {
@@ -433,23 +429,37 @@ func (o *redhatBase) scanPackages() (err error) {
 		// Only warning this error
 	}
 
-	if o.getServerInfo().Mode.IsOffline() {
+	if o.getServerInfo().Mode.IsOffline() || (o.Distro.Family == constant.RedHat && o.getServerInfo().Mode.IsFast()) {
 		return nil
-	} else if o.Distro.Family == constant.RedHat {
-		if o.getServerInfo().Mode.IsFast() {
-			return nil
-		}
 	}
 
-	updatable, err := o.scanUpdatablePackages()
+	updatables, modules, err := func() (models.Packages, []string, error) {
+		if err = o.yumMakeCache(); err != nil {
+			return nil, nil, xerrors.Errorf("Failed to `yum makecache`: %w", err)
+		}
+
+		modules, err := o.detectEnabledDnfModules()
+		if err != nil {
+			return nil, nil, xerrors.Errorf("Failed to detect installed dnf modules: %w", err)
+		}
+
+		updatables, err := o.scanUpdatablePackages()
+		if err != nil {
+			return nil, nil, xerrors.Errorf("Failed to scan updatable packages: %w", err)
+		}
+
+		return updatables, modules, nil
+	}()
 	if err != nil {
-		err = xerrors.Errorf("Failed to scan updatable packages: %w", err)
+		// Only warning this error
+		err = xerrors.Errorf("Failed to scan updatable packages and dnf modules: %w", err)
 		o.log.Warnf("err: %+v", err)
 		o.warns = append(o.warns, err)
-		// Only warning this error
-	} else {
-		o.Packages.MergeNewVersion(updatable)
+		return nil
 	}
+	o.Packages.MergeNewVersion(updatables)
+	o.EnabledDnfModules = modules
+
 	return nil
 }
 
@@ -645,10 +655,6 @@ func (o *redhatBase) yumMakeCache() error {
 }
 
 func (o *redhatBase) scanUpdatablePackages() (models.Packages, error) {
-	if err := o.yumMakeCache(); err != nil {
-		return nil, xerrors.Errorf("Failed to `yum makecache`: %w", err)
-	}
-
 	isDnf := o.exec(util.PrependProxyEnv(`repoquery --version | grep dnf`), o.sudo.repoquery()).isSuccess()
 	cmd := `repoquery --all --pkgnarrow=updates --qf='%{NAME} %{EPOCH} %{VERSION} %{RELEASE} %{REPO}'`
 	if isDnf {

I tested it and it looked fine, so I modified it as you suggested.
Thank you!

@MaineK00n MaineK00n force-pushed the wada/fix_RHEL branch 2 times, most recently from 123b43d to e8afbdd Compare December 15, 2023 06:45
@MaineK00n
Copy link
Collaborator

Detecting the dnf module is a necessary task regardless of offline mode or redhat fast mode.
The proposed fix for this issue was to create the cache before detecting the dnf module.

@MaineK00n MaineK00n merged commit b2c9117 into future-architect:master Dec 15, 2023
4 checks passed
@MaineK00n MaineK00n changed the title fix(scanner): yum makecache was not working when scanning redhat fix(scanner/redhat): make cache before detect dnf modules Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants