Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use fewer modprobes #2351

Merged
merged 1 commit into from
Jul 25, 2019
Merged

Use fewer modprobes #2351

merged 1 commit into from
Jul 25, 2019

Conversation

daym
Copy link
Contributor

@daym daym commented Mar 26, 2019

It's not customary for user space programs to modprobe stuff in general. Modern distributions will autoload modules (if it is indeed a module and not built-in) on access anyway, so it's also not necessary to manually modprobe stuff.

Therefore, this replaces modprobe by actual accesses of the devices. See also moby/moby#38930

logrus.Warnf("Running modprobe nf_nat failed with message: `%s`, error: %v", strings.TrimSpace(string(out)), err)
path, err := exec.LookPath("iptables")
if err != nil {
return
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this still print a warning perhaps, if iptables cannot be found?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, why not.

Note: detectIptables, where I copied that chunk from, doesn't either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, hm, yes, if it wasn't there (not sure why though 🤔 😅) (I'm not a maintainer in this repository, so perhaps there was a reason for that 😅)

@thaJeztah
Copy link
Member

ping @euanh @arkodg PTAL

@thaJeztah
Copy link
Member

ping @euanh @arkodg @mavenugo PTAL

ns/init_linux.go Outdated
if out, err := exec.Command("modprobe", "-va", "xfrm_algo").CombinedOutput(); err != nil {
return fmt.Errorf("Running modprobe xfrm_algo failed with message: `%s`, error: %v", strings.TrimSpace(string(out)), err)
}
// Those are automatically loaded when someone opens the socket anyway.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is not exported, so perhaps it should be removed altogether; looks like checkXfrmSocket() is also no longer called now, so could be removed (do we need to try that socket to force the modules to load?)

perhaps add a comment in the calling function about these being automatically loaded, e.g.;

diff --git a/ns/init_linux.go b/ns/init_linux.go
index 6a56b837..ed78bfa6 100644
--- a/ns/init_linux.go
+++ b/ns/init_linux.go
@@ -74,17 +74,10 @@ func NlHandle() *netlink.Handle {
 }
 
 func getSupportedNlFamilies() []int {
-       fams := []int{syscall.NETLINK_ROUTE}
-       // NETLINK_XFRM test
-       if err := loadXfrmModules(); err != nil {
-               if checkXfrmSocket() != nil {
-                       logrus.Warnf("Could not load necessary modules for IPSEC rules: %v", err)
-               } else {
-                       fams = append(fams, syscall.NETLINK_XFRM)
-               }
-       } else {
-               fams = append(fams, syscall.NETLINK_XFRM)
-       }
+       // No need to "modprobe" xfrm_user and xfrm_algo, as they are loaded
+       // automatically when a socket is opened
+       fams := []int{syscall.NETLINK_ROUTE, syscall.NETLINK_XFRM}
+
        // NETLINK_NETFILTER test
        if err := loadNfConntrackModules(); err != nil {
                if checkNfSocket() != nil {
@@ -99,21 +92,6 @@ func getSupportedNlFamilies() []int {
        return fams
 }
 
-func loadXfrmModules() error {
-       // Those are automatically loaded when someone opens the socket anyway.
-       return nil
-}
-
-// API check on required xfrm modules (xfrm_user, xfrm_algo)
-func checkXfrmSocket() error {
-       fd, err := syscall.Socket(syscall.AF_NETLINK, syscall.SOCK_RAW, syscall.NETLINK_XFRM)
-       if err != nil {
-               return err
-       }
-       syscall.Close(fd)
-       return nil
-}
-
 func loadNfConntrackModules() error {
        if out, err := exec.Command("modprobe", "-va", "nf_conntrack").CombinedOutput(); err != nil {
                return fmt.Errorf("Running modprobe nf_conntrack failed with message: `%s`, error: %v", strings.TrimSpace(string(out)), err)

Copy link
Contributor Author

@daym daym Apr 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. But is XFRM supposed to be optional? maybe checkXfrmSocket was supposed to be called still.

Frankly, I hadn't seen the part

        if err := loadXfrmModules(); err != nil {

Maybe it would be better to call checkXfrmSocket anyway--and warn on error.

If the XFRM modules are mandatory for this module anyway, I agree with your approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, was suspecting the same, hence my question above;

do we need to try that socket to force the modules to load?

So probably call it unconditionally, and indeed produce an error on failure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the branch to do that.

Signed-off-by: Danny Milosavljevic <dannym@scratchpost.org>
@thaJeztah
Copy link
Member

ping @arkodg @kolyshkin PTAL

Copy link
Contributor

@kolyshkin kolyshkin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@selansen selansen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@selansen
Copy link
Collaborator

Thanks @daym for working on this PR.
Thanks @thaJeztah @kolyshkin for reviews

@selansen selansen merged commit 5eb06eb into moby:master Jul 25, 2019
thaJeztah added a commit to thaJeztah/docker that referenced this pull request Jul 30, 2019
full diff: moby/libnetwork@83d30db...09cdcc8

changes included:

- moby/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- moby/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes moby/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- moby/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes moby/libnetwork#2413 Network with --config-from does not honor --internal
- moby/libnetwork#2351 Use fewer modprobes
  - relates to moby#38930 Use fewer modprobes
- moby/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of moby/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes moby/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker-archive/docker-ce that referenced this pull request Jul 31, 2019
full diff: moby/libnetwork@83d30db...09cdcc8

changes included:

- moby/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- moby/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes moby/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- moby/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes moby/libnetwork#2413 Network with --config-from does not honor --internal
- moby/libnetwork#2351 Use fewer modprobes
  - relates to moby/moby#38930 Use fewer modprobes
- moby/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of moby/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes moby/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 6f234db9fef23c591d8376f96db062e7107b658f
Component: engine
thaJeztah added a commit to thaJeztah/docker that referenced this pull request Sep 16, 2019
full diff: moby/libnetwork@83d30db...09cdcc8

changes included:

- moby/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- moby/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes moby/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- moby/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes moby/libnetwork#2413 Network with --config-from does not honor --internal
- moby/libnetwork#2351 Use fewer modprobes
  - relates to moby#38930 Use fewer modprobes
- moby/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of moby/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes moby/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f234db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker-archive/docker-ce that referenced this pull request Sep 17, 2019
full diff: moby/libnetwork@83d30db...09cdcc8

changes included:

- moby/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- moby/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes moby/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- moby/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes moby/libnetwork#2413 Network with --config-from does not honor --internal
- moby/libnetwork#2351 Use fewer modprobes
  - relates to moby/moby#38930 Use fewer modprobes
- moby/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of moby/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes moby/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f234db9fef23c591d8376f96db062e7107b658f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: b6190c2713623ab455d29da4771b684e4eafc63f
Component: engine
@daym daym deleted the fewer-modprobes branch October 2, 2019 21:18
burnMyDread pushed a commit to burnMyDread/moby that referenced this pull request Oct 21, 2019
full diff: moby/libnetwork@83d30db...09cdcc8

changes included:

- moby/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- moby/libnetwork#2411 Macvlan network handles netlabel.Internal wrong
  - fixes moby/libnetwork#2410 Macvlan network handles netlabel.Internal wrong
- moby/libnetwork#2414 Allow network with --config-from to be --internal
  - fixes moby/libnetwork#2413 Network with --config-from does not honor --internal
- moby/libnetwork#2351 Use fewer modprobes
  - relates to moby#38930 Use fewer modprobes
- moby/libnetwork#2415 Support dockerd and system restarts for ipvlan and macvlan networks
  - carry of moby/libnetwork#2295 phantom ip/mac vlan network after a powercycle
  - fixes moby/libnetwork#1743 Phantom docker network

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: zach <Zachary.Joyner@linux.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants