[OpenSUSE] IPV6 isn’t disabled by Yast and/or sysctl config

[tyukh]

Problem:

rockstor:~ # ip a 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 
    inet 127.0.0.1/8 scope host lo 
       valid_lft forever preferred_lft forever 
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000 
    link/ether b0:5a:da:87:90:c3 brd ff:ff:ff:ff:ff:ff 
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000 
    link/ether b0:5a:da:87:90:c3 brd ff:ff:ff:ff:ff:ff 
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 
    link/ether b0:5a:da:87:90:c3 brd ff:ff:ff:ff:ff:ff 
    inet 192.168.1.10/24 brd 192.168.1.255 scope global dynamic noprefixroute bond0 
       valid_lft 86229sec preferred_lft 86229sec 
rockstor:~ # lsof -i 
COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME 
NetworkMa 1180   root   22u  IPv4  34099      0t0  UDP rockstor.local:bootpc->192.168.1.1:bootps 
chronyd   1214 chrony    5u  IPv4  30050      0t0  UDP localhost:323 
chronyd   1214 chrony    6u  IPv6  30052      0t0  UDP localhost:323 
sshd      1217   root    3u  IPv4  30163      0t0  TCP *:ssh (LISTEN) 
sshd      1217   root    4u  IPv6  30174      0t0  TCP *:ssh (LISTEN) 
sshd      1611   root    4u  IPv4  33710      0t0  TCP rockstor.local:ssh->DESKTOP-IPI71Q1.local:53257 (ESTABLISHED) 
rockstor:~ # ss -tulwn 
Netid      State       Recv-Q      Send-Q            Local Address:Port             Peer Address:Port      Process 
udp        UNCONN      0           0                     127.0.0.1:323                   0.0.0.0:* 
udp        UNCONN      0           0                         [::1]:323                      [::]:* 
tcp        LISTEN      0           128                     0.0.0.0:22                    0.0.0.0:* 
tcp        LISTEN      0           244                   127.0.0.1:5432                  0.0.0.0:* 
tcp        LISTEN      0           511                     0.0.0.0:443                   0.0.0.0:* 
tcp        LISTEN      0           2048                  127.0.0.1:8000                  0.0.0.0:* 
tcp        LISTEN      0           256                     0.0.0.0:8001                  0.0.0.0:* 
tcp        LISTEN      0           128                        [::]:22                       [::]:* 

Effects (need verification):
Docker: cannot load an image from the repository on the first run attempt

Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: dial udp [::1]:53: connect: cannot assign requested address 

Postfix: get in error when adding email account in Rockstor's Email alerts section

Traceback (most recent call last): 
File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception yield File "/opt/rockstor/src/rockstor/storageadmin/views/email_client.py", line 291, in post update_sasl(smtp_server, port, username, password) 
File "/opt/rockstor/src/rockstor/storageadmin/views/email_client.py", line 179, in update_sasl run_command([POSTMAP, sasl_file]) File "/opt/rockstor/src/rockstor/system/osi.py", line 176, in run_command raise CommandException(cmd, out, err, rc) 
CommandException: Error running a command. cmd = /usr/sbin/postmap /etc/postfix/sasl_passwd. rc = 1. stdout = ['']. stderr = ['postmap: fatal: parameter inet_interfaces: no local interface found for ::1', ''] 

Solution:
Edit /etc/default/grub and append ipv6.disable=1 to GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="ipv6.disable=1"
Now, need to regenerate the grub.cfg file by running the grub2-mkconfig command :
grub2-mkconfig -o /boot/grub2/grub.cfg

[phillxnet]

@tyukh Thanks for another find and fix report. Much appreciated.
Re the system wide ipv6 disable via grub:

Solution:
Edit /etc/default/grub and append ipv6.disable=1 to GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="ipv6.disable=1"

I think I'd rather go the sysctl route as we already do some editing there in initrock I believe where as we currently don't mess with grub and I'd rather leave it that way if possible.

Do you have, or can you provide, the equivalent sysctl entries to globally disable ipv6? As for Yast, our current plan is to have our installer based on JeOS Leap15.1 so there is no Yast by default.

Nice find and I'm surprised to see the email issue in there. But we can presumably build on the recent changes in that area.

[phillxnet]

@tyukh OK just had a quick look and our existing sysctl edits originate in initrock but are centered around our flash_optimize.py file:

https://github.com/rockstor/rockstor-core/blob/master/src/rockstor/scripts/flash_optimize.py

So we could do another script like this that in turn adds/edits the required ipv6 disable commands, and likewise is called from initrock so we have the central orchestration still.

[tyukh]

Nice find and I'm surprised to see the email issue in there. But we can presumably build on the recent changes in that area.

I did not check it in Leap, most likely it (email issue) is specific to the Tumbleweed. I decided to give Tumbleweed one more chance, got a lot, ahem, entertainment.

Do you have, or can you provide, the equivalent sysctl entries to globally disable ipv6? As for Yast, our current plan is to have our installer based on JeOS Leap15.1 so there is no Yast by default.

I mentioned Yast simply because the topic mentions disabling IPV6 with Yast. Yast itself simply adds a line net.ipv6.conf.all.disable_ipv6 = 1 to the sysctl conf files. However, the use altogether of the

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

does not work, some services ignore this way of disabling and continue to try to use the IPV6. For example, Docker in both branches (Leap and Tumbleweed) and Postfix in Tumbleweed. The only thing that completely disables IPV6 is the kernel argument.

[FroggyFlox]

Thanks a lot for the report and the commands to check it, @tyukh !

As I have seen that docker error in the past a few times in my leap 15.1 VM, I was curious and wanted to verify it. I actually have seen it with my VMs running on a laptop that moves between networks while being asleep. As a reboot of the laptop fixes it, I thought it was due to my VirtualBox network being a little lost with the laptop waking up in a different network and not refreshing. As I never had the opportunity to test for it on a fix hardware and had thus dismissed it :-\ . Note that I don't recall seeing the ipv6 address format in the docker error when I had seen it (only v4, I believe, but I wouldn't necessarily trust my memory on that).

I just tried to replicate it on a freshly installed 3.9.2-54 ISO install on real hardware, and fortunately it all worked as expected:

rockhome:~ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 74:27:ea:dc:c6:85 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.121/24 brd 192.168.1.255 scope global noprefixroute dynamic eth0
       valid_lft 82506sec preferred_lft 82506sec
    inet6 fe80::6a43:a5eb:cf3b:b7c6/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:56:6b:a8:68 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
rockhome:~ #
rockhome:~ # lsof -i
-bash: lsof: command not found
rockhome:~ #
rockhome:~ #
rockhome:~ # ss -tulwn
Netid  State      Recv-Q Send-Q                                                         Local Address:Port                                                                        Peer Address:Port
???    UNCONN     0      0                                                                          *:58                                                                                     *:*
udp    UNCONN     0      0                                                                    0.0.0.0:68                                                                               0.0.0.0:*
tcp    LISTEN     0      128                                                                  0.0.0.0:443                                                                              0.0.0.0:*
tcp    LISTEN     0      128                                                                127.0.0.1:8000                                                                             0.0.0.0:*
tcp    LISTEN     0      128                                                                  0.0.0.0:8001                                                                             0.0.0.0:*
tcp    LISTEN     0      128                                                                  0.0.0.0:22                                                                               0.0.0.0:*
tcp    LISTEN     0      128                                                                127.0.0.1:5432                                                                             0.0.0.0:*
tcp    LISTEN     0      100                                                                127.0.0.1:25                                                                               0.0.0.0:*
tcp    LISTEN     0      128                                                                     [::]:22                                                                                  [::]:*

I then set up the Rock-ons service, turned it ON, and then installed the Emby server rock-on, which worked flawlessly the first time (as it was a brand new install, it pulled the image without error).

I also tried setting up a new email account for email alerts and it all worked as expected as well: test new settings worked, and sending a test email worked (was sent and I received it right away).

I still think the ipv6 situation needs to be dealt with, but I wanted to report this here to show that we seem to be in a better situation in our current installer on JeOS Leap 15.1.

Hope this helps,

[tyukh]

I just tried to replicate it on a freshly installed 3.9.2-54 ISO install on real hardware, and fortunately it all worked as expected

As I see the IPv6 protocol is not disabled in your configuration (IPv6 addresses are assigned for lo and eth0 interfaces). The problem occurs if there is no IPv6 assigned to the interfaces, but the services continue to try to use IPv6 protocol. Apparently, non-disabled support for IPv6 is also a solution to the problem, with the exception of support for IPv6 by Rockstor.

[tyukh]

For clarification: my steps was following

• clean install latest Tumbleweed with enabled NetworkManager
• config interfaces via nmtui, IPv6 DHCP settings - "disabled"
• IPv6 disable by Yast's network settings
• Rockstor's install from Rockstor-Testing repo
• geting error in Rockstor's email settings
• attempted disable IPv6 with sysctl configs
• still getting error
• disabling IPv6 with kernel args
• error gone

Of course, this sequence has been "flavored" with numerous 'reboot now'

[phillxnet]

@tyukh Thanks for the clarification. I get it now, and I had initially mis-understood your point: that yast / sysctl does not completely disable ipv6 system wide for all programs. And yes the intention was to do just this as currently our Web-UI can only configure for ipv4 so we should, with our appliance aim, ensure the entire system is only using ipv4. Otherwise there are network configurations at play that were not intended by our users. I had initially though that the Yast suggestion in the dev notes would be sufficient.

I'm exploring our options and think I have a way to do this within our ISO installer config, I'll report back once I've looked at that. The default Leap15.1 ISO install is a development platform for us and ultimately we are aiming at releasing an installer that has all the system customisation pre-applied. Still not happy about adding this grub config to our initrock script and is just overkill for this. As a package we are over stepping quite a few lines to reach out this far. So will likely end up adding your grub suggestion to the dev notes.

Will report back soon on this one.

[phillxnet]

@tyukh OK, so I'm sold and I have now updated our canonical forum post re preparing a Leap15.1 as a stand in for our as-yet unreleased Kiwi config to build our pending next gen ISO installer. The relevant config in Kiwi was:

kernelcmdline="ipv6.disable=1 ...

which results in the consequent install from an resulting ISO having the option thus implemented within the resulting grub config:

GRUB_CMDLINE_LINUX_DEFAULT="ipv6.disable=1 ...
GRUB_CMDLINE_LINUX=""

Thanks @tyukh and @FroggyFlox for your efforts on this one. I think it's best we just go with this as we don't, yet, have any capability to configure IPv6 and in the interests of removing as many unknowns as possible this looks like the way to go. We can then be more assured that what is set within the Web-UI network wise is what actually exists at the system level. Especially given @tyukh's various findings to date on IPv6 anomalies with some of our existing related services. And @FroggyFlox re a pending service suspected IPv6 related anomaly.

@tyukh in light of the above I'm proposing that we now close this issue as having added to our canonical 'Built on openSUSE dev notes...' and that our proposed installer config also now includes this fix (as detailed above). If you agreed then do please close this issue at your leisure.

[tyukh]

Thanks for kind words and work you and @FroggyFlox do. Issue is closed.

[kri164]

Hi tyukh,
can I ask you,, why do you need to disable IPv6? Thank you for explain.

[tyukh]

can I ask you,, why do you need to disable IPv6? Thank you for explain.

Hi!
Since I am not a maintainer, I can only give my opinion. IPv6 is not used, has not been tested, and is completely unsupported in Rockstor codebase. Because of this, enabled IPv6 can cause potential problems and strange behavior.
For example, the configuration for a some service may have IPv6 settings (e.g. interface binding) that Rockstar does not expect and process.

[joschi127]

This change causes the following error when installing latest Duplicati Rock-On:

docker: Error response from daemon: driver failed programming external connectivity on endpoint duplicati-lsio ...
... Error starting userland proxy: listen tcp6

When I re-enable ipv6 Duplicati is working fine.