Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uplink going down since 18.7.7 #2988

Closed
rene-demonsters opened this issue Nov 30, 2018 · 3 comments
Closed

Uplink going down since 18.7.7 #2988

rene-demonsters opened this issue Nov 30, 2018 · 3 comments
Labels
support Community support

Comments

@rene-demonsters
Copy link

rene-demonsters commented Nov 30, 2018 •
edited
Loading

About a 1,5 week ago I upgraded our OPNSense firewall from 18.7.5 to 18.7.7. This caused some minor issues where it seemed to have lost it's sense of DHCP and the uplink. A reboot and some tweaking later, it seemed to have been resolved.

Fast forward a few days and our internet suddenly drops out. After investigating the firewall has lost it's uplink. When I try to ping a host from the VGA console it tells me no route to host. When I tell it to reload all services (option 11 on the menu), it starts to work again. Sometimes for a few hours, sometimes for a few days.

I've checked the firewall logs for any errors or general stuff out of the ordinary and I noticed that on each moment of downtime /usr/local/etc/rc.newwanip has been run. Though I don't see anything being off looking at the messages from it (it gives the same ones upon Reload all), but I might be looking at the wrong place.

Our office has 2 external IP adresses, one starting with 84.x.x.x and another starting with 178.x.x.x, since it breaks on /usr/local/etc/rc.newwanip I think that might be relevant. Also we don't use IPv6.

In an attempt to resolve this I've upgraded it to 18.7.8, hoping that this would resolve the issue, but unfortunately it's still present. Is there maybe some setting I need to change between those versions in order to get this to work reliably or have I stumbled upon a bug somewhere?

Please let me know if you need any more information.

Firewall log:

Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bce0_vlan128'
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: On (IP address: 84.x.x.x) (interface: WAN[wan]) (real interface: bce0_vlan128).
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to lan
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Nov 29 18:31:54 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 29 18:31:56 OPNsense opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Nov 29 18:35:31 OPNsense kernel: ugen1.4: <Lite-On Tech USB 1.1 2 port downstream low-power hub> at usbus1
Nov 29 18:35:31 OPNsense kernel: uhub5 on uhub4
Nov 29 18:35:31 OPNsense kernel: uhub5: <Lite-On Tech USB 1.1 2 port downstream low-power hub, class 9/0, rev 1.10/2.01, addr 4> on usbus1
Nov 29 18:35:31 OPNsense kernel: uhub5: 4 ports with 2 removable, bus powered
Nov 29 18:35:32 OPNsense kernel: ugen1.5: <Lite-On Tech IBM USB Travel Keyboard with UltraNav> at usbus1
Nov 29 18:35:32 OPNsense kernel: ukbd0 on uhub5
Nov 29 18:35:32 OPNsense kernel: ukbd0: <Lite-On Tech IBM USB Travel Keyboard with UltraNav, class 0/0, rev 1.10/1.16, addr 5> on usbus1
Nov 29 18:35:32 OPNsense kernel: kbd0 at ukbd0
Nov 29 18:35:32 OPNsense kernel: uhid0 on uhub5
Nov 29 18:35:32 OPNsense kernel: uhid0: <Lite-On Tech IBM USB Travel Keyboard with UltraNav, class 0/0, rev 1.10/1.16, addr 5> on usbus1
Nov 29 18:35:33 OPNsense kernel: ugen1.6: <Synaptics Inc. Composite TouchPad  TrackPoint> at usbus1
Nov 29 18:35:33 OPNsense kernel: ums0 on uhub5
Nov 29 18:35:33 OPNsense kernel: ums0: <Rel> on usbus1
Nov 29 18:35:33 OPNsense kernel: ums0: 3 buttons and [XY] coordinates ID=0
Nov 29 18:35:33 OPNsense kernel: ums1 on uhub5
Nov 29 18:35:33 OPNsense kernel: ums1: <Rel> on usbus1
Nov 29 18:35:33 OPNsense kernel: ums1: 3 buttons and [XY] coordinates ID=0
Nov 29 18:44:43 OPNsense opnsense: user 'root' authenticated successfully
Nov 29 18:44:43 OPNsense login: login on ttyv0 as root
Nov 29 18:44:43 OPNsense login: ROOT LOGIN (root) ON ttyv0
Nov 29 18:44:43 OPNsense login: ROOT LOGIN (root) ON ttyv0
Nov 29 18:45:01 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce0_vlan128: 3
Nov 29 18:45:01 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce0_vlan128: 3
Nov 29 18:45:01 OPNsense kernel: vlan0: changing name to 'bce0_vlan128'
Nov 29 18:45:01 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce1: 3
Nov 29 18:45:03 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bce0_vlan128'
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: On (IP address: 84.x.x.x) (interface: WAN[wan]) (real interface: bce0_vlan128).
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to lan
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Nov 29 18:45:04 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: entering configure using defaults
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: IPv4 default gateway set to lan
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: no IPv6 default gateway set, assuming wan
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: setting IPv4 default route to 84.x.x.x
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: removing /tmp/bce1_defaultgw
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: creating /tmp/bce1_defaultgw using '84.x.x.x'
Nov 29 18:45:05 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: skipping IPv6 default route
Nov 29 18:45:09 OPNsense opnsense: /usr/local/etc/rc.reload_all: Resyncing OpenVPN instances.
Nov 29 18:45:09 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface ovpns1: 3
Nov 29 18:45:09 OPNsense kernel: ovpns1: link state changed to DOWN
Nov 29 18:45:11 OPNsense kernel: ovpns1: link state changed to UP
Nov 29 18:45:11 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpns1'
Nov 29 18:45:11 OPNsense opnsense: /usr/local/etc/rc.newwanip: Interface '' is disabled or empty, nothing to do.
Nov 29 18:47:38 OPNsense opnsense: /index.php: Session timed out for user 'root' from: 192.168.1.243
Nov 29 18:47:38 OPNsense api[4891]: no active session, user not found
Nov 29 18:47:48 OPNsense opnsense: /index.php: Successful login for user 'root' from: 192.168.1.243
Nov 29 19:11:30 OPNsense sshlockout[1676]: sshlockout/webConfigurator v3.0 starting up
Nov 29 20:13:41 OPNsense kernel: ugen1.4: <Lite-On Tech USB 1.1 2 port downstream low-power hub> at usbus1 (disconnected)
Nov 29 20:13:41 OPNsense kernel: uhub5: at uhub4, port 1, addr 4 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ugen1.5: <Lite-On Tech IBM USB Travel Keyboard with UltraNav> at usbus1 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ukbd0: at uhub5, port 3, addr 5 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ukbd0: detached
Nov 29 20:13:41 OPNsense kernel: uhid0: at uhub5, port 3, addr 5 (disconnected)
Nov 29 20:13:41 OPNsense kernel: uhid0: detached
Nov 29 20:13:41 OPNsense kernel: ugen1.6: <Synaptics Inc. Composite TouchPad  TrackPoint> at usbus1 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ums0: at uhub5, port 4, addr 6 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ums0: detached
Nov 29 20:13:41 OPNsense kernel: ums1: at uhub5, port 4, addr 6 (disconnected)
Nov 29 20:13:41 OPNsense kernel: ums1: detached
Nov 29 20:13:41 OPNsense kernel: uhub5: detached
Nov 29 21:35:12 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bce0_vlan128'
Nov 29 21:35:12 OPNsense opnsense: /usr/local/etc/rc.newwanip: On (IP address: 84.x.x.x) (interface: WAN[wan]) (real interface: bce0_vlan128).
Nov 29 21:35:13 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Nov 29 21:35:13 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to lan
Nov 29 21:35:13 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 29 21:35:13 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Nov 29 21:35:13 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 29 21:35:14 OPNsense opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Nov 30 09:52:52 OPNsense kernel: ugen1.4: <Lite-On Tech USB 1.1 2 port downstream low-power hub> at usbus1
Nov 30 09:52:52 OPNsense kernel: uhub5 on uhub4
Nov 30 09:52:52 OPNsense kernel: uhub5: <Lite-On Tech USB 1.1 2 port downstream low-power hub, class 9/0, rev 1.10/2.01, addr 4> on usbus1
Nov 30 09:52:53 OPNsense kernel: uhub5: 4 ports with 2 removable, bus powered
Nov 30 09:52:54 OPNsense kernel: ugen1.5: <Lite-On Tech IBM USB Travel Keyboard with UltraNav> at usbus1
Nov 30 09:52:54 OPNsense kernel: ukbd0 on uhub5
Nov 30 09:52:54 OPNsense kernel: ukbd0: <Lite-On Tech IBM USB Travel Keyboard with UltraNav, class 0/0, rev 1.10/1.16, addr 5> on usbus1
Nov 30 09:52:54 OPNsense kernel: kbd0 at ukbd0
Nov 30 09:52:54 OPNsense kernel: uhid0 on uhub5
Nov 30 09:52:54 OPNsense kernel: uhid0: <Lite-On Tech IBM USB Travel Keyboard with UltraNav, class 0/0, rev 1.10/1.16, addr 5> on usbus1
Nov 30 09:52:54 OPNsense kernel: ugen1.6: <Synaptics Inc. Composite TouchPad  TrackPoint> at usbus1
Nov 30 09:52:54 OPNsense kernel: ums0 on uhub5
Nov 30 09:52:54 OPNsense kernel: ums0: <Rel> on usbus1
Nov 30 09:52:54 OPNsense kernel: ums0: 3 buttons and [XY] coordinates ID=0
Nov 30 09:52:55 OPNsense kernel: ums1 on uhub5
Nov 30 09:52:55 OPNsense kernel: ums1: <Rel> on usbus1
Nov 30 09:52:55 OPNsense kernel: ums1: 3 buttons and [XY] coordinates ID=0
Nov 30 09:53:44 OPNsense opnsense: user 'root' authenticated successfully
Nov 30 09:53:44 OPNsense login: login on ttyv0 as root
Nov 30 09:53:44 OPNsense login: ROOT LOGIN (root) ON ttyv0
Nov 30 09:53:44 OPNsense login: ROOT LOGIN (root) ON ttyv0
Nov 30 09:53:47 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce0_vlan128: 3
Nov 30 09:53:47 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce0_vlan128: 3
Nov 30 09:53:47 OPNsense kernel: vlan0: changing name to 'bce0_vlan128'
Nov 30 09:53:47 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface bce1: 3
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bce0_vlan128'
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: On (IP address: 84.x.x.x) (interface: WAN[wan]) (real interface: bce0_vlan128).
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to lan
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv4 default route
Nov 30 09:53:50 OPNsense opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: entering configure using defaults
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: IPv4 default gateway set to lan
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: no IPv6 default gateway set, assuming wan
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: setting IPv4 default route to 84.x.x.x
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: removing /tmp/bce1_defaultgw
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: creating /tmp/bce1_defaultgw using '84.x.x.x'
Nov 30 09:53:51 OPNsense opnsense: /usr/local/etc/rc.reload_all: ROUTING: skipping IPv6 default route
Nov 30 09:53:56 OPNsense opnsense: /usr/local/etc/rc.reload_all: Resyncing OpenVPN instances.
Nov 30 09:53:56 OPNsense kernel: ifa_maintain_loopback_route: deletion failed for interface ovpns1: 3
Nov 30 09:53:56 OPNsense kernel: ovpns1: link state changed to DOWN
Nov 30 09:53:57 OPNsense kernel: ovpns1: link state changed to UP
Nov 30 09:53:57 OPNsense opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpns1'
Nov 30 09:53:57 OPNsense opnsense: /usr/local/etc/rc.newwanip: Interface '' is disabled or empty, nothing to do.
Nov 30 10:02:37 OPNsense kernel: ugen1.4: <Lite-On Tech USB 1.1 2 port downstream low-power hub> at usbus1 (disconnected)
Nov 30 10:02:37 OPNsense kernel: uhub5: at uhub4, port 1, addr 4 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ugen1.5: <Lite-On Tech IBM USB Travel Keyboard with UltraNav> at usbus1 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ukbd0: at uhub5, port 3, addr 5 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ukbd0: detached
Nov 30 10:02:37 OPNsense kernel: uhid0: at uhub5, port 3, addr 5 (disconnected)
Nov 30 10:02:37 OPNsense kernel: uhid0: detached
Nov 30 10:02:37 OPNsense kernel: ugen1.6: <Synaptics Inc. Composite TouchPad  TrackPoint> at usbus1 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ums0: at uhub5, port 4, addr 6 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ums0: detached
Nov 30 10:02:37 OPNsense kernel: ums1: at uhub5, port 4, addr 6 (disconnected)
Nov 30 10:02:37 OPNsense kernel: ums1: detached
Nov 30 10:02:37 OPNsense kernel: uhub5: detached

Relevant interface:

bce0_vlan128: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether bc:30:5b:d7:01:f4
        inet6 fe80::be30:5bff:fed7:1f4%bce0_vlan128 prefixlen 64 scopeid 0x8
        inet 84.x.x.x netmask 0xffffff00 broadcast 84.x.x.x
        inet 178.x.x.x netmask 0xffffffff broadcast 178.x.x.x
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex,master>)
        status: active
        vlan: 128 vlanpcp: 0 parent interface: bce0
        groups: vlan
@AdSchellevis AdSchellevis added the support Community support label Nov 30, 2018
@rene-demonsters
Copy link
Author

In order to reach a wider audience and in hopes of getting down to the case, I've opened a topic on the forum: https://forum.opnsense.org/index.php?topic=10525

@rene-demonsters
Copy link
Author

So I think I may have resolved the issue. I have marked one of my gateways as the default gateway and the logging now looks like this:

Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bce0_vlan128'
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: On (IP address: 84.x.x.x) (interface: WAN[wan]) (real interface: bce0_vlan128).
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 84.x.x.y
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '84.x.x.y'
Dec 19 19:49:04	opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Dec 19 19:49:05	opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.

So it actually sets the IPv4 gateway to WAN instead of LAN now and doesn't delete any routes that would be outgoing. I'll keep an eye on it for a few days and if it's resolved I'll close the ticket.

@fichtner
Copy link
Member

@rene-demonsters thanks for the feedback. I'm hoping the issue went away, if not feel free to reopen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
support Community support
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fichtner @AdSchellevis @rene-demonsters