Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interfaces, performance / usability issues when having a huge amount of vlans. #3567

Closed
AdSchellevis opened this issue Jul 11, 2019 · 1 comment

Comments

Projects
1 participant
@AdSchellevis
Copy link
Member

commented Jul 11, 2019

While trying to operate an instance with a lot of interfaces (around 800), one of our customers experienced slugginess in the interface on various points.

test patches on OPNsense 19.7-r1

pkg install -f opnsense-devel
opnsense-patch  e250b14 6ae3566 4c4bdea 292358b  e6a228d  67ba6bf 082b566 29b383c 197e70d 0a73a6f 028e7a2 2e6b138  c72e682 45b3d22  aa25ecf 1da0794

@AdSchellevis AdSchellevis self-assigned this Jul 11, 2019

AdSchellevis added a commit that referenced this issue Jul 11, 2019

console: banner, don't call ifconfig for each interface. one of the u…
…sability issues in #3567

When there are a lot of interfaces, these calls consume quite some time and eventually the output of legacy_interfaces_details() is what matters to all of them.
@AdSchellevis

This comment has been minimized.

Copy link
Member Author

commented Jul 11, 2019

also related 6ae3566

fichtner added a commit that referenced this issue Jul 11, 2019

console: banner, don't call ifconfig for each interface. one of the u…
…sability issues in #3567

When there are a lot of interfaces, these calls consume quite some time and eventually the output of legacy_interfaces_details() is what matters to all of them.

(cherry picked from commit e250b14)

AdSchellevis added a commit that referenced this issue Jul 11, 2019

add cache helper function in config.inc, needed for #3567
get_cached_json_content() deserialize json data when the file is found and not yet expired, returns null otherwise.
Since there might be some spots in the code where this can be convenient, it seemed like a good idea to wrap it in a function within the legacy code base.
Maybe config.inc is not enterily the right spot for this, but util.inc is quite diverse already.

AdSchellevis added a commit that referenced this issue Jul 11, 2019

UI: auth.inc, use cached addresslist in referer check, for #3567
This prevents ifconfig is executed on every request, which could take a long time when there are a lot of interfaces.

AdSchellevis added a commit that referenced this issue Jul 11, 2019

AdSchellevis added a commit that referenced this issue Jul 11, 2019

interfaces: performance for #3567
* interfaces_addresses() only request single interface data when there's only one interface requested
* interface_configure() mtu check only needs a specific interface, don't request all

AdSchellevis added a commit that referenced this issue Jul 11, 2019

filter/performance, improve loading speed by parsing legacy_interface…
…s_details() output through the plugin system. move verbose output while here. for #3567

AdSchellevis added a commit that referenced this issue Jul 12, 2019

interfaces, performance. support passing the output of our ifconfig p…
…arser for #3567 so we can prevent additional requests on non changing data

AdSchellevis added a commit that referenced this issue Jul 12, 2019

interfaces, cleanup. while working on #3567
Since $realhwif is the parent interface, it doesn't make sense to check for _vlan. We might as well remove the check and leave the rest as is, since it only triggers when either media or mediaopt is set.

AdSchellevis added a commit that referenced this issue Jul 12, 2019

interfaces: interfaces_staticarp_configure(), don't flush arp table w…
…hen booting, arp is really slow with a lot of interfaces. for #3567

AdSchellevis added a commit that referenced this issue Jul 12, 2019

dhcpdv4: performance improvements, only call interfaces_staticarp_con…
…figure() when staticarp is used on this interface, prevent excessive legacy_interfaces_details() calls for #3567

AdSchellevis added a commit that referenced this issue Jul 12, 2019

fichtner added a commit that referenced this issue Jul 15, 2019

style fix, while working on #3567, noticed a naming issue in legacy_i…
…nterfaces_details()

(cherry picked from commit 67ba6bf)

@AdSchellevis AdSchellevis added this to the 20.1 milestone Jul 15, 2019

@fichtner fichtner added this to Done in 20.1 Jul 18, 2019

EugenMayer added a commit to EugenMayer/core that referenced this issue Jul 22, 2019

console: banner, don't call ifconfig for each interface. one of the u…
…sability issues in opnsense#3567

When there are a lot of interfaces, these calls consume quite some time and eventually the output of legacy_interfaces_details() is what matters to all of them.

(cherry picked from commit e250b14)

EugenMayer added a commit to EugenMayer/core that referenced this issue Jul 22, 2019

style fix, while working on opnsense#3567, noticed a naming issue in …
…legacy_interfaces_details()

(cherry picked from commit 67ba6bf)

EugenMayer added a commit to EugenMayer/core that referenced this issue Jul 22, 2019

console: banner, don't call ifconfig for each interface. one of the u…
…sability issues in opnsense#3567

When there are a lot of interfaces, these calls consume quite some time and eventually the output of legacy_interfaces_details() is what matters to all of them.

(cherry picked from commit e250b14)

EugenMayer added a commit to EugenMayer/core that referenced this issue Jul 22, 2019

style fix, while working on opnsense#3567, noticed a naming issue in …
…legacy_interfaces_details()

(cherry picked from commit 67ba6bf)

fichtner added a commit that referenced this issue Jul 23, 2019

UI: auth.inc, use cached addresslist in referer check, for #3567
This prevents ifconfig is executed on every request, which could take a long time when there are a lot of interfaces.

(cherry picked from commit e6a228d)
(cherry picked from commit fb9e8a5)

fichtner added a commit that referenced this issue Jul 23, 2019

interfaces: performance for #3567
* interfaces_addresses() only request single interface data when there's only one interface requested
* interface_configure() mtu check only needs a specific interface, don't request all

(cherry picked from commit 082b566)

fichtner added a commit that referenced this issue Jul 23, 2019

interfaces, performance. support passing the output of our ifconfig p…
…arser for #3567 so we can prevent additional requests on non changing data

(cherry picked from commit 197e70d)

fichtner added a commit that referenced this issue Jul 23, 2019

interfaces, cleanup. while working on #3567
Since $realhwif is the parent interface, it doesn't make sense to check for _vlan. We might as well remove the check and leave the rest as is, since it only triggers when either media or mediaopt is set.

(cherry picked from commit 028e7a2)

fichtner added a commit that referenced this issue Jul 23, 2019

interfaces: interfaces_staticarp_configure(), don't flush arp table w…
…hen booting, arp is really slow with a lot of interfaces. for #3567

(cherry picked from commit 2e6b138)

fichtner added a commit that referenced this issue Jul 23, 2019

interfaces: performance support passing ifconfig details to legacy_ge…
…t_interface_addresses() and callers for #3567

(cherry picked from commit 45b3d22)

fichtner added a commit that referenced this issue Jul 23, 2019

fichtner added a commit that referenced this issue Jul 23, 2019

dhcpdv4: performance improvements, only call interfaces_staticarp_con…
…figure() when staticarp is used on this interface, prevent excessive legacy_interfaces_details() calls for #3567

(cherry picked from commit c72e682)

fichtner added a commit that referenced this issue Jul 23, 2019

fichtner added a commit that referenced this issue Jul 23, 2019

filter/performance, improve loading speed by parsing legacy_interface…
…s_details() output through the plugin system. move verbose output while here. for #3567

(cherry picked from commit 29b383c)

fichtner added a commit that referenced this issue Jul 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.