You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Most of the time, the same set of switch ports are being updated (or confirmed) in swportvlan, meaning deletions of obsolete records are likely rare. Yet, this method, through Django, can cause an SQL statement of over 500KB in size to be sent to the PostgreSQL server on an install with over 40K switch ports.
The statement produced follows the template
DELETEFROM"swportvlan"WHERE NOT ("swportvlan"."interfaceid"IN (1, 2, 3, 4, .... ));
Which seems to be really inefficiently processed by PostgreSQL (upwards of 30 seconds, resulting in no records deleted on the test installation). It seems that processing the set difference of interface primary keys in Python is a lot more efficient.
Calculating the difference in Python would also make debug logs more readable, since we can print exactly which records to delete, rather then try to parse a 500KB list of interface id's to NOT delete.
The text was updated successfully, but these errors were encountered:
This method appears to produce SQL statements that become extremely inefficient on large NAV installs (with many, many interfaces):
https://github.com/UNINETT/nav/blob/178616318edf35b7250b889b53f87d09229996fa/python/nav/topology/vlan.py#L409-L411
Most of the time, the same set of switch ports are being updated (or confirmed) in
swportvlan
, meaning deletions of obsolete records are likely rare. Yet, this method, through Django, can cause an SQL statement of over 500KB in size to be sent to the PostgreSQL server on an install with over 40K switch ports.The statement produced follows the template
Which seems to be really inefficiently processed by PostgreSQL (upwards of 30 seconds, resulting in no records deleted on the test installation). It seems that processing the set difference of interface primary keys in Python is a lot more efficient.
Calculating the difference in Python would also make debug logs more readable, since we can print exactly which records to delete, rather then try to parse a 500KB list of interface id's to NOT delete.
The text was updated successfully, but these errors were encountered: