Skip to content

Commit

Permalink
Speed up Interface.networks
Browse files Browse the repository at this point in the history
MySQL is particularly bad at optimizing subqueries, so in cases like
the query in `Interface.networks` it would do a full table scan to
find networks that match the given parent Network IDs.

This change forces the inner query to be evaluated by wrapping it with
`list()`, making this two queries that MySQL can efficiently optimize.

It turns out that there's even a blurb in the Django docs about this, so
we're not the first to run into this:
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#nested-queries-performance

To test this, I used pynsot's CLI client in a for loop, creating 10
interfaces with a dataset that's similar to our prod dataset:
```
time (for i in {0..9}; do echo $i; nsot interfaces add -D 30167 -n "eth$i"; done)
```

Without this change, creating 10 interfaces took 9.606 seconds. After,
it took 4.723 seconds.
  • Loading branch information
nickpegg committed Sep 20, 2017
1 parent fe126a3 commit 88a3e84
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion nsot/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -1304,7 +1304,7 @@ class Meta:
def networks(self):
"""Return all the parent Networks for my addresses."""
return Network.objects.filter(
id__in=self.addresses.values_list('parent')
id__in=list(self.addresses.values_list('parent', flat=True))
).distinct()

@property
Expand Down

0 comments on commit 88a3e84

Please sign in to comment.