If there are two lists of different sizes,
iterating over the smaller list and then
looking up in the larger list is cheaper
than vice versa, because lookups scale
sublinearly.

The associated_items(def_id) call
allocates internally.
Previously, we'd have called it for
each pair, so we'd have had O(n^2)
many calls. By precomputing the
associated items, we avoid
repeating so many allocations.

The only instance where this precomputation
would be a regression is if there's only
one inherent impl block for the type,
as the inner loop then doesn't run.
In that instance, we just early return.

Also, use SmallVec to avoid doing an
allocation at all if the number is small
(the case for most impl blocks out there).