When I use swiper-all, I often get results from TAGS buffers as well as the buffers I'm editing.
How would you feel about a patch to ignore TAGS buffers? Whilst I understand that swiper-all is supposed to be everything, TAGS file buffers are just an implementation detail and are rarely opened intentionally.