Simplify spectral clustering solver logic

[amueller]

We need to simplify the logic for selecting a solver in spectral_clustering.
See discussion here:
#10715 (comment)
#14647 (comment)
#10720 (comment)

[alexshacked]

Hi @amueller. I would like to take a shot at this. I can see it's demanding but would still like to take a look. Is it OK, or is someone already looking at it?

[amueller]

@alexshacked please go for it!

[alexshacked]

Thank you, I will have a design based on #10715 (comment), #14647 (comment), #10720 (comment) in a couple of days

[alexshacked]

After reading the issues and PRs referenced here and also studying the spectral clustering implementation, I understand that this enhancement's goal is to refactor the logic for chosing an eigen solver in function spectral_embedding() located in manifold/spectral_embedding.py
I opened a [WIP ] PR #15136, where I show an inital suggestion for refactoring spectral_embedding(). My intention is just to present an idea for simplifing the logic that choses the solver. It felt easier to write code, then to write a design document. I hope to get feed-back that will focus me onwards.
I also intend to add more regressinon tests, but I thought to work on the tests only after the refactoring goal becomes more clear to me.

[alexshacked]

After we finish the refactoring and achieve a clear code, there are a couple of candidates for improving the algorithm. I found those by reading the issues referenced here. For now I have 2 ideas based on @lobpcg suggestions .

1. call scipy.linalg.eigh() always if n_nodes < 5 * n_components.
   Meaning, if number of samples is smaller than the (number_of_requested_eigen_vectors x 5)
   original comment: Amg arpack workaround fix #14647 (comment)

2. When calling arpack with shift-invert use a sigma value of -1e-5 instead of 1.0 and do not multiply the laplacian by -1, just before calling arpack
   original comment: Amg arpack workaround fix #14647 (comment)

@amueller what do you think?