ENH add dtype preservation to FactorAnalysis

[svenstehle]

Reference Issues/PRs

In scope of #11000
Continuing work of @thibsej already started in #13303

What does this implement/fix? Explain your changes.

Added dtype preservation to FactorAnalysis

Any other comments?

The test passes, but did we also check the numerical stability? PR #13303 added a test for this, is this obsolete with our new check_transformer_preserve_dtypes?

• No - we still need to check for numerical stability.

sklearn/tests/test_common.py::test_estimators[FactorAnalysis()-check_transformer_preserve_dtypes] PASSED

[glemaitre]

It was quite complex to make a review. Instead, I pushed the changes directly into the branch. So what changed is the following:

• improve the test to check the dtype of the fitted attribute and public function
• check the equivalence of the transform function between 32 and 64 bits

The latter point is difficult. Doing it I saw 2 other things to change:

• a bug where the stopping criterion was not the absolute value
• a SMALL value that was hard coded and actually too small for 32 bits. So I used eps instead depending of the input type.

[glemaitre]

We can now check if the tests are passing on all platforms.

[jeremiedbb]

1e-1 is really high tolerance. It's almost like having no test at all. I'm not sure we should even add such a test. To me there are 2 possibilities:

• We care that the results should really be close between float32 and float64. In that case we can't say that this estimator preserves dtype and we force conversion to float64.
• We consider that preserving dtype essentially means that all computations are done without conversion and it's not so bad that the results between float32 and float64 are not close (up to a reasonable tol).

This PR in its current state assumes the 2nd option. I could live with that, but I'd like the opinion of other devs before merging.