API design with penalty and datafit jitted inside fit

[mathurinm]

This attempts to simplify the API of datafits by jit complying classes when fitting models.

Currently, we have a distinct jit class for each type (e.g. Quadratic, Quadratic_32)

[PABannier]

I pushed a streamlined version of our_jit_compiler that uses built-in LRU cache.
The performance remains the same:

Minor change to make it work: change all specs to tuples instead of lists.

[mathurinm]

@PABannier thanks!

I have merged my design proposal into the existing code. It seems to be working OK.

For now I did it only for L1 and Quadratic. It should be straightforward to extend to other datafits and penalties.
It removes then need for the *_32 classes, and the need to hardcode them into generalized estimator.

Something needs to be fixed to handle path correctly (there is duplicated code currently that I could not fix easily)

WDYT of the design @QB3 @Klopfe ?

[PABannier]

Looks very clean to me. The kind of PR that removes a lot of hurdles at once.
You mentioned an issue with path @mathurinm , what is it?

[mathurinm]

I was surprised at first that fit does not call self.path (there is duplicated code in these functions). But path is designed for coherence with the Lasso (I think) and does not use penalty nor datafit.

I think in the end we can/should just call self.path in fit without defining the jitted penalty and datafit there, just define them in the path. That should be OK. @Badr-MOUFAD if you read this :)

[Badr-MOUFAD]

I liked the design. Big appreciation @mathurinm!

Before we proceed forward, we need to enhance spec_to_float32 to handle non-float attributes (e.g. spec of QuadraticGroup ). Indeed, in this case, it returns a wrong spec (cf. debug_script).

[mathurinm]

I don't understand what is meant here, can you explain?

[Badr-MOUFAD]

This is to indicate that Datafit, resp Penalty, is passed and not Datafit(), resp Penalty().

[mathurinm]

Last thing to do is to add get_spec and params_to_dict to BaseDatafit and BasePenalty, to add docstring explaining what these methods do. then LGTM!