a couple find_MAP issues

[bwengals]

Ive noticed a couple issues with find_MAP:

• scipy's lbfgs doesn't support float32
• unlike sampling or advi, when ran it produces a big jump in memory usage
• live_disp=True fails silently, because FlexBox in ipywidgets is depreciated in newer versions

[twiecki]

I think we should remove find_MAP and merge the functionality into pm.fit(). That should be fairly straight-forward, right @ferrine? I think SVGD with 1 particle is equivalent (or was it the expectation? which would be even better than MAP).

[junpenglao]

Agree, but in that case, should we move pm.fit() outside of VI and put it the same level with pm.sampling?

I can see potential extension to using, for example, BayesianOptim algorithm for fit()

[aseyboldt]

But what would fit return? The MAP is not an approximation to the posterior, so the methods of the Approximation object don't make sense (eg sample).

[ferrine]

@twiecki using SVGD for find_MAP is a bad idea. Scipy optimize is much more smart than sgd. I also want to hold consistency there and allow pm.fit return only pm.Approximation object. If there are strong arguments against we can think it over a little more

[twiecki]

OK, those are good arguments for keeping find_MAP.

[bwengals]

@aseyboldt Maybe find_MAP could track its progress and return a trace? I also think it would be cool to use bayesian optimization from findmap.

[junpenglao]

Yep, there has been some attempts:
https://discourse.pymc.io/t/using-skopts-minimizing-functions-in-find-map/162

[aseyboldt]

@bwengals Returning a trace (optionally) might be nice, but I don't think it solves the problem that the map is not an approximation to the posterior. A trace from a sampler has fundamentally different meaning than the trace of an optimizer.

[junpenglao]

Fixed in #2539