Model initialization with find_MAP does not seem to work well

[twiecki]

In most examples we use find_MAP() to find a good starting position for NUTS. My experience which seems to match with many others is that find_MAP() does a really poor job at providing a good starting point. More often, a manual setting is better.

I think STAN actually just starts multiple samplers at different points (sampled from the prior?) as they found the same thing.

We should probably start by changing the examples to not use find_MAP() as most people will just copy that approach. Another interesting avenue to explore is to use ADVI to find the starting point in complex models.

[springcoil]

Yeah - it's probably worth doing an example of a model - (maybe the Stochastic Volatility model) is a good one with comparing ADVI etc.
:+1; for this, and I do indeed copy the approach that's in the examples.

[aloctavodia]

Another reason to run several chains starting from different points is to then run converge diagnostics tests. This is the approach recommended in BDA.

[twiecki]

Check out #1088 for NUTS initialization using ADVI in the stoch vol model.

[fonnesbeck]

Sampling from the prior is what we did with PyMC 2. A nice approach might be to intiialize top-level hyperpriors using either expected value or ADVI, then sample from the stochastics below to initialize them.

[twiecki]

Fixed by #1523.