Add multivariate distributions

[fonnesbeck]

We at least need to have MVNormal, Multinomial, Dirichlet and Wishart.

[mrocklin]

What number of dimensions do you anticipate here. 3, 10, or 10000?

[fonnesbeck]

I've encountered multivariate nodes of length 100, at least in applications of Bayesian hierarchical models I've seen, but thousands or tens of thousands are rare.

[jsalvatier]

@fonnesbeck I've added these in https://github.com/pymc-devs/pymc/blob/8dd5bdd0e6f517a8a4b10178f76922a412f5d528/pymc/distributions/multivariate.py

I think there are still some kinks to work out though. Do you want to continue with the k-1 approach for dirichlet?

[fonnesbeck]

I think k-1 for Dirichlet values and categorical/multinomial probabilities make sense, as it removes the potential for error when specifying vectors that must sum exactly to one.

[jsalvatier]

My current approach to this is to specify the Dirichlet distribution in the canonical way and then apply a 'transform' from the size k-1 space to the size k space (i.e. concatenate(p, 1 - sum(p)) ).

This approach should generalize to other transformations, such as the one for simplexes described in http://arxiv.org/abs/1301.6064 .

Sometimes its also convenient to sample in the log space and you can describe this the same way.

[jsalvatier]

I've pushed these changes see https://github.com/pymc-devs/pymc/blob/pymc3/pymc/distributions/transforms.py