Distributions should return log probability

[stephentu]

the distributions should also have log probability (we could call it a score instead). this would encourage and allow consumers (such as HMM) to work in log space as much as possible for better numerical stability.

[rcurtin]

This is related to #230 (and maybe #231).

I would agree that log-probability is a good idea. Maybe it's a good idea to make both available; i.e. leave Probability() intact but also add LogProbability() or something. That should help keep any downstream consumers from converting to and from logspace unnecessarily.

[stephentu]

Yes I agree, the implementations should offer both, implement one of them (which ever is faster/more stable), and defer the other to Exp(LogProbability()) or Log(Probability()).

[stephentu]

This is addressed in cb4df6f. Now when the HMM code gets around to using log probabilities, we might consider storing log probabilities directly in DiscreteDistribution to avoid having to compute a log every time. Closing this for now.