Add numerical tests to transformations

[yebai]

The current test file for transform.jl only test the functionality of logpdf_with_trans

Turing.jl/test/transform.jl/transform.jl

Line 11 in ce780b8

logpdf_with_trans(dist, x, true)

We need to add correctness tests for logpdf, e.g.

Distributions.logpdf(d, x) == Turing.logpdf_with_trans(d, link(x), true)

[willtebbutt]

Good catch. I completely forgot to sort that out when I was refactoring. Will sort it out now.

[willtebbutt]

#543 now resolves most of these issues, but there remain a couple of outstanding issues:

1. A few distributions (Dirichlet, Wishart, InverseWishart) don't have tests to ensure that their link functions are consistent with their logpdf_with_trans implementation. Due to the way that we transform between the constrained and unconstrained representations the Jacobian of the transformation is singular. This isn't a fundamental issue, but I'm not completely sure what the best way to resolve it is beyond directly hacking away at stuff.
2. The logpdf of the Kolmogorov distribution returns -Inf for x close to zero. I assume this is a numerical stability issue, so I've disabled testing for that for now as it messes everything up, and is probably on the Distributions.jl end.

[yebai]

A few distributions (Dirichlet, Wishart, InverseWishart) don't have tests to ensure that their link functions are consistent with their logpdf_with_trans implementation. Due to the way that we transform between the constrained and unconstrained representations the Jacobian of the transformation is singular. This isn't a fundamental issue, but I'm not completely sure what the best way to resolve it is beyond directly hacking away at stuff.

I did some eye-ball tests a while ago and didn't find any obvious errors.

[willtebbutt]

I did some eye-ball tests a while ago and didn't find any obvious errors.

Sure, I agree that it might be fine, but we really should be testing this stuff heavily as, as you point out, the numerical stability of all of our HMC-related code depends on it quite heavily.

[xukai92]

Due to the way that we transform between the constrained and unconstrained representations the Jacobian of the transformation is singular. This isn't a fundamental issue, but I'm not completely sure what the best way to resolve it is beyond directly hacking away at stuff.

Will a numerical approximation of Jacobian work here?

[willtebbutt]

A numerical approximation would have the same issues, and would just introduce numerical errors vs the AD-based one. We're representing a vector that has D-1 degrees of freedom using a D-dimensional vector, so the Jacobian of the transform will be singular by definition. The solution is to re-write the tests slightly so that they're aware of this property, and to compute the Jacobian slightly differently. It's not particularly hard, it's just a bit awkward.

[xukai92]

I got it! Thanks for the explanation!

[yebai]

Closed in favour of TuringLang/Bijectors.jl#5