Rational Quadratic Spline

[torfjelde]

Implementation of Rational Quadratic Spline (RQS) [1] as a Bijector.

TODOs

• Add testing (have some locally, just need to clean up a bit)
• At the moment, RationalQuadraticSpline assumes all parameters to be of same type. It makes ensuring that return-types make sense, but it will cause issues if you try to use AD only for one of the parameters. Should we drop this restriction and just try to deal with it?
  • Resolution: keeping it like this for now.
• Need to clean up parts of the implementation, e.g. batched multivariate impls.
  • Resolution: still not as I want it, but I'm leaving it for now and we can implement if necessary.
• The way we currently do batching in forward for RQS is not the most efficient. So maybe drop the use of "merging" named tuples and instead implement "by hand" a batched version of forward. Technically we can just implement the multivariate version using Stacked. Uncertain about whether or not it's worth it performance-wise.
  • Resolution: implemented for univariate case, but leaving the rest for now. Can implement in future if need be.

Notes

• At the moment RationalQuadraticSpline takes "unconstrained" parameters, i.e. expects input on ℝᵈ, despite the fact that under the hood we transform widths and heights to lie on a (d-1)-simplex S ⊂ ℝᵈ and derivatives to be positive. I initially didn't do this way, but instead let the user choose between passing the parameters wrapped in a Unconstrained struct, in which case we would perform the transformation, or as raw values, in which case it was assumed the parameters indeed was contained within their respective domains. I dropped the use of Unconstrained to reduce "new" code in this PR and since this is very similar to an issue we've expressed interest in solving in more generality.

References

1. Durkan, C., Bekasov, A., Murray, I., & Papamakarios, G., Neural Spline Flows, CoRR, arXiv:1906.04032 [stat.ML], (), (2019).

[codecov]

Codecov Report

Merging #80 into master will increase coverage by 1.98%.
The diff coverage is 88.18%.

@@            Coverage Diff             @@
##           master      #80      +/-   ##
==========================================
+ Coverage   54.98%   56.97%   +1.98%     
==========================================
  Files          27       28       +1     
  Lines        1726     1836     +110     
==========================================
+ Hits          949     1046      +97     
- Misses        777      790      +13     

Impacted Files	Coverage Δ
src/interface.jl	63.88% <ø> (ø)
src/bijectors/rational_quadratic_spline.jl	88.18% <88.18%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a70914e...e301948. Read the comment docs.