proper sampling in "pre-whitened" spaces

[joshspeagle]

In the PolyChord paper, they discuss slice sampling in spaces that have been transformed such that the appropriate step size is ~1. I had interpreted this as saying we should propose steps and directions based on, e.g., the set of bounding ellipsoids. However, I now believe that this may be incorrect: the correct way is to always propose orthogonal steps on the native unit cube, but scale the initial step size based on the proposal distribution in that direction. This is because linear transformations make it more likely that we propose in certain directions vs others, which we fail to correct for. This doesn't matter as much for random walks since we always satisfy detailed balance with any static proposal (and the "right" way to do proposals is Quite Hard), but does matter for slices since we've now distorted our sampling space.

This shouldn't be a huge change I think: we just need to map the orthogonal unit cube basis onto the ellipsoid to get proper scale factors.

[joshspeagle]

Upon review, I actually think this isn't a problem. With detailed balance, the proposal distribution doesn't really matter except in terms of sampling efficiency (auto-correlation time, etc.). So the proposal distribution we pick shouldn't matter either, except in terms of sometimes having an extremely hard time properly representing a target distribution if it's pathological. This probably was the case with my previous tests using Gaussian shells (which are quite pathological).