If low-res disparity has gross errors, it can make the run-time explode. Filtering is desired.
It would be nice to have a testcase.
Can the filtering function used in disparity post-processing be used? My impression was that it did not deal with outliers.
May need to filter disparity even later, such when going say from level 3 to level 2 in full-res calculation. It is at such a stage in case of snow images that more detail comes into view and the disparity can get seriously confused.
May need hole-filling in addition to filtering at each step (save perhaps the last). After all, disparity is meant to be continuous. Hole-filling should be cheap at high levels of the pyramid.
Before doing any work it would be good to look at the literature.