Refactoring in prep work for gpu clipping. …
This fixes the main performance regressions from the last round of refactoring.