Ok I swapped to openxr, i created dedicated framebuffers for vr, heavily improved performance, added motion controls, first person, made it so that ui is at least passable.
Long story short, w/ this version, if you're dedicated enough, you could probably reasonably play the whole game. It's a little janky. I'll have a lot of updates in the future to make this a real solid vr game, but this is pretty good for now.