Why are my mouse events inaccurate and have the wrong target mesh

Mouse related events are implemented using color picking. This involves rendering all visible meshes with a specific uniform color. Afterwards the color underneath the mouse cursor is read back and the corresponding mesh is used as the event target. In general this works as expected, however, if custom shaders are involved things can become tricky. In case a custom vertex shader modifies the mesh position, for example displacement mapping, the final rendering and the picking result will differ because the picking implementation still uses the original mesh position for the colored rendering.

The solution to this problem is to use an Xflow operator (see below) on the mesh data to change the position of the mesh. This ensures that the picking implementation uses the correct data. For now, however, all xflow operators are run in JavaScript on the CPU. For huge meshes and complex calculations this can become a performance bottleneck. There is unfortunately no solution to this, except to not use mouse related events on meshes that have a custom vertex shader attached that changes its position.

Xflow.registerOperator("xflow.displace", {
    outputs: [{type: "float3", name: "position"}],
    params:  [
        {type: "float3", source: "position"},
        {type: "float", source: "height"},
    ],
    evaluate: function(displacedPosition, originalPosition, height, info) {
        var count = originalPosition.length / 3;
        for (var i = 0; i < count; ++i) {
            var idx = i * 3;
            displacedPosition[idx] = originalPosition[idx];
            displacedPosition[idx + 1] = originalPosition[idx + 1];
            displacedPosition[idx + 2] = originalPosition[idx + 2] * height[0];
        }
    }
});

The operator has to be used in the markup like this:

<mesh>
<data compute="position = xflow.displace(oposition, height)">
<!-- mesh data like position, normal, height and color goes here -->
</data>
</mesh>