CSS3DRenderer interactive blind spot

[jouvychen]

Description

When CSS3DRenderer renders iframe interactive web pages, there are interaction blind spots at certain angles, especially when the frustum does not contain the rendering div generated by CSS3DRenderer or the angle between CSS3DRenderer and the cameraElement is too small.

Online example：
https://codepen.io/jouvychen/pen/zYMMyKp

Reproduction steps

1.First, initialize the CSS3DObject and set it's position at the origin, it works fine.
2.Then, click the upper-left button to change the CSS3DObject's position at (50, 80, -20) and camera's position at (0, 100, -100), it can't trigger scrolling events for iframe nested page.
3. Finally, click the button again, reset the CSS3DObject position at (0, 0, 0), the rolling event returned to normal.

Code

import { CSS3DRenderer, CSS3DObject } from 'https://cdn.skypack.dev/three@0.136.0/examples/jsm/renderers/CSS3DRenderer.js';
import {OrbitControls} from "https://cdn.skypack.dev/three@0.136.0/examples/jsm/controls/OrbitControls";
var scene;
var camera;
var renderer;
var control;
var css3DRenderer;
var domEleObj;
let hadChanged = false;
function init() {
  scene = new THREE.Scene();

  camera = new THREE.PerspectiveCamera(
    70,
    window.innerWidth / window.innerHeight,
    0.1,
    1000
  );
  camera.position.set(0, 10, 200);
  // camera.lookAt(220,220,220)

  renderer = new THREE.WebGLRenderer({ antialias: true });
  renderer.setClearColor("#e5e5e5");
  renderer.setSize(window.innerWidth, window.innerHeight);

  document.body.appendChild(renderer.domElement);

  window.addEventListener("resize", () => {
    renderer.setSize(window.innerWidth, window.innerHeight);
    camera.aspect = window.innerWidth / window.innerHeight;

    camera.updateProjectionMatrix();
  });
  render();
}

function initControl() {
  control = new OrbitControls(camera, renderer.domElement);
  control.enableDamping = true;
  control.update();
}

function initCreateScene() {
  const helper = new THREE.GridHelper(2000, 100);
  helper.position.y = 0;
  helper.material.opacity = 0.25;
  helper.material.transparent = true;
  scene.add(helper);

  const axesHelper = new THREE.AxesHelper(50);
  scene.add(axesHelper);
}

function initCss3d() {
  css3DRenderer = new CSS3DRenderer();
  css3DRenderer.setSize(window.innerWidth, window.innerHeight);
  css3DRenderer.domElement.style.position = "absolute";
  css3DRenderer.domElement.style.top = "0";
  // css3DRenderer.domElement.style.pointerEvents = "none";
  document.body.appendChild(css3DRenderer.domElement);
  
  control.domElement = css3DRenderer.domElement;
  control.update();

  const url = "https://threejs.org/";
  const domEle = document.createElement("iframe");
  domEle.src = url;
  // domEle.style.width = window.innerWidth + "px";
  // domEle.style.height = window.innerHeight + "px";
   domEle.style.width = "1220px";
  domEle.style.height = "860px";
  domEle.style.border = "none";
  domEle.style.left = "0";
  domEleObj = new CSS3DObject(domEle);
  domEleObj.position.set(0, 0, 0);
  domEleObj.scale.multiplyScalar(0.1);
  domEleObj.rotateY(-Math.PI * 0.25);
  scene.add(domEleObj);
  
  
}
const render = function () {
  control?.update();
  requestAnimationFrame(render);
  css3DRenderer?.render(scene, camera);
  renderer?.render(scene, camera);
};
const onChangePosition = () => {
  if (!hadChanged) {
    domEleObj.position.set(50, 80, -20)
    // domEleObj.rotation.y = Math.PI * 0.75;
  camera.position.set(0, 100, -100)
  // camera.lookAt(200, 200, -200)
  camera.updateProjectionMatrix();
  } else{
    domEleObj.position.set(0, 0, 0)
  }
  hadChanged = !hadChanged;
}
window.onload = function(){
  init();
  initControl();
  initCreateScene();
  initCss3d();
  document.getElementById('changePosition').onclick = onChangePosition;
}

Live example

https://codepen.io/jouvychen/pen/zYMMyKp

Screenshots

No response

Version

145

Device

Desktop

Browser

Chrome

OS

Windows

[yomotsu]

I tested similar situations without three.js in desktop Chome.
I think this is a browser issue.

https://jsfiddle.net/obL2szdc/

[Mugen87]

Indeed. The fiddle does work in Firefox and Safari.

Since the test case breaks with Chrome Canary 118.0.5949.0 as well, I'll file a bug at the Chromium bug tracker.

[Mugen87]

https://bugs.chromium.org/p/chromium/issues/detail?id=1472979

[jouvychen]

@yomotsu Yes, you are right. With too much trust in Google Chrome, I was so immersed in computing the transform to solve interaction problems that I forgot about browser compatibility issues.

[yomotsu]

BTW, I found another problem with the demo. When I scroll, the element disappear😹
Not sure this happens on other devices though...

Screen.Recording.2023-08-15.at.23.47.26.mp4

[jouvychen]

The edge[115.0.1901.203 (Official Version) (64-bit)] browser also has this problem. Although Firefox can interact normally, some css behavior is still problematic. For example, the border exception and the mesh edge is jagged.

[yomotsu]

That is expected. Both Edge and Chrome are built on top of Chromium. Therefore, the behavior should be the same.
Regarding Firefox, that is another browser issue. You could report the bug in Bugzilla.

[jouvychen]

That is expected. Both Edge and Chrome are built on top of Chromium. Therefore, the behavior should be the same. Regarding Firefox, that is another browser issue. You could report the bug in Bugzilla.

Yes, run the example with Edge[115.0.1901.203] 、 Chrome[115.0.5790.171] and the elements disappear, Firefox[116.0.2] works ok. It seems that using CSS3DRenderer will encounter more compatibility issues.

[Mugen87]

BTW, I found another problem with the demo. When I scroll, the element disappear😹

Maybe this gets automatically be fixed when the interactivity issues are getting solved. I guess I'll wait with filing another bug at the Chromium bug tracker until the first one is resolved.

@jouvychen Any chances to provide a live example with CSS3DRenderer that demonstrates the Firefox issue? I guess I can make a report at Bugzilla.

[jouvychen]

顺便说一句，我发现了演示的另一个问题。当我滚动时，元素消失😹了

也许当交互问题得到解决时，这会自动修复。我想我会等待在 Chromium 错误跟踪器上提交另一个错误，直到第一个错误得到解决。

有机会提供一个演示Firefox问题的现场示例吗？我想我可以在Bugzilla做一个报告。CSS3DRenderer

@Mugen87 Hi, when I published this as an online example, I was surprised to find that the border style issue no longer exists, but it still exists in local development. The linkCSS3DObject

[KurtPachinger]

I had similar behavior last month on a PC laptop with CSS3D. The bug behavior is reproducible with GUI panel bug behavior GUI panel.

In this case, a GUI panel on top would leave an inset mask relative to the bottom-left. Here the CSS3D would be invisible. If you moved the Controls, the inset rectangle stayed fixed with the GUI panel, as the content underneath was partially invisible.

It seemed related to position:static/absolute and/or some unrepresented scroll offset. Definitely sliding content and a bad offset from an obscure element.

[donmccurdy]

I'm not positive this is exactly the same issue, but it's similar enough that I'll share. I'm using CSS3DRenderer to display an HTML element within a container element that uses position:sticky and is located inline within the page body. The transformed 3D position of the object differs greatly in Chrome, depending on whether el.offsetLeft for the container is a whole number or not. Interestingly, Chrome DevTools highlights the element in the expected position either way, regardless of where the element is actually displayed.

.offsetLeft is a whole number:

.offsetLeft is a non-whole number:

Note how the div's highlight from DevTools does not match the text in the second screenshot.

---

I'm also seeing a very similar issue in Safari, where the object is offset vertically if the height of the container is not an even number.

I'll see if I can create a simple reproduction for both issues...

[donmccurdy]

I've filed a new issue for the problems I mentioned above:

• CSS3DRenderer: Transforms broken depending on container size and position #26950

[donmccurdy]

I believe the issue can be worked around with this change – would someone be able to test against one of the original issues in this thread?

• CSS3DRenderer: Modify perspective transform to avoid issues in Chrome and Firefox #27017