Raycaster is off center or scaled based on window size

[maximus123123]

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
OnClick events and the raycaster will give the wrong scaling of the model depending on the size of the screen resulting in missing the target when it should hit or hitting the target when it should miss.
If the current behavior is a bug, please provide the steps to reproduce.
add a mouse down event in the basic example then click next the torus knot or just above/below it

	function mousedown( event ) {
		var raycaster = new THREE.Raycaster();
		var mouse = new THREE.Vector2();
		mouse.x = ( event.clientX / window.innerWidth ) * 2 - 1
		mouse.y = - ( event.clientY / window.innerHeight ) * 2 + 1
		event.stopPropagation(); 
		raycaster.setFromCamera( mouse, camera );
		var intersects =raycaster.intersectObjects([mesh],true);
		console.log(intersects);
		if (intersects.length>0) {
			alert("clicked");
		}
	}
	window.addEventListener( 'mousedown', mousedown, false );

Please mention other relevant information such as the browser version, Operating System and Device Name
Chrome on PC and Phone
What is the expected behavior?
Accurate mousedown events causing the correct
If this is a feature request, what is motivation or use case for changing the behavior?

[kalwalt]

Hi @maximus123123 , thank you! Does it happens also with the other examples?

[maximus123123]

Hi @maximus123123 , thank you! Does it happens also with the other examples?

I believe so. Heres another person with the same issue: jeromeetienne/AR.js#584

The issue, I think,
It's either the camera:
The camera view perhaps is different as you need to change the example's camera from new THREE.Camera to THREE.PerspectiveCamera.

Or the canvas size isn't updating correctly the size correctly for the calculation thus giving the wrong calculation. I have tried using vector3 as well but it didn't help

[kalwalt]

Thank you @maximus123123, we also need to know which version of Ar.js are you using, at the moment we have ar.js and ar-nft.js because they use different version of jsartoolkit5: the first the oldest the last the newest version with NFT. Maybe this bug occur the oldest but not the newest version? Please tell me exactly you setup. 🙂

[maximus123123]

It occurs in the old and new version.
Both "basic" setups make the issue occur and the issue has existed for a while.
Resizing the window will make it better then worse again depending on how far you resize.

I am currently using the new basic version:
https://github.com/AR-js-org/AR.js/blob/master/three.js/examples/basic.html

[kalwalt]

Ok thank you so much! We will see how to fix this, i have not a clear idea how to solve now.

[maximus123123]

Are you able to reproduce the error?

[kalwalt]

Are you able to reproduce the error?

i will try later if i have time. 😄

[kalwalt]

@maximus123123 with the basic example i got this error:

three.min.js:869 THREE.Raycaster: Unsupported camera type.
setFromCamera @ three.min.js:869
mousedown @ basic.html:151
basic.html:153 []

i tested only on Desktop (Ubuntu 18.04.03)

[kalwalt]

I will look deeper into it when i have time, maybe it's not a bug, probably needs the right setup. 😄

[maximus123123]

Right, you need to change THREE.Camera to THREE.PerspectiveCamera

[XiaNi]

@maximus123123 , try to replace this part:

mouse.x = ( event.clientX / window.innerWidth ) * 2 - 1
mouse.y = - ( event.clientY / window.innerHeight ) * 2 + 1

with this:

var rect = renderer.domElement.getBoundingClientRect();
mouse.x = ( (event.clientX - rect.left) / rect.width ) * 2 - 1
mouse.y = - ( (event.clientY - rect.top) / rect.height ) * 2 + 1

[Andraz89]

I am using ar.js with aframe and have similar problem. If I am not using embedded on the a-scene images and videos are distorted and clicks accurate but when I use embedded ratios of elements (images, video) are correct but clicks are off target big time.

[riggitynick]

Same issues here. The targets are off, and they're off in different ways depending on if its a mobile sized screen or a regular desktop/laptop.

Here's a link to my codepen with the issue: https://codepen.io/riggitynick/pen/MWaOOeb?editors=1010

There are 3D shapes, and when you click on them, they should vanish, the targets are off a bit(and in really strange ways-weather I open it on my phone or laptop).

[aoimewo]

Is this problem solved?

I have been developing similar functions recently
and have the same problem as this post and this one

In my test
It seems that the ratio of the video element (arjs-video) and canvas does not match, will causes the raycaster coordinates to be distorted

raycaster I usage

cursor="rayOrigin: mouse" 
raycaster="objects: .collidable"

Coordinate formula of rayOrigin (A-frame official built-in, aframe.js)

registerComponent('cursor' ...
    ...
    ......
    left = point.clientX - bounds.left;
    top = point.clientY - bounds.top;
    mouse.x = (left / bounds.width) * 2 - 1;
    mouse.y = -(top / bounds.height) * 2 + 1;

Very similar to @XiaNi provided above

My Test

version

A-frame: 0.9.2
ar.js: 3.1.0

Test Methods

I will generate a object (3D model) on the AR marker, and click on the object with the mouse to stick on a small ball at the click position to display the raycaster coordinates.

Situation.1

<a-scene
     embedded
     arjs="sourceWidth:640; sourceHeight:480; displayWidth: 1920; displayHeight: 1440;">

The ratio of display (1920x1440) is setting to same as the source (640x480), both are 4:3.

• At 1920x1440 in the browser screen
  The ratio scale of 3D model and raycaster coordinates are displayed perfectly.
  The ball is perfectly at the position of the object I clicked.
  Because 640x480 and 1920x1440 are both 4:3 ratio.

• At 1920x1080 in the browser screen
  The ratio scale of the 3D model is displayed perfectly, but the raycaster coordinates will be distorted.
  The position of the ball has offset.
  Because 640x480 is 4:3 and 1920x1080 is 16:9 ratio.

In the above cases, due to different screen heights, the offset can be clearly seen as the closer to the upper and lower sides of the screen, the greater the Y-axis offset, and the closer to the center point, the smaller the offset.

BTW, other screen tests can also have perfect results as long as the screen ratio setting same as source.
In this case (source=640x480, 4:3) like 1280x960/2560x1920 (4:3) can also work perfectly.

Situation.2

<a-scene
     embedded
     arjs="sourceWidth:640; sourceHeight:360; displayWidth: 1920; displayHeight: 1080;">

The ratio of source (640x360) is setting to same as the display (1920x1080), both are 16:9.

• At 1920x1080 in the browser screen
  It looks like I should get the result I want.
  Unfortunately, the raycaster coordinates are displayed perfectly.
  And the ball is also perfectly at the position of the object I clicked.
  BUT the ratio scale of the 3D model is distorted...

So, it seems that I cannot adjust the ratio of source to match the display.
It can only adjust the display to match the source ratio.
But as long as the display and source ratios are different, the raycaster coordinates will be distorted.
Otherwise, rewrite the coordinate formula of rayOrigin?

It is difficult to ensure that the camera (source) and screen resolution (display) supported by each device will be in the same ratio.
Like the browser window can also adjust the screen size at will.

Any suggestions? Or this problem already correcting?
Thanks!

[mikeSBIT]

Hi.
I'm also having similar issue while using AFrame and click handling.
Did anyone find a solution?

[williamqian]

how to solve this problem?

[Bramichou]

hello @kalwalt, do you have any news related to this issue ? thanks !

[dynamic-rick]

Hi anyone have an update on this problem???

[Markkop]

Hey, people. I've fallen in this problem as well.
It seems to happen only with embedded component as I've tested on #66.
Here's a visual evidence about this issue:

[Markkop]

Hello there!
I've managed to fix this problem by following this Stackoverflow Question and attaching a cursor component after the scene is initialized

const scene = AFRAME.scenes[0];
if (!scene) {
  return;
}
const mouseCursor = document.createElement('a-entity');
mouseCursor.setAttribute('cursor', 'rayOrigin', 'mouse');
scene.appendChild(mouseCursor);

I hope this can help us to fix the problem in its source ✌️

[dynamic-rick]

Thank you!! I'll try this soon....

…

----------------------------------------------------------- *DYNAMIC MANAGEMENT GROUP LTD.* ----------------------------------------------------------- *ADDRESS:* Room A, 4th Floor Sing Teck Factory Bldg. 44 Wong Chuk Hang Road, Hong Kong Tel: +852-26633524 Cell: +852-90888971 ---------------------------------------------------------------------------- Dynamic Events and Marketing Specialist https://.dynevents.com <https://www.dynevents.com> ---------------------------------------------------------------------------- KidsGOLF - the fastest way to learn golf. https://.kidsGOLF.hk <https://www.kidsGOLF.hk> -----------------------------------------------------------------------------

On Fri, 1 Oct 2021 at 22:14, Marcelo Kopmann ***@***.***> wrote: Hello there! I've managed to fix this problem by following this Stackoverflow Question <https://stackoverflow.com/questions/67519054/wrong-position-of-a-clickable-object-on-ar-js-scene/67534948#67534948> and attaching a cursor component after the scene is initialized const scene = AFRAME.scenes[0]; if (!scene) { return; } const mouseCursor = document.createElement('a-entity'); mouseCursor.setAttribute('cursor', 'rayOrigin', 'mouse'); scene.appendChild(mouseCursor); I hope this can help us to fix the problem in its source ✌️ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#40 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOXFELGSBUQQEOCNDVEPG23UEW6ZTANCNFSM4LCTD5XQ> .

[Yzuriha]

Hello there! I've managed to fix this problem by following this Stackoverflow Question and attaching a cursor component after the scene is initialized

const scene = AFRAME.scenes[0];
if (!scene) {
  return;
}
const mouseCursor = document.createElement('a-entity');
mouseCursor.setAttribute('cursor', 'rayOrigin', 'mouse');
scene.appendChild(mouseCursor);

I hope this can help us to fix the problem in its source ✌️

This worked for me! I basically just created a new Component and attached it to the . The component is like this:

AFRAME.registerComponent('custom-cursor', {
    init: function () {
        setTimeout(() => {
            const scene = AFRAME.scenes[0];
            const mouseCursor = document.createElement('a-entity');
            mouseCursor.setAttribute('cursor', 'rayOrigin', 'mouse');
            mouseCursor.setAttribute('raycaster', 'objects', '.clickable');
            scene.appendChild(mouseCursor);
        }, 2000);
    },
});

and then just <a-scene arjs embedded custom-cursor>...</a-scene>

I was literally looking for an answer for days and finally came across the solution.

[val14-98]

Hello, @Yzuriha thanks for your answer that helped me a lot. However when I tried your example, the target intersection seems to be with the marker rather than the entity which has ".clickable" class. (I'm trying to use it, with an <a-plan> which is taller than my marker)

I tried different things, but none of them work.

This line has no effect, Raycaster only detect intersection inside marker area
mouseCursor.setAttribute('raycaster', 'objects', '.clickable');

If you have a solution, I'm a taker !

[Anulo2]

Hey everyone I've created this little example to show how you can make it work with aframe 0.9.2. Unfortunately it doesn't work with more recent versions than that :(
https://glitch.com/edit/#!/aframe-ar-js-issue
Hope this will help while this isn't fixed.
the working example works without the embedded tag too :D

[aaweb]

Same here with Aframe 1.2 :( Does not work, not even with @Yzuriha or @Markkop 's workarounds. Any other ideas?

[Curacao-lb]

Has anyone examined the source code of this demo from 8th Wall? How did they overcome this problem?
https://templates.8thwall.app/placeground-aframe/ with mobile tablet or phone