Use Oculus line-of-sight to control cursor

[zachkinstner]

@ddutchie wrote:

How would i go about using my sight to select and move the cursor?

I already have a script attached to an empty game object which casts a ray from the left camera. I was using it to get coordinates for my depth of field script. It works, but unfortunately it intersects with mesh colliders only and my models are rarely optimized. I have to be sneaky with what gets collision. DOF off for now.

Instead of the cursor on my right index (which has been causing issues with my coworkers (architects), it should appear when i stare at your VR menu. Using the coordinates it gets from the empty game object.

[ddutchie]

Script:

using UnityEngine;
using System.Collections;

public class FocusPoint : MonoBehaviour {
public GameObject focusObject;
public Camera camera;

void Update () {
    if(camera==null||focusObject==null) return;
    Vector3 fwd = camera.transform.TransformDirection(Vector3.forward);
    RaycastHit hit;        

    if (Physics.Raycast(camera.transform.position, fwd, out hit)){
        Vector3 temp = new Vector3(0,0,hit.distance);
        focusObject.transform.localPosition = temp;
    }
}

}

[ddutchie]

Actually it seems it works in combination with this script which is attached to a GameObject called Focus.

using UnityEngine;
using System.Collections;

public class Reticle : MonoBehaviour {
public Camera CameraFacing;
private Vector3 originalScale;

// Use this for initialization
void Start () {
    originalScale = transform.localScale;
}

// Update is called once per frame
void Update () {
    RaycastHit hit;
    float distance;
    if (Physics.Raycast (new Ray (CameraFacing.transform.position,
                                 CameraFacing.transform.rotation * Vector3.forward),
                         out hit)) {
        distance = hit.distance;
    } else {
        distance = CameraFacing.farClipPlane * 0.95f;
    }
    transform.position = CameraFacing.transform.position +
        CameraFacing.transform.rotation * Vector3.forward * distance;
    transform.LookAt (CameraFacing.transform.position);
    transform.Rotate (0.0f, 180.0f, 0.0f);
    if (distance < 10.0f) {
        distance *= 1 + 5*Mathf.Exp (-distance);
    }
    transform.localScale = originalScale * distance;
}

}

[zachkinstner]

@ddutchie sorry, I'm not understanding what this code does yet. Is it somehow setting the cursor's Transform?

I'm thinking about the following approach:

• Create a new "input module" that uses Leap input for the menu hand, and uses the Oculus orientation for the cursor.
• All the menu items are on a 3D plane. Imagine the 3D plane of the menu extending far beyond the edge of the menu. The cursor would appear at the intersection point where the Oculus viewing "ray" crosses that plane.
• From there, everything else works as usual for the menu. The items are selected once the view-based cursor gets close enough to the menu items, etc.

I'm going to give this a try today.

[ddutchie]

Sorry I didn't clarify. These are the old scripts used to set the transform position of a Focus object to the intersection of a collider and a ray cast from one of the oculus eye cameras in the scene.

But your way works. I'm just throwing ideas your way. Once my coding is on par, i'll contribute in an legible manner.

Very excited to see what happens here.
Props for sharing your work.

Sent from my iPhone

On 16 Feb 2015, at 10:48 PM, Zach Kinstner notifications@github.com wrote:

@ddutchie sorry, I'm not understanding what this code does yet. Is it somehow setting the cursor's Transform?

I'm thinking about the following approach:

Create a new "input module" that uses Leap input for the menu hand, and uses the Oculus orientation for the cursor.
All the menu items are on a 3D plane. Imagine the 3D plane of the menu extending far beyond the edge of the menu. The cursor would appear at the intersection point where the Oculus viewing "ray" crosses that plane.
From there, everything else works as usual for the menu. The items are selected once the view-based cursor gets close enough to the menu items, etc.
I'm going to give this a try today.

—
Reply to this email directly or view it on GitHub.

[zachkinstner]

The new "Leap Look" input module is fun to use! It follows the approach I outlined above. Please see the HovercastDemo-LeapLookVR.unity Unity scene.

To move from the LeapVR scene to the LeapLookVR scene:

• Replace the "HandController" object's HovercastLeapInputProvider script with the new HovercastLeapLookInputProvider script.
• Drag the Oculus "CenterEyeAnchor" object into the HovercastLeapLookInputProvider script's "Headset Camera Transform" field. Note that this Transform-based approach is not dependent on Oculus.
• Optionally, reduce the default distances of the HovercastCustomInteraction script. The look-based cursor is always on the menu's 3D plane, so these distance settings no longer need to accommodate any "depth" distance. Instead, the distances now represent a 2D radius. I chose:
  • Highlight Distance Min 0.15
  • Highlight Distance Max 0.6
  • Sticky Release Distance 0.25

[zachkinstner]

The latest commit includes a "Cursor Horizontal Offset" setting for the HovercastLeapLookInputProvider script. This shifts the cursor away from the menu side, providing more room for movement of the menu hand. For example, when using the left-hand menu, the cursor will shift to the right of its center position.

A value of 0 leaves the cursor at the center of your vision. A value of 1 moves the cursor to the edge of your vision. I set the default value to 0.4, which felt comfortable to me.

[zachkinstner]

When using this mode, I find that I don't "look" as much as I just move the menu hand to meet a mostly-still cursor.

I suppose I do this somewhat with the regular Leap input mode, too. I often rotate the menu hand so that (for example) the items at the top of the arc are closer to my fingertip-based cursor.

This is an interesting side-effect of proximity-based selections, where both "sides" move freely. You can treat both sides as dynamic/moving, or choose to treat one side as being static/fixed.

[ddutchie]

This is it. Great Job man.

Your observations are correct. I've been doing the same and so have some testers.

[ddutchie]

Here's a conceptual build

https://dl.dropboxusercontent.com/u/7090379/Orientation1.zip

xbox controller (Y) or Keyboard T teleports the player to various locations.

[zachkinstner]

@ddutchie, thanks for that feedback -- I'm glad it's working well for you and your testers! I'll need to create a "LeapLook" version of the demo so that others can try it out.

Thanks for sharing that build -- it loaded and ran for me without any problems. Looks like you've got Hovercast integrated nicely! I used the menu to change the materials, sky, and lighting in the scene. Once you're ready with a full prototype/release, let me know, and we can add your project to the new Hovercast Showcase page!

[ddutchie]

I had some issues with leap quads not overlaying leap input and decided to adopt the latest updates from unity nav branch.

I am unable to get the leap passthrough, thats probably disabled somewhere in one of the scripts.
Also xbox controller left analogue stick is very slow to move the character with the latest version.
Any ideas?

Great new system. Haven't had to look at documentation yet.

Only annoyance is that everything is prefixed with demo. Jks

But seriously. This is good.

I'll start making some color combos. Perhaps some more presets?

Sent from my iPhone

On 17 Feb 2015, at 7:31 PM, Zach Kinstner notifications@github.com wrote:

@ddutchie, thanks for that feedback -- I'm glad it's working well for you and your testers! I'll need to create a "LeapLook" version of the demo so that others can try it out.

Thanks for sharing that build -- it loaded and ran for me without any problems. Looks like you've got Hovercast integrated nicely! I used the menu to change the materials, sky, and lighting in the scene. Once you're ready with a full prototype/release, let me know, and we can add your project to the new Hovercast Showcase page!

—
Reply to this email directly or view it on GitHub.

[zachkinstner]

adopt the latest updates from unity nav branch

The unitynav branch was merged into master a while ago -- it's now obsolete.

I am unable to get the leap passthrough

I disabled these in Hovercast's demo environment. You can find the "Quad" objects within "LeftEyeAnchor" and "RightEyeAnchor" (these are all part of the Oculus prefab) and re-enable them. These "Quad" objects display the Leap Motion pass-through images.

Also xbox controller left analogue stick is very slow to move

Sorry, no ideas on this one. It seems unlikely that it's related to Hovercast.

Great new system.

Thanks!

[zachkinstner]

I created a new Input Modules document. It includes details about the LeapLook module.

[zachkinstner]

I created a build (PC/Mac) using the latest HovercastDemo-LeapLookVR.unity scene. I think there may be some additional improvements to make, but I'll track those in new issues.

Download the 2015-02-18 build >

[zachkinstner]

The change in #10 (increased "hit area") is a big improvement for the "look" mode. You can now make selections via the center/inner parts of the menu segments. Previously, the "hit area" was only along the outer edge of the segment.

[zachkinstner]

Watch the new "Leap Look" demo video >.