Add a default MRTK profile for HoloLens 1 Interactions

[DrDecipher]

The original question / bug highlighted a gap in usability/configurability of MRTK for HoloLens 1. By default interactions are set up for HoloLens 2 and there's no easy way to get to a HoloLens 1 friendly MRTK profile. This item tracks adding a config profile that's hololens 1 friendly (for HoloLens 1 interactions).

This profile should use Gestures input simulation mode instead of articulated hands to simulate how HoloLens 1 works.

Original Question

Gaze cursor is no longer re-enabled after using the 'Hand' in the latest release and can not be altered manually via the inspector.

Steps to reproduce the behavior:

1. Press Play( Gaze Cursor will be visible/active highlighting correctly )
2. Hold Down the 'Space Bar' to engage 'Hand'.
3. Release 'Space Bar'
4. Gaze Cursor remains inactive and disabled.
5. Can not be activated in the inspector as something is turning it off per frame.

Expected behavior

After using either 'Hand', the Gaze cursor should reappear and be active after releasing the space bar.

Your Setup (please complete the following information)

• Unity Version
  Unity : 2019.1.6f1
• MRTK : MRTK RC2.1

Platforms

• HoloLens
• HoloLens 2

[wiwei]

This is by design AFAIK to be consistent with how the shell works (i.e. gaze cursor visible, then hand shows up, you have to say select order to get the cursor to show up again).

@julenka FYI, also to correct me if I'm wrong here.

Is there a specific scenario/use case that you're getting blocked on because of this?

[DrDecipher]

We are developing for both the HL1 and the HL2. When testing we are often checking controls with an air-tap and a hand. This type of behavior should not be hidden. As great as the MRTK2 is becoming there seem to be little consideration for the fact that many of us will be stuck in 'HL1 land' for the foreseeable future. How can I reenable gaze it after the hand is used in the editor? Any assistance would be appreciated :)

[wiwei]

@julenka, Q for you, maybe a question for folks who don't want the default shell behavior that the MRTK is mimicing

[DrDecipher]

It would seem the behaviors don't need to be contradictory. I wrote a service simply to 'add back' the ability to use the 'left mouse' with gaze 'air-tap' that triggers the 'Select' event. All I need is the ability to get the gaze cursor back. I would really think this would be a good addition to the UI. H_ow can I re-enable/intercept whatever is forcing the cursor to be disabled?_ Upon release of the space bar I need the cursor back. Thank you again for the quick responses. I know how much effort you all are putting into the tool kit and it is appreciated!

[julenka]

Hi you can just press the “1” key and it will trigger a simulated “select” voice command, which will bring back the cursor. You can also press “1” to click items with that head cursor. Sent from phone, please excuse typos.

…

________________________________ From: DrDecipher <notifications@github.com> Sent: Tuesday, June 25, 2019 2:07:43 PM To: microsoft/MixedRealityToolkit-Unity Cc: Julia Schwarz; Mention Subject: Re: [microsoft/MixedRealityToolkit-Unity] Gaze Cursor lost after using Hand (#5046) It would seem the behaviors don't need to be contradictory. I wrote a service simply to 'add back' the ability to use the 'left mouse' with gaze 'air-tap' that triggers the 'Select' event. All I need is the ability to get the gaze cursor back. I would really think this would be a good addition to the UI. H_ow can I re-enable/intercept whatever is forcing the cursor to be disabled?_ Upon release of the space bar I need the cursor back. Thank you again for the quick responses. I know how much effort you all are putting into the tool kit and it is appreciated! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2FMixedRealityToolkit-Unity%2Fissues%2F5046%3Femail_source%3Dnotifications%26email_token%3DAABJELBWKRS2NK67WZXNVE3P4KCJ7A5CNFSM4H3LGYHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYRTFSQ%23issuecomment-505623242&data=02%7C01%7Cjschwarz%40microsoft.com%7C6a6e1b29775d4b215a4b08d6f9b12ac3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636970936650672810&sdata=nm%2BEYetih0OmOqFYAU0qUFUwOKsYa70oLthEIkNU4Ds%3D&reserved=0>, or mute the thread<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAABJELAGZHGVBJMXXFHYX6TP4KCJ7ANCNFSM4H3LGYHA&data=02%7C01%7Cjschwarz%40microsoft.com%7C6a6e1b29775d4b215a4b08d6f9b12ac3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636970936650672810&sdata=VmeKwpr3MqJ8OIfcDiKNIFGqlsd2AdZSeHvLlSsvGQ4%3D&reserved=0>.

[DrDecipher]

That is something I did not know and a great workaround that I can use for now, but this is a bigger issue than is being recognized in this development. Please take the following as constructive and it is directed at the development as a whole not you personally:) Anyone who makes a living developing HoloLens applications needs to be able to use MRTK with the HL1 or we simply will not use MRTK wwhich would be a shame. We have clients, now, that we must test for and support. It is confusiong why the simplest of things to test HL1 apps where removed. The HL2 has no timeline for release and it's being treated as if we are all developing for it right now. I'm not proposing backward compatibility for anything in the old tool kit, just a workflow for the HL1 that does not require extra steps. You must understand when we are testing an enterprise application and send it to QA we can not provide extra steps to them, as it is easy to confuse a user error with a bug. Does that make sense? I assure you it's not intended as a rant ;)

[DrDecipher]

Thank you for entertaining this thread. I'm trying to get more of our team to use MRTK and the above mentions are a show stopper in production right now. It is possible to get the behavior back? 1: When releasing the spacebar to get the Gaze cursor. 2: Can '1' be remapped to Mouse 0 somehow. This is all we need to adopt MRTK fully.

[julenka]

Anyone who makes a living developing HoloLens applications needs to be able to use MRTK with the HL1 or we simply will not use MRTK wwhich would be a shame. We have clients, now, that we must test for and support. It is confusiong why the simplest of things to test HL1 apps where removed.

There is a way to test HL1 behavior in the editor, and we've tried to make sure that all our components still work with HL1 inputs. The way to use HoloLens 1 behavior is currently a bit hidden in the editor. The default hands that are enabled in MRTK are "HoloLens 2 hands" aka "Fully articulated hands". To get behavior that matches HoloLens 1, you will want to change your input simulation profile to use "GGV Hands" aka "HoloLens 1 hands". When you do that, you will see that when you let go of spacebar the gaze cursor will appear again.

How to change input simulation profile (using MRTK RC2 release)

1. Select MixedRealityToolkit
2. Under Input, hit "Clone" to create a custom input profile
   
3. Expand Input Data Providers, then under Input Simulation Service, clone to create custom input simulation profile
   
4. Now that you can customize the input simulation service, scroll down until you see "Hand Simulation Mode, and select 'Gestures' instead of "Articulated".

This will update the input simulation system so that the type of hands that will be used are HoloLens 1 hands, and then when you drop your hand, you will still see the cursor.

[julenka]

@wiwei maybe the action from this issue should be that we should provide a DefaultHoloLens1ConfigurationProfile? I've already started making one. Here's a branch with MRTK profiles specific for HoloLens 1 that people can use to test HoloLens 1 behaviors: https://github.com/julenka/MixedRealityToolkit-Unity/tree/issue/hl1profile . The profiles are under Assets/MixedRealityToolkit.SDK/Profiles/HoloLens1/

[julenka]

Hopefully this answers your question, hopefully I understood the question correctly. If you want more help with how to set up MRTK so that you can test HoloLens 1 behavior in the editor, please consider asking a question on stackoverflow to ensure that others can also benefit and also in case somebody else can help answer. Just make sure to add tag mrtk.

[DrDecipher]

### This is wonderful and thank you again for all of this! A HL1 profile would be a great addition to the Kit. and I will decminate the above to our entire team.

[DrDecipher]

After a lot of testing we're still stuck so I have posted to Stack Overflow an extended explanation of the need. For anyone who finds this thread head here for more...
https://stackoverflow.com/questions/56821449/hl1-input-for-mrtk-v2-1

[michel-de-verdelhan]

Same problem here : We have applications we want to port to the new MRTK in order to run on futur Hololens v2, but we also need to support Hololens V1 for all our clients already using it.

the debugging in the editor is a real pain due to gaze cursor problem and we havn't found any way to simulate it properly.

[julenka]

the debugging in the editor is a real pain due to gaze cursor problem and we havn't found any way to simulate it properly.

Have you found Gestures mode (see instructions above) sufficient? If not, what isn't working for you when you use Gestures mode?

[DrDecipher]

The issue is that in gestures mode MRTK2 buttons and UI elements are not triggered by the left mouse (Air-Tap) when using gaze.

Prior to 'Hand' emulation being added gaze and left mouse behaved as an air-tap. After they were added the left mouse no longer acts as an air-tap with the mode set to 'Gestures'. All you can do in Gestures mode is move the camera and see the gaze cursor, but no interaction. I believe it was mentioned that pressing #1 would initiate a 'Select', but there is no way to map this to the left mouse as it was with HL1.

We want the old behavior when set to Gestures. Left mouse = Select/Air-Tap/Press just like when you build to the HL1, but in the editor.

Does that make sense?

[DrDecipher]

I did post this to Stack Overflow and Slack as well, but have received no answers. :/ This seems so simple, but I have not found a way to accomplish it with the new emulation.

[julenka]

I did post this to Stack Overflow and Slack as well, but have received no answers. :/ This seems so simple, but I have not found a way to accomplish it with the new emulation.

Hi sorry for not replying yet! We have seen your question and I've been thinking about it. I think I understand what you want but there's no built in way to do this. Trying to build something now, I just had an idea. I'll post it to stackoverflow if I can get it working: https://stackoverflow.com/questions/56821449/hl1-input-for-mrtk-v2-1

[julenka]

Hi, I posted an answer that I think answers the question you asked. I provided a script that will allow you to simulate input down and up using just a mouse, without needing to bring up a hand via spacebar. I think this is what you were asking, but not sure. It might help to edit the question down a bit to focus on just one specific question, to help folks better distill the actual question you're asking. Stackoverflow has a great guide for how to ask good questions that I would recommend reading.

I've also attached a .zip containing both the script that I think will solve your problem as well as a prefab showing how to use the script. Simply drag the prefab onto your scene to get the desired behavior.

SO Mouse Only Input Simulation.zip

[DrDecipher]

I will have an opportunity to test this in the next couple of days. Thank you for the effort in advance!

[DrDecipher]

This is exactly the behavior we have needed. Thank you so much for your effort. This is a little thing that is huge to us :)

[julenka]

This is exactly the behavior we have needed. Thank you so much for your effort. This is a little thing that is huge to us :)

Glad it helped! Would you mind please marking the stackoverflow answer I wrote as the correct answer, and then upvoting, so others can also benefit if they look for the same info in the future?