App runs but ARToolkit not working

[dtlafever]

I have tried both running the samples in unity and running a compiled version to the Hololens. The program runs, but the video preview is just a white box and the the tracking of the patterns does not work. Any thoughts on what could be going wrong?

I have tried using both unity 2017 and unity 5.6.f3. Have also tried both Visual studio 2015 and 2017.

[doughtmw]

Is the scale of the marker images that you printed off the same as what is listed within Unity? Check the ARUWP Controller asset -> ARUWP Marker and size in mm field. Also, depending on which test scene you are running, you want to make sure that you have printed off the correct marker script, as in the one that is listed under the File Name field in the ARUWP Marker script (for the HoloLensARToolKitSingle scene, there are marker images - Hiro, and Kanji - found here https://github.com/artoolkit/artoolkit5/tree/master/doc/patterns).

[qian256]

@Derekioh , did you import the ARToolKitUWP.unitypackage into your project?

[DoctorWarrior]

I have the same issues. Just to make sure that I did things correctly, here are some questions:

• I wanted to run the "single marker" sample. Which sample exactly should I use? I assumed it was HoloLensARToolKitSingle.unity. However, there were only Hiro and Kanji.
• which maker and what size exactly should I use? I found the marker in ARUMP Controller that are "hiro.patt" and "kanji.patt" and the sizes were "80mm" (I assumed that means they are 80x80mm). However, it was not working.
  Please respond here at your earlier convenience. Your help will be greatly appreciated. Thanks very much!

[qian256]

@DoctorWarrior

1. Yes. It only has Hiro and Kanji
2. Yes it is 80x80mm. Is the white light on that indicates the camera is running? Do you see the video preview plane?

[DoctorWarrior]

Thanks for your response!
Yes I see the light and the preview plane. However, the video preview is just a white box.
Pretty much I think its a thing related to the camera. I tested the camera on my Hololens and it works well. However when I run your sample there is nothing on the preview box. Any tips for troubleshooting? Thanks!

[qian256]

White light that is visible outside, indicating the camera is running. In your case, the camera is not running. It may be because the "Webcam" capability is not switched on.

[DoctorWarrior]

Thanks! BTW, where did you run the app? From unity or visual studio?

[qian256]

Deploy to HoloLens and run on HoloLens

[DoctorWarrior]

deploy from what? unity or visual studio?

[qian256]

I deploy it from Visual Studio.

[McCulloughRT]

I second this issue. I see the preview pane showing pure white. The white light on the exterior of the device is not lit, indicating the camera is not running, however I do have "webcam" capability enabled in unity settings. Also "Video" and "Track" both display 0 frames per second on the UI.

Using Unity 2017.4 LTS, and Visual Studio 2017. Trying to track the Hiro marker, printed at the correct size (I measured).

[qian256]

@McCulloughRT Is your HoloLens using RS4 software build?

[McCulloughRT]

Yes, it is. If it helps, I get a console message in the VS debugger:
Error (2): unable to open camera parameters file "Data/StreamingAssets/hololens1344x756.dat" for reading.

Maybe because the param file is "hololens896x504.dat"?

Looks like the message is coming from the native code, which I'm less familiar with, but I'll keep digging...

[qian256]

If it is RS4, then the main issue will be the MediaCapture Source Group order has changed. Please modify line 214 of ARUWPVideo.cs to

SourceGroup = allGroups[1],

[qian256]

Also, I noticed that with RS4, the performance of tracking is a little unstable compared to previous build. But I haven't got time to look into it.

[McCulloughRT]

Update: Changing the name of the camera parameters file (the actual file name, the public variable on the ARUWP Controller script did nothing) did fix the camera not turning on issue. I tried changing to allGroups[1] as you recommended but that broke the camera again.

The application is functioning now, but tracking is off by about 3 inches in each axis and is very unstable.

[qian256]

What is frame rate of tracking, and the video?

[McCulloughRT]

right at 30fps, rotational tracking is almost perfect, its just positioning thats off

[qian256]

Then the system is already running at the fastest. The issue is the lack of calibration. HoloLens don't how the relative pose between your eye and the camera. You should tune the magicMatrix in ARUWPMarker to manually offset it. It is a systematic and consistent offset.

[McCulloughRT]

Since rotation is fine and translation fixes need to be applied, I think I need a 4th row in the magic matrix to encode translation, is that correct? Or am I not understanding how the magic matrix is being used?

[linhdoan8]

@McCulloughRT Hi, I have a the same problem you had, that only the white box is showing but in my debugging console theres is no such error as

Error (2): unable to open camera parameters file "Data/StreamingAssets/hololens1344x756.dat" for reading.

it throws some exceptions and also errors that it could not open sensor information and access denied.

ERROR FindDefaultInterface:CM_Get_Device_Interface_List_Size failed:0x00000000, InterfaceListSize = 1
ERROR SensorOpenByType:Failed to open sensor error:0x00000490

I made sure that

allow unsafe code

is checked in Unity Player Settings. But @qian256 metioned in the How To section, that we have to set the Assembly C sharp to allow unsafe code, how can I do that, because in my visual studio there is no possibility to set 'allow unsafe code' only in the Unity Player Settings. Also tried to change the hololens dat files like @McCulloughRT but didnt help. Do you have any ideas where the bug lies? Were there anything else you have to set or configure?

Another strange thing is that in the debugging console many of these lines appear, if i give the permission to use the cam afer the app ask for permission:

onecoreuap\xbox\devices\api\winrt\pnpapiwrapper.cpp(385)\Windows.Gaming.Input.dll!712D2F3E: (caller: 712D5E27) ReturnHr(1) tid(1108) 8685C003 Access is denied.

Hope for your help and thanks!

EDIT : now it works! After some research, I had to turn the Scripting Backend to .NET. You can find it in Unity under File-->Build Settings-->Configuration-->Scripting Backend set to .NET instead of IL2CPP.
After that build the project and open it in visual studio. Go to Project-->Properties-->Build and make sure allow unsafe code is checked. No run and the cam should get on.

[linhdoan8]

@qian256 Thanks very much for the advice to set
SourceGroup = allGroups[1],
This also has to be set because the MediaCpature order in RS4 has changed and elsewise Visual Studio would give you a access denied error!