Marker position and object placement

[thezelus]

Hi Long,

Thanks for this amazing project and open sourcing it.

I am trying to use markers for defining initial position of holograms using following workflow

1. start the app
2. walk to a stationary marker (on a surface) and scan it
3. use the marker position to place the gameObject / hologram

I have couple of questions regarding this.

• After some quick Google-fu I found out that I can get marker position using ARToolkit (using the low level unity API), is it possible to do the same when using HoloLensARToolkit?
• When running the sample project with single marker tracking (Assets/Sample/HoloLensARToolKitSingle.unity) the frame rate is dropping down to 7-9 fps, is that expected? I haven't made any modifications to the code.
• Is there a way to control the lifecycle of video capture / marker tracking so that it can be triggered based on user input. I am trying to keep the FPS stable and I am assuming that this might be helpful when doing that.

Any other options and suggestions are definitely appreciated.

[qian256]

is it possible to do the same when using HoloLensARToolkit?

Yes, in the ARUWPMarker script, the function getMarkerTransformationMatrix returns the current transformation from the marker to the camera, in Matrix4x4 format. You can use it with the function getMarkerVisibility to check if the marker is currently visible.

the frame rate is dropping down to 7-9 fps, is that expected?

It is not normal, for me, it is at least 20fps. When you build the app from Visual Studio to HoloLens, did you choose the release version? or choose "start without debugging". When it is being debugged using VS, the frame rate is low.

Is there a way to control the lifecycle of video capture / marker tracking so that it can be triggered based on user input

Yes, in the ARUWPController script, there is a parameter enable. I think you can tune this to enable/disable tracking.

[thezelus]

Hello @qian256 ,

Thank you for the response and apologies for late reply. I tried using the enable parameter and stopRunning() and startRunning() methods. They seem to be working as expected, though there seems to be a different issue that might be related to the life cycle.

The web camera on HoloLens doesn't turn off (white LED indicating camera being active is on) even when ARUWPController is disabled as and all related gameObjects (webcam plane, display HUDs for FPS and AR etc.) are deactivated. Is there a way to explicitly turn off the web camera on HoloLens after ARUWPController has been disabled?

[qian256]

The functions stopRunning() and startRunning() controls the native libraries, not the behavior of the hololens camera. If you want to shut down the camera, you can try use Unity functions of WebCamTexture.Stop(), and you need to code a little bit to achieve it.

[thezelus]

Hello,

Thanks for the response and the suggestion. Yes I am already using WebCamTexture to shut down the camera but shouldn't the controller do this automatically specially when it explicitly starts the camera.

 void Start () {
        ARUWPUtils.aruwpRegisterLogCallback(Log);
        ARUWPUtils.aruwpSetLogLevel((int)currentLogLevel);
        lastTick = Time.time;
        deltaTime = 0;
        startWebcam();
        startARUWP();
    } 

  private void startWebcam() {
        // init Webcam
        WebCamDevice[] cam_devices = WebCamTexture.devices;
        webcamTexture = new WebCamTexture(cam_devices[0].name, frameWidth, frameHeight, 15);
        if (displayWebcam) {
            webcamMaterial.mainTexture = webcamTexture;
            webcamPlane.SetActive(true);
        }
        else {
            webcamPlane.SetActive(false);
        }
        webcamTexture.Play();
}

What is your opinion about stopping the webcamTexture in either stopRunning() or OnDestroy() or shutDownAR()? I can create a pull request if you think this is something that should be implemented.

[qian256]

Thank you for the proposal of adding the function to properly shut down the webcam.
I suggest adding a new function called stopWebcam() disable it. What is your opinion? I would definitely merge your implementation if it works.

[thezelus]

Hello @qian256

I have already been experimenting with this and have added two methods to my local clone of the repository

 public void stopWebcam()
    {
        if(webcamTexture != null)
        {
            webcamTexture.Stop();
        }
    }

    public void restartWebcam()
    {
        if(webcamTexture == null)
        {
            startWebcam();
        } else
        {
            webcamTexture.Stop();
            webcamTexture.Play();
        }
    }

Below are my observations:

• Calling stopWebcam() does stop the web camera and this works fine the first time.

• When trying to start the camera after stopping it from the code results in no video feed, which I suspect is due to this issue: Unity/Linux - Unity does not free webcam when stopped

• I also tried to destroy the object that has ARController and ARMarker scripts attached, this does solve the issue but is a hack and is a finicky solution.

(by the way, should we move this to a separate issue?)

[qian256]

Thank you for the update! Yes please go ahead and open another issue regarding the control of camera feed, and the control of tracking functionality.

[dtoceaneering]

is it possible to put an Object at few distance from cube marker in scene (HoloLensARToolKitCube)?