Extended_Reality

Built in XR features.

Currently, LUTE has a number of XR orders available for people to use.

Toggling XR on and off

To start off, if you want to open up the phone camera and start using XR, you will need to use the Toggle XR order. This automatically initialises the XR system and opens up the camera.

The order itself has a toggle to enable or disable. It has two default prefabs for plane visualisation and point cloud visualisation, which you can look at and change to fit your needs. You can also chose if the plane detection only detects horizontal, vertical, or both kinds of planes.

Image Detection and Tracking

One of the most popular uses of AR on a phone is detecting an image and tracking it - LUTE has an order built in for that. The Track Image order consists of a list of images to be detected:

Once you add an image with the plus symbol, you have a number of options to fill in:

• The texture of the image you want to track;
• A name of the XR object so you can reference it in other orders;
• The physical width of the actual image, as the computer vision algorithm used needs it;
• The position the provided game object should be spawned at as an offset from the centre of the image; and
• The object you want to spawn on top of the image - this can range from a simple overlay image to a 3D object and more.

!!Important information regarding the tracking image!!

The order uses the unity ReferenceImageLibrary in the background, and adds the images during runtime. This could potentially lead to an instance where your image doesn't have enough features for the computer vision algorithms to work, in which case it would silently not work and you will be left wondering why. For this, ensure the images you decide to use have enough features. Google provides a list of best practices in creating image markers here, and here is a tool that you can use to check how unique the image you have chosen is. Keep in mind, the features detected by the webpage might not be the same as the ones by Unity, but it can be used as a guide. For example, here are two images, one bad, and one good:

Another very important thing For the script to be able to access the images during runtime, each image needs to have the Read/Write flag set in the Import Settings:

If you have followed the two important things, you should now be able to use the order and it should track the image once detected, putting the 3D object on top of it. Keep in mind that the way the algorithm works, the more images you have, the more computation needs to be done on each frame, thus slowing down the app - don't use too many images, and test often.

Place object on a plane

This is the simplest way to add a 3D object to the AR feed on the phone. There are a number of options to choose from:

• First off, the actual game object you want to place in AR;
• You have to give the object a name, so you can reference that object in other orders;
• You can specify if you want the object to specifically be put on a Horizontal or Vertical detected plane, or if you don't care which one, you can choose None;
• You can either choose for the phone to automatically place the object in the scene as soon as it detects the specified plane alignment, or if the user has to click on the screen to place it on that plane;
• Then you have 3 options, where you can select whether the user can rotate, scale, and move the object around;

Place Object at Location

For this, you need to have created a LocationVariable in the LUTE engine beforehand, with the specified GPS coordinates you want for the object to spawn at. You then have access to that Variable in the location row, as well as the game object you want to place, and a given name to that object for later reference.

Interact with XR object

This order waits for the user to interact with the specified XR object - if you don't provide anything else to the script, it only wait for a touch interaction, and then continues. You can provide a class that is derived from InteractorEvent which invokes the onInteracted UnityEvent when you want it. The order itself just subscribes to the event and wait for it to be invoked.

You can also add any object from the scene to the On Interaction() list, and it will be invoked when the InteractorEvent gets finished.

Toggle Plane detection on and off

This order basically enables and disables the plane detection and the rendering of the planes.

This is an example of how the planes look in the Unity built in XR Environment.