Creating an augmented reality app with 4DViews Volumetric Capture

CLick Here For a Video of the Capture + iOS APP

For my second volumetric class I wanted to not only get an engaging capture, but also deploy it to an augmented reality (AR) app on my iPhone. After successfully deploying an AR app using a 4DViews Capture + Unity plugin, I was very aware of the lack of tutorials and documentation around each step of the process. To make easier for my classmates to deploy and showcase their awesome work, I decided to make a step-by-step guide.

While this is a good, general Vuforia tutorial, this is really meant for people utilizing a 4DViews system. I have my full repository uploaded, this guide should be follow step by step. Also, I am using a Mac and I'm deploying to iPhone. Using windows and android should not only be similar but easier to deploy.

I am not an iOS developer, so I am encouraging people to submit Pull Request if their code is better. (It probably is)

Apple Developer Program

If you are not planning on deploying to iPhone you can ignore this.

While trying to deploy my X-Code app I was forced to sign up for a developer account. Some people have said I didn't need to spend the 100 dollars to be enrolled in Apple's program, but I haven't figured out how. I believe you need the securiry certificates because we're using "Advanced Features" (AR).

Click here to begin enrollment in Apple Development Program

Click here to Download X-Code

Step 1

Capture, Render, and Export your volumetric capture off the 4DViews system.

4DViews exports a filetype .4ds, you need plugins to make Unity understand the system's capture. Typically, 15 second capture, at 30fps would be a little less than half a gigabyte.

Step 2

Download Unity Hub + Unity 2018.4.13f1

Click this link to Create a Unity account (if you don't have one)

Click this link to download Unity Hub

Sign Into Unity Hub with your new account and click License Management. Click "Manual Activation". If this isn't popping up click the settings button on the top right.

Click Save License Request and save it somewhere easily accessible. Either click on the link below "Save License Request" or click here.

Upload the file you just saved and click next.

On the next page make sure to click "Unity Personal Edition". Then click "I don’t use Unity in a professional capacity."

Download the new license file in a place you can easily find it and return back to the unity hub.

On Unity Hub click next and locate the file you just downloaded. It should have an filetype of ".ulf". Click Confirm. Now go to the top right where it says "preferences" and click back.

Click the "Installs" tab on the left and then click "Add".

Locate Unity 2018.4.13f1 LTS and click next. Now chose which Modules you want.

• Check Vuforia Augmented Reality Support
• If you are on Windows and deploying to android chose Windows Build Support and Android Build Support".
• If you are on a Mac and deploying to iPhone chose Mac Build Support and iOS Build Support".

Let it Download.

Step 3

Setting up Vuforia

Create a Vuforia Account.

Create A Development Key

• Click "Get Development Key" and name it.
• On the top toolbar next to "License Manager", click "Target Manager"

Target Manager is where you are you uploading your "targets", or what you want your AR app to cast on. Now the photos have to be less than 2mbs, so you might have to lower the file size using an app of your choice.

• Click "Add Database" on the Top Right and name it something relevant. For the "Type" chose "Device".
• Click on the database you just created, and then click "Add Target".
• Chose Single Image as the type and file location
• It might throw an error, usually dealing with size or color space. These are usually an easy fix.
• I would sit the width to "10" and name it.

Now once you uploaded your photos, refresh the page. It will automatically rate your photo. The rating is based on how many target points it can detect. I would try different images until you get at minimum a three star photo. Now you really only need to upload one photo here, as this tutorial deals with one target.

• Click "Download Database(ALL)" and save the file somewhere you can easily access it.

Step 4

Downloading the 4DViews Plugin

Click this link to download the 4DViews Unity plugin

Step 5

Creating your project + Getting Packages

Go back to Unity Hub and click "Projects" on the left menu bar and then click new. If you have multiple Unity installs make sure to use 2018.4.12f1.

Chose "3D", name it, and store it somewhere other than your Desktop.

Now with Unity open look at the top toolbar for "Window" and chose "Package Manager". Search for "Vuforia" and click install.

Next under Assets click "Import Custom Packages" and import both your database and 4Dviews plugin.

A window will appear with everything you are importing, just click import.

Step 5

Getting Vuforia Working

Make sure you are viewing "Scene" and not "Game"

First thing you need to do is right click both the main camera and directional light and delight them. Next right click where the camera and light were, find "Vuforia" and click AR Camera.

Next click the AR Camera and look over in the "inspector". You should see a button called "Open Vuforia Engine Configuration". Click it.

• Go back to your development key and copy the entire string.
• Find "App License Key" and paste your key.

Under Databases, what ever the name of the database you imported should appear. I also uncheck Video Background

Next right click AR Camera and chose "Image". Now what ever photo was in your database should autoappear. If not look in the inspector for Image Target Behavior.

Step 6

Adding a 3D Object + Testing

Now we need to add a test object. Right click the ImageTarget, click 3D object, and then Cube. I had to scale it down (Press R) to be a more relevant size to my target. Next I move it upwards so it's fully on top of my image. I move it by pressing W and dragging up, you can also do this in the inspector.

Click Play

Now your webcam should turn on and it should auto switch to the game mode. What ever photo you uploaded to the database pull up on your phone, and if the glare isn't too bad the cube should pop up.

If your webcam comes up black, I found deleting the AR Camera and all its children + readding them should work.

Step 7

Adding your 4DViews asset.

• Now that we got our cube to pop up on top of our target image we can delete the cube.

• Next under the project window click Assets -> 4DViews -> Prefabs

• Drag the asset called "Sequence4Ds" to "Image Target" so it becomes a child.

Locate your 4dViews file (.4ds) and add it to the StreamingAssets folder.

• Click the drop down next to Sequence4Ds in the hierarchy (where your camera and image target are).
• Click on "Mesh4ds" and look at the inspector.
• Locate the section "Plugin 4DS (script)
• Drag your 4DSViews file to this bar and it will autopopulate the path.

Your 3D asset should pop up. Make sure to scale it up or down to be relevant to your target.

Click play and your app should work.