Design Sprint 3: Design For Another World

Western Shootout VR Game for DS3: Design For Another World for CS485-1 Human Computer Interactions Spring 2023 at Emory University
Made in Unity 2021.3.20f1, compatible with iOS or Android devices.
By Samantha Lin @Seraclin, Emma Klein, Muhammad Shahid, Allen Zhang @theJokerEvoker, Jonathan Kim @JKim0212

Demo Video:https://youtu.be/MGABwUmfoZM

How to Play: Western Shootout

Tilt your phone device to aim the gun. You can shoot either by tapping the screen, using the H key, or using an external controller with a bumper button. VR mode can be turned off by tapping the X, and can be turned on again by tapping anywhere on the screen. (To change the keybind for shooting, you will have to edit the SimpleShoot.cs script)
Shoot the "Shoot to Start" target to start the game. A timer will appear and start counting down. Shoot the targets that pop up before time runs out. Red/blue/rabbit targets are worth 10/20/50 points each. When time runs out, the targets will disappear, and your final total score will be displayed in the top right.
To play this VR game, download this repository and open the DS3_Game folder in Unity 2021.3.20f1. Then build the Desert_Sam scene in the "Assets/Scenes" folder via File > Build Settings for your respective device following the instructions detailed below in the Android/iOS VR with Unity section.

Tips and Tricks

I used Unity 2021.3.20f1 on Windows 10 with an Android phonen for VR development. And here are some common things I came across. There might be different behavior for MacOS and iPhone or different versions of Unity.

Getting Started

Here's my recommended start setup for using iOS/Android with Unity:

1. Install the latest Unity LTS with the Android and iOS modules
2. Make a new project in "Unity 3D (VR)" and update the packages it comes with. And make sure to have these built-in packages from Package Manager: XR Interactions Toolkit, XR Plugin Managment, and OpenXR Plugin
3. Import the git package Google Cardboard SDK to your Unity project
4. Go to the Google Cardboard SDK Sample scenes (e.g. VrMode) and copy the Player gameObject.
   • If the CameraPointer.cs script is throwing errors/warnings just remove it or comment out the lines.
5. Now you should be able to put the PlayergameObject into any scene, and it should make it into a VR gyroscopic view when you build it to your device. Just make sure there's only one camera in the scene.
6. Build the scene(s) to your iOS (requires XCode) or Android device (needs to be in developer mode). Make sure to follow the Cardboard quickstart guide.
   • Make sure for Android go to 'Edit > Project Settings > XR Plug-in Management > Android' and have "Initialize XR on Startup" unchecked, and to only have "Cardboard XR Plugin" and "OpenXR" checked.
   • For iOS, 'Edit > Project Settings > XR Plug-in Management > iOS' and have "Initialize XR on Startup" unchecked, and to only have "Cardboard XR Plugin" and "ARKit" checked. DO NOT UPDATE YOUR IOS DEVICE UNLESS IT'S COMPATIBLE WITH THE LATEST STABLE XCODE VERSION.
7. Check that it works on your device. More details can be found below.

General Unity Stuff

• For general VR development with VR headsets (e.g. Oculus, HTC Vive, Valve Index, or any device compatible with SteamVR) and motion controllers, you should use the XR Interactions Toolkit package. You can also use XR Device Simulator to simulate a VR headset. Both packages can be installed via "Tools > Package Manager > XR Interaction Toolkit" and make sure to import the Samples: "Starter Assets" and optionally the "XR Device Simulator". You can find the controls for the device simulator by double-clicking the Default XRI Input Actions asset or opening the Input Manager. The Unity 3D (VR) starting template also comes pre-installed with many of the needed packages but make sure to update them. A good XR tutorial can be found here.
• Unity Input has two seperate systems: the old built-in Input Manager and the new Input System Package. The new Input System package (installed via 'Tools > Package Manager > Input System') is a bit more difficult to understand, but it is much more flexible than the older input system, especially for different controller inputs. To check what input system your project is using for your platform(s), go to 'Edit > Project Settings > Player > Other Settings > Configuration > Active Input Handling'. I generally like to set it to "Both" as the XR Interaction Toolkit utilizes the new Input System. Here's a tutorial for the new input system.
• For Canvas/UI elements, the Canvas 'Render Mode' should be changed to World Space and change the Canvas z position to somewhere in front of the player. If you don't do use World Space, UI elements wll appear in the middle of your camera rather than spaced out for VR. You also need to make sure that the Canvas doesn't clip into close-up objects. This can be done by adjusting it's distance from the player and resizing it to be consistent (requies some math).
• Optional: If you find your project to be running pretty choppy/laggy, you might have some performance issues. There's a trade-off between good graphics vs. good performance, and for VR having a high, consistent framerate is very important to the user experience. However, mobile devices are not very powerful, so you might have to spend some time balancing your framerate and graphics quality. Try to minimize having shadows, multiple/detailed material shaders, reflections, realtime lights, and large meshcounts. Basically, you want to minimize your graphics quality as much as possible. This video series covers VR optimization. Also Unity has something called the Universal Render Pipeline (URP) Package which could help as it's somewhat more efficient than the Standard Unity shaders and is covered by this video. Overall, this is kind of advanced stuff, so only mess with this if you have no other options.

Git and Version Control

• If you are using Git for version control, make sure to use the default Unity .gitignore. You might have to remove the front '/' for the lines [Ll]ibrary/ to [Mm]emoryCaptures/ (e.g. /[Ll]ibrary/ --> [Ll]ibrary/) if your project directory isn't in the same directory as your .gitignore.
• Unity .scene and .prefab files don't merge nicely, so make sure only one person is editing them at a time. Otherwise, you will have to deal with merge conflicts.

Android/iOS VR with Unity

• In Unity Hub when you're installing the latest LTS version of Unity, make sure to add modules for other platforms (e.g. Android, iOS, etc.). You can alternatively install an older version (e.g. 2019) since some features (e.g. WebVR) have become deprecated in newer versions.
• If you're using an iOS/Android phone, your best bet is to use the Google Cardboard SDK which is designed for the Google Cardboard. Here's the documentation for setting it up in Unity with a video walkthrough. It should come with basic gyroscopic movement and stereoscopic rendering. It was recently updated in 2021 from the now-deprecated GoogleVR, so there's not many tutorials about the new version.
  • If the Cardboard SDK isn't working, you can try the older deprecated GoogleVR version. Here's a video tutorial for the older version, but it most likely requires an older version of Unity (e.g. 2019). There are no guarantees that the older version will work.
  • Here are some Carboard SDK common issues
  • For Android, make sure in 'Edit > Project Settings > XR Plug-in Management > Android' to have "Initialize XR on Startup" unchecked, and to only have "Cardboard XR Plugin" and "OpenXR" checked.
  • For iOS, 'Edit > Project Settings > XR Plug-in Management > iOS' and have "Initialize XR on Startup" unchecked, and to only have "Cardboard XR Plugin" and "ARKit" checked.
• Sensor inputs for Android or iOS can be found here. Generally, you'll want to use the Gyroscope and AttitudeSensor. These have to be enabled via script in order to be used. Note: Gyroscope returns a Vector3, while AttitudeSensor returns a Quaternion.
  • If you are directly using the gyroscope and attitudeSensor, you have to do some Quaternion math to rotate them correctly in Unity. A formula for this can be found here (note: this formula will only rotate the camera with respect to your device's position rather than to your current position. You can add in an offset to fix this).
  • Another code snippet can be useful for reorienting a TrackedPoseDriver to face a certain direction.
• To build your project to iOS, you MUST have Xcode, which is only available on MacOS. Thus, you cannot build to an iOS device unless you have a Mac (or an emulator at your own risk). Here's a tutorial for building to Apple for testing. I recommend downloading directly from the Xcode downloads rather than installing it from the App Store.
  • Note: iOS can only have up to 3 devices registered for developement.
  • If Xcode is throwing an error about "Failed to Prepare Device for Deployment..." and incompatable versions, do NOT update your iOS device until trying: a) restarting your iOS and Mac devices, b) checking your version of iOS and MacOS is compatible with an older version of Xcode, c) checking the latest iOS version and MacOS is compatible with the latest stable Xcode version. Sometimes Apple will publish an iOS update and not update the compatibility with the latest stable Xcode version. It's difficult to revert iOS versions, so it's easier to download an older version of Xcode. If you already installed the latest iOS update and discover it isn't compatible with the latest stable Xcode version, some workarounds are:
    • Download a beta version of Xcode that is compatible with your iOS version. This might involve updating your MacOS too.
    • (Requires an Apple Dev account) This method allows you to keep your stable Xcode version while making it compatible with the latest iOS. Download the RC3 build from developer.apple.com - extract the xip; Right click, "show package contents", then navigate to the directory Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport; Copy the 16.4 (or your latest iOS version) directory to your local installed XCode instance (same directory Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport); You will probably find that you need to copy the AppleWatch (9.4) and AppleTV equivalent directories as required to solve the problem.
• For Android, there shouldn't be a restriction. Here's a tutorial for building for Android for testing.
  • If you're getting an error like Theme.AppCompat.NoActionBar not found, you might have to check Custom Main Gradle Template and Custom Gradle Properties Template in 'Edit > Project Settings > Player > Android > Publishing Settings > Build Settings' and edit them accordingly.
• For viewing on Android/iOS phones without having to build/export your project each time, I recommend using the Unity Remote app which can be downloaded from your respective app store. Connect your phone via cable to your computer (Note: iOS requires also having iTunes open) and have both the app and Unity editor open (if not detected, you have to replug or restart the editor). In Unity, navigate to "Edit > Project Settings > Editor" and select your device in the Unity Remote section. When you press play in the Editor, you should also see the scene on your phone and receive some inputs from your mobile device. Note: the framerate will be significantly lower on Unity Remote than if you had build it.
  • If you're using Cardboard SDK, Unity Remote won't work, and you will have to build the project in order to test it.

The ShineCon Bluetooth Controller

• The ShineCon Bluetooth controller has different behavior between Android and iOS devices. When I paired it to my iPhone, it didn't do anything in VR mode besides change the Volume +/-. When you put it into Music mode on iOS (@+B) it allows you to control music Prev/Next Song with volume controls, but only within the music app. Since it's not "Made for Apple", it's not considered a controller, even though the instruction manual describes it as an "iCade" controller. I observed that it functions like key inputs on iOS when in VR Mode:
  • Up Stick w/e
  • Down Stick x/z
  • Right Stick d/c
  • Left Stick a/q
  • @ button o/g
  • Power button l/v
  • A - volume down
  • X - volume up
  • B - u/f
  • Y - j/n
  • Bumper h/g
• For Android, it seems that the input controls are read differently:
  • Bumper - Joystick1Button0
  • B - Joystick1Button1
  • Y - JoystickButton3
  • The actual Joystick, A, and X weren't detected
• I found the controller can only connect to iOS or Android devices via Bluetooth. I tried to connect it directly to my laptop, but got a 'Driver Error'.
• Unity Remote does not register the controller input, so you may have to build the project directly to the phone for testing.

WebXR/VR for Unity

• I don't recommend using WebXR since it's very experimental. Unity XR does not support WebGL, so it is not guaranteed to work on certain devices/browsers and is basically unsupported on iOS. Use at your own risk.
• In general, the WebVR API is no longer supported and is deprecated. It might still work on some older browsers and devices, but it's unlikely to work on most modern browsers. Also a lot of the packages used are now deprecated and work best on Unity 2019.
• The replacment WebXR API is the newer version, but it's still pretty experimental.
• WebXR Exporter by Mozilla Firefox is deprecated and not officially supported by Unity, so there is no guarantee of it working properly for certain platforms/devices or future versions of Unity. Instead there's the De-Panther WebXR package which is a basically a maintained version of the Firefox WebXR. It can be used to make WebXR experiences similar to A-Frame using the WebXR API. Although, I found it to be quite buggy and laggy on my iPhone and laptop, so it seems Android devices perform the best. You need to have the WebGL module installed. You also might need to use an older version of Unity. An (outdated) tutorial can be found here.
• If you're getting an graphics error, you need to make sure to go to 'Edit > Project Settings > Player > WebGL > Other Settings > Rendering > Color Space' and set the Color Space to "Linear" with the Auto Graphics API unchecked. Alternatively, you can use "Gamma" Color space with Auto Graphics API still checked, but it tends to perform worse.

Finding Assets

• The Unity Asset Store has a bunch of free 3D assets. A lot of them even come ready-made with animations and scripts, for example this gun asset. Just make sure it is VR compatible and not too graphics heavy.
• DevAssets by Brackeys has many free Unity assets. Brackeys also has many useful Unity tutorials on his YouTube Channel although he retired from making tutorials in 2020.
• SketchFab also has some free .fbx 3D models, but it's less likely that they will be compatible with Unity.