When the application launches for the first time, two objects will be created in front of you: a status window showing the current connection status to external services which provide data for visualization (see /net/LolaCommsNative/LolaCommsNative/iface/tools for a small set of tools to send data for testing without a real robot), and a coordinate frame axis which represents the robot's world origin.
By default, all network services are active in the application. On startup, the Vision and Pose services should be healthy (green) even if their respective provider is not yet available on the network. Tapping the Reset button for any service will shut down any active connections and reinitialize. The port used for each can be changed by tapping on the textbox and entering a new port with a bluetooth keyboard (holographic keyboards are not yet well supported in Unity).
To set the world origin, first tap on the yellow center sphere to select it. It should begin flashing, and as you move your gaze it will snap to the world mesh generated by the Hololens. Tap again, anywhere, to deselect the origin and anchor it in its current position. To change the origin's orientation, tap on the sphere for the axis you want to rotate around (red == x, green == y, blue == z) to select it, then pinch and drag your fingers vertically to rotate about that axis. Tap again to deselect the axis.
Once the world origin has been changed, it will be anchored in place and this will persist across application sessions. This means the origin only needs to be re-positioned if the robot's world origin changes. Future work will include automatically positioning the origin.
The ideas behind this code are published under:
The results of these algorithms can be seen on:
Compilation of Autonomous Walking
- Visual Studio 2017, including:
- .NET Framework 3.5 development tools
- .NET Portable Library targeting pack
- Visual C++ runtime for UWP
- Visual Studio Tools for Unity (will be installed by Unity)
- Windows 10 SDK for UWP: C# and C++
- Unity 5.6.2f1 or greater (earlier versions may build the project, but contain bugs in some APIs!)
If the following steps don't work, please follow the complete instructions here.
- Open the Unity project in Unity (folder HoLola)
File->Build Settings...- Double-check that the PLatform is set to Windows Store and that at least one scene is listed under Scenes In Build
- Click the Build button. You will be asked to choose a directory to put the binaries in (e.g
.\Build\). - After the build completes, navigate to the build directory you selected and open
HoLola.slnin Visual Studio (*NOTE: This is not the sameHoLola.slnas you will find in the Unity project root directory!) - Set your CPU architecture to
x86, then set the target (next to the green arrow) to eitherDevice(if the Hololens is connected via USB) orHololens Emulator. Note: check out that the build type coincides with the one of the dlls,Debugby default. - Press
F5to deploy and launch the application.
If you do not have sufficient permissions on your build machine, or if you want to package the application for later redistribution or deployment, you can create an APPX package then side load it onto the device through the web interface.
Starting from step 5 in the Normal build/deploy steps above:
- After building Holola.sln, right-click the Holola project in the solution explorer and select
Store... - When asked if you want to submit the application to the Windows Store select No
- Follow the steps in the package creation wizard. If you're testing a lot of builds, it may be useful to check the auto-increment version box to easily distinguish between new and old builds.
- When asked about target platforms, the only one you need is
x86 - After the package is created, open a web browser and navigate to the Hololens' IP address (
http://192.168.0.100:10080over WiFi by default orhttp://127.0.0.1:10080if the device is connected via USB) - In the browser select
Appson the left - In the App Manager, under Install app use the
Choose Filebutton to select the.appxfile created after step 5 - Click
Add dependency, and add all of the.appxfiles under `/Dependencies/x86' - Click
Go - Your app should now be installed! Bloom to find it in the app list, or use the Insalled apps menu in the App Manager to start it.
When distributing the application, simply zip up the directory containing the .appx package. In addition to the application this will also include debugging symbols (*.pdb), the application's dependencies (under the Dependencies directory--note that you can safely delete all but the x86 dependencies if you want to save space), and some scripts which can make deploying the app from the commandline a little easier. The only files which are strictly necessary for the app to work on another device are the .appx and the files under Dependencies\x86.
https://developer.microsoft.com/en-us/windows/mixed-reality/using_the_hololens_emulator
NOTE: Recent builds are checked in under the HoLola Unity project, so this should only be necessary if you are making changes to either DLL.
NOTE: Do not mix & match Debug and Release builds! If you want to deploy a Release version of the final Unity project, these DLLs should also be built with the Release target! For the time being, the checked-in versions are Debug.
Both the native and managed DLLs can be built from LolaComms.sln under /net/LolaComms.
For deployment to Hololens or the emulator you must build x86, not x64.
Resulting DLLs should be automatically deployed to the Unity project:
- Managed DLLs need to be copied to
<Unity Project Root>/Assets/Plugins/ - Unmanaged (C++) DLLs need to be copied to
<Unity Project Root>/Assets/
Once the DLLs have been updated in the Unity project, build from Unity and follow the instructions above to deploy to the device or emulator.
You only need to rebuild from the final .sln. You do not have to re-export anything from Unity.
You need to rebuild from Unity, then rebuild the final .sln.
You need to rebuild LolaComms.sln, then rebuild from Unity, then rebuild the final .sln
Unity produces a log file containing all the output from Debug.Log* calls, callstacks if a Monobehavior hits an unhandled exception, etc. The log file is over-written every time the app launches, so make sure to download the logs after every test!
The log is located in: User Files \ LocalAppData \ HoLola \ TempState
ETW Trace data is available for some native components, which may help track down issues in the LolaComms DLLs. To enable them and get trace data, open the Logging section of the device portal, then under Custom Providers enter the GUID 8EB119A9-2FE5-46F5-998B-A396CA3F74B7 and click Enable. You should now see live event data at the bottom of the page when the application is active.
See net/LolaCommsNative/LolaCommsNative/LolaCommsNative.man for event definitions.
Note: For some reason, provider info and event payloads are not appearing in the device portal. This may be related to this issue, meaning a future update on the Hololens (and possibly a matching SDK update) could fix it, but it might also be that the configuration is not quite right on our end...
- Enable mixed-mode debugging in the project properties to examine behavior in LolaCommsNative
- To catch exceptions and breakpoints in both DLLs, ensure
Enable Just My Codeis NOT checked in your debugging Options.
Unfortunate side-effects: Doing both of the above will also cause you catch a lot of exceptions in Unity and other components you likely don't want to deal with. This can make debugging tedious since Unity often throws a bunch of exceptions during initialization, so the recommendation is to keep your debugging setup for managed-only and Just My Code unless you need to dig into the communications DLLs.
- Sometimes when deploying to the Hololens Emulator the app crashes on startup with a CLR error. Under a debugger this appears to be a fatal error in Windows.Mirage.dll, but there are also often bad references in Unty's D3D code (null textures, etc) which can lead to an early crash. Re-deploying without making any changes often fixes this.