Plugin doesn't work on HoloLens #502

argallegos · 2021-06-25T21:19:18Z

Possibly related to the android build issues (#190 #472 ), a user on the forum reported issues packaging for HoloLens. Another forum user reported similar issues with launching a packaged project, indicating that the problem arose after adding Cesium World Terrain.

I don't think Hololens has been extensively tested with the plugin, but as far as I know it should work in general. Please add any relevant logs or errors to this issue.

drakelzh · 2021-06-28T06:13:10Z

According to my issue in https://community.cesium.com/t/does-this-plugin-support-hololens-platform-packaging/14095/3, I post my packaging record here. Although there is no error, there are some warnings.

Package.txt

getinthedamnbox · 2021-06-30T16:37:28Z

One more bug report regarding the HoloLens: https://community.cesium.com/t/no-terrain-shown-on-hololens-2/14250

I'm going to try to run a simple Cesium for Unreal level on the HoloLens 2, and I'll post my findings in this issue.

getinthedamnbox · 2021-07-01T20:23:19Z

Documenting what I've tried and what the results were, even though I haven't been able to figure out the problem:

There are at least four ways of testing HoloLens 2 applications:

Install package to HoloLens 2 [link]
Install package to HoloLens 2 Emulator [link]
Deploy to device from Unreal editor [link]
Use Holographic Remoting Player [link]

I was able to run the application with 1 and 2. More on that in a moment.

For 3, I get this error in Unreal:

LogPlayLevel: Warning: WARNING: Project will be packaged to support deployment to remote HoloLens device http://169.254.220.62/.
LogPlayLevel: Error: ERROR: Device connection failed: Requesting operating system information,

I know Unreal can talk to the device, because I can turn it off or restart it through Unreal. It fails to deploy the application, though.

For 4, I can't launch Holographic Remoting. I get this error:

Initialization Failure
Failed to initialize. Please make sure that Windows Holographic is installed on your system. Windows Holographic will be installed automatically when you attach your Head-mounted Display.

I haven't been able to figure out why Windows thinks Windows Holographic isn't installed. I'll start by trying to update Windows to get version 21H1, but I don't know how long that update will take, so I'm not doing it right now.

Back to 1 and 2 (testing on the HoloLens and Emulator):

Just as forum users described, I can see non-Cesium Actors in the level just fine, but I can't see any tilesets. I know from printing messages to the screen that the constructors and BeginPlay() functions of ACesium3DTileset and ACesiumGeoreference aren't being called on the HoloLens or the Emulator.

The process of retrieving logs can be mildly finicky, but it eventually works. Here are the errors I got on the HoloLens:

[2021.07.01-19.29.00:238][  0]LogStreaming: Error: Couldn't find file for package /Script/CesiumRuntime requested by async loading code. NameToLoad: /Script/CesiumRuntime
[2021.07.01-19.29.00:238][  0]LogStreaming: Error: Found 1 dependent packages...
[2021.07.01-19.29.00:238][  0]LogStreaming: Error:   /Game/Main
[2021.07.01-19.29.00:238][  0]LogPackageName: Error: DoesPackageExist: DoesPackageExist FAILED: '/CesiumForUnreal/CesiumCreditSystemBP' is not a standard unreal filename or a long path name. Reason: Path does not start with a valid root. Path must begin with: '/Engine/', '/Game/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/WindowsMixedReality/', '/Takes/', '/Paper2D/', '/AnimationSharing/', '/GeometryMode/', '/SpeedTreeImporter/', '/DatasmithContent/', '/ChaosClothEditor/', '/GeometryProcessing/', '/GeometryCollectionPlugin/', '/ChaosSolverPlugin/', '/ChaosNiagara/', '/Niagara/', '/MotoSynth/', '/MediaCompositing/', '/Synthesis/', '/AudioSynesthesia/', '/Config/', '/Script/', '/Extra/', '/Memory/', or '/Temp/'
[2021.07.01-19.29.00:238][  0]LogStreaming: Error: Couldn't find file for package /CesiumForUnreal/CesiumCreditSystemBP requested by async loading code. NameToLoad: /CesiumForUnreal/CesiumCreditSystemBP
[2021.07.01-19.29.00:238][  0]LogStreaming: Error: Found 1 dependent packages...
[2021.07.01-19.29.00:238][  0]LogStreaming: Error:   /Game/Main
[2021.07.01-19.29.00:248][  0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:248][  0]LogStreaming: Error:     Export 6 /Game/Main.Main:PersistentLevel.Cube
[2021.07.01-19.29.00:248][  0]LogStreaming: Error:     Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:248][  0]LogStreaming: Error:         Dep C_BEFORE_S Export     8    /Game/Main.Main:PersistentLevel.CesiumGeoreferenceDefault     (class CesiumGeoreference)
[2021.07.01-19.29.00:248][  0]LogStreaming: Error:         Dep C_BEFORE_S Export     7    /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault     (class CesiumCreditSystemBP_C)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_S Export     6    /Game/Main.Main:PersistentLevel.Cube.TileSet     (class Cesium3DTilesetRoot)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import    12   /Script/CesiumRuntime.Cesium3DTileset
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import     8   /Script/CesiumRuntime.Default__Cesium3DTileset
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import     9   /Script/CesiumRuntime.Default__Cesium3DTileset:TileSet
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_C Export    14    /Game/Main.Main:PersistentLevel     (class Level)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.Cesium3DTileset but it hasn't been created yet.
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Could not find class Cesium3DTileset to create Cube
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Export 7 /Game/Main.Main:PersistentLevel.Cube.TileSet
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_S Import     8   /Script/CesiumRuntime.Default__Cesium3DTileset
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import    13   /Script/CesiumRuntime.Cesium3DTilesetRoot
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import     9   /Script/CesiumRuntime.Default__Cesium3DTileset:TileSet
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_C Export     5    /Game/Main.Main:PersistentLevel.Cube     (class Cesium3DTileset)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.Cesium3DTilesetRoot but it hasn't been created yet.
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Could not find class Cesium3DTilesetRoot to create TileSet
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Export 8 /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_S Export    20    /Game/Main.Main:PersistentLevel.CesiumCreditSystemDefault.DefaultSceneRoot     (class SceneComponent)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import     2   /CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import    10   /CesiumForUnreal/CesiumCreditSystemBP.Default__CesiumCreditSystemBP_C
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_C Export    14    /Game/Main.Main:PersistentLevel     (class Level)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Missing Dependency, request for /CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C but it hasn't been created yet.
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Could not find class CesiumCreditSystemBP_C to create CesiumCreditSystemDefault
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: ****DumpDependencies [Dependencies]:
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Export 9 /Game/Main.Main:PersistentLevel.CesiumGeoreferenceDefault
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:     Linker is ../../../CesiumHoloLens/Content/Main.umap
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import    14   /Script/CesiumRuntime.CesiumGeoreference
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep S_BEFORE_C Import    11   /Script/CesiumRuntime.Default__CesiumGeoreference
[2021.07.01-19.29.00:253][  0]LogStreaming: Error:         Dep C_BEFORE_C Export    14    /Game/Main.Main:PersistentLevel     (class Level)
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Missing Dependency, request for /Script/CesiumRuntime.CesiumGeoreference but it hasn't been created yet.
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Could not find class CesiumGeoreference to create CesiumGeoreferenceDefault
[2021.07.01-19.29.00:253][  0]LogStreaming: Error: Could not find outer CesiumCreditSystemDefault to create DefaultSceneRoot

Also worth noting that when packaging the application for HoloLens, I get the following warnings (similar to the log posted here):

UATHelper: Packaging (HoloLens):   LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C, but it was never saved as an export.
UATHelper: Packaging (HoloLens):   LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.Default__CesiumCreditSystemBP_C, but it was never saved as an export.
UATHelper: Packaging (HoloLens):   LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP, but it was never saved as an export.
UATHelper: Packaging (HoloLens):   LogInit: Display: LogSavePackage: Warning: /Game/Main imported Serialize:/CesiumForUnreal/CesiumCreditSystemBP.CesiumCreditSystemBP_C:DefaultSceneRoot_GEN_VARIABLE, but it was never saved as an export.

I didn't attempt to change cesium-unreal/Source/CesiumRuntime/CesiumRuntime.Build.cs or cesium-unreal/extern/CMakeLists. That might be the obvious problem (HoloLens has an ARM architecture), but I would need help understanding how to fix it.

If anyone has a better grasp of the errors and warnings in the logs above, let me know.

kring · 2021-07-02T00:39:41Z

Thanks for the writeup @getinthedamnbox. I suspect if you scroll up further in the log, you'll see the Cesium for Unreal plugin isn't being loaded at all. Can you confirm that? I'm not really sure why that would be, though. You said the HoloLens is ARM. It's 64-bit ARM, right? Is it Android? Does it use the same toolchain as regular Android builds?

getinthedamnbox · 2021-07-06T14:42:05Z

I suspect if you scroll up further in the log, you'll see the Cesium for Unreal plugin isn't being loaded at all. Can you confirm that?

You're right, I don't see it listed as one of the mounted plugins. Here's the full log: CesiumHoloLens-backup-2021.07.01-19.29.14.log

And here's the log from packaging the application for HoloLens: CesiumHoloLens.log

You said the HoloLens is ARM. It's 64-bit ARM, right? Is it Android? Does it use the same toolchain as regular Android builds?

It's 64-bit ARM, but to be honest I don't know enough about toolchains to answer your other questions. What I know is that in order to package and deploy the application, I had to ensure Visual Studio Build Tools had the component MSVC v142 - VS 2019 C++ ARM64 build tools (v14.26) installed.

kring · 2021-07-06T14:45:02Z

Thanks @getinthedamnbox. Yeah that definitely doesn't sound like "normal" Android, at least. So we probably need a special build of cesium-native.

argallegos · 2021-07-14T20:47:00Z

@kring Alban had some insights:

If you plan to run on the device, you need to have an ARM64 build of Cesium.

When you run the application using holoremoting, it basically runs on your computer, and streams the 3d render in realtime like a video. On the other hand, the HL2 sends its interactions to the device, and you close the loop...
In that mode (Tethered or not - I prefer to use Wifi), the app has to be packaged in Win64, so it's very easy to do so using the current Cesium plugin. (I did an app for GTC showcasing that).

But it you plan to deploy and run it "on" the device (meaning without any computer to generate the 3D content), your app needs to be packaged for Hololens2, that expects an ARM64 build. It's fine with most UE libraries, but any included external lib needs an ARM64 version.

I had to do that to port the GeoReferencing plugin I wrote for 4.27. It relies on PROJ, so I had to build PROJ for ARM64 to make it run. Fortunately, it was not that hard when using vcpkg toolchain.

Then, as usual, you need to edit your 3rd party libs build.cs to point to the right libs, with a distinction on architecture. See this sample file for the proj.build.cs

else if (Target.Platform == UnrealTargetPlatform.HoloLens)
		{
			if (Target.WindowsPlatform.Architecture == WindowsArchitecture.x64) // emulation target, bBuildForEmulation
			{
				PublicSystemIncludePaths.Add(Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-x64-uwp", "include"));

				string LibPath = Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-x64-uwp", "lib");
				PublicAdditionalLibraries.Add(Path.Combine(LibPath, "proj.lib"));
				PublicAdditionalLibraries.Add(Path.Combine(LibPath, "sqlite3.lib"));
			}
			else if (Target.WindowsPlatform.Architecture == WindowsArchitecture.ARM64) // device target, bBuildForDevice
			{
				PublicSystemIncludePaths.Add(Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-arm64-uwp", "include"));

				string LibPath = Path.Combine(ModuleDirectory, VcPkgInstalled, "overlay-arm64-uwp", "lib");
				PublicAdditionalLibraries.Add(Path.Combine(LibPath, "proj.lib"));
				PublicAdditionalLibraries.Add(Path.Combine(LibPath, "sqlite3.lib"));
			}
			else
			{
				throw new System.Exception("Unknown architecture for HoloLens platform!");
			}
		}

Don't be afraid by the "uwp". This is specific to PROJ to allow it to access its data file on a sandboxed device.
Proj.Build.cs

// Copyright Epic Games, Inc. All Rights Reserved.

using System.IO;
using UnrealBuildTool;

If you want to pick up some code, it's on UE Github there : https://github.com/EpicGames/UnrealEngine/tree/4.27/Engine/Plugins/Runtime/GeoReferencing/Source/GeoReferencing
Having a build of cesium for ARM64 might be complicated, because you are using lots of external libs.
But once done, it will be great. I hope the HL2 GPU will be able to deal with this geometry, without raising too much the screen space error value.

ahsan4313 · 2023-02-05T17:25:37Z

any update regarding the issue

kring · 2023-02-06T03:11:44Z

No update. If someone wants to attempt this, follow the Developer Setup instructions, adjusting them to build cesium-native for the HoloLens platform, probably by defining a cmake toolchain. It is not currently on our roadmap, but we would welcome a pull request from the community.

ahsan4313 · 2023-02-06T04:09:38Z

Sure will try it.
Will this work remotely without packaging?

ahsan4313 · 2023-02-08T16:56:56Z

After running camke files for Cesium native. what steps i have to take for Hololens 2 plugin. please update me

ahsan4313 · 2023-02-08T20:05:42Z

Got this html file running after following the process for cesium native. Now what to do?

kring · 2023-02-09T03:43:00Z

If we knew the exact steps, it would already be done. I'm afraid I don't know, and you'll have to figure it out, or wait for someone else to figure it out. From what I can see in your VS Code window, you're just building cesium-native for Windows, so that's not going to help. You need to build it for HoloLens by configuring a toolchain appropriately, but I don't know any more details than that.

I'm not sure what that Eigen3 page is. It's not part of cesium-native, but maybe one of our third-party libraries uses it indirectly.

ahsan4313 · 2023-02-09T04:37:17Z

Great. Just want to confirm is this for desktop hologram or android hologram?

Although i think its for hologram (doesn't matter platform) but want to clear ambiguity of my senior.

ahsan4313 · 2023-02-09T04:38:41Z

Can I access cesium for unreal plugin while connecting HoloLens remotely?

kring · 2023-02-09T05:06:51Z

Yes, Cesium for Unreal should already working with HoloLens remoting.

ahsan4313 · 2023-02-09T06:54:13Z

Great

kring added the bug Something isn't working label Jun 29, 2021

getinthedamnbox changed the title ~~Hololens build failure~~ Plugin doesn't work on HoloLens Jun 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Plugin doesn't work on HoloLens #502

Plugin doesn't work on HoloLens #502

argallegos commented Jun 25, 2021 •

edited

Loading

drakelzh commented Jun 28, 2021 •

edited

Loading

getinthedamnbox commented Jun 30, 2021

getinthedamnbox commented Jul 1, 2021

kring commented Jul 2, 2021

getinthedamnbox commented Jul 6, 2021

kring commented Jul 6, 2021

argallegos commented Jul 14, 2021

ahsan4313 commented Feb 5, 2023

kring commented Feb 6, 2023

ahsan4313 commented Feb 6, 2023 •

edited

Loading

ahsan4313 commented Feb 8, 2023

ahsan4313 commented Feb 8, 2023

kring commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

kring commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

Plugin doesn't work on HoloLens #502

Plugin doesn't work on HoloLens #502

Comments

argallegos commented Jun 25, 2021 • edited Loading

drakelzh commented Jun 28, 2021 • edited Loading

getinthedamnbox commented Jun 30, 2021

getinthedamnbox commented Jul 1, 2021

kring commented Jul 2, 2021

getinthedamnbox commented Jul 6, 2021

kring commented Jul 6, 2021

argallegos commented Jul 14, 2021

ahsan4313 commented Feb 5, 2023

kring commented Feb 6, 2023

ahsan4313 commented Feb 6, 2023 • edited Loading

ahsan4313 commented Feb 8, 2023

ahsan4313 commented Feb 8, 2023

kring commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

kring commented Feb 9, 2023

ahsan4313 commented Feb 9, 2023

argallegos commented Jun 25, 2021 •

edited

Loading

drakelzh commented Jun 28, 2021 •

edited

Loading

ahsan4313 commented Feb 6, 2023 •

edited

Loading