Is it possible to stream the Hololens content on my Mobile?

[bharath09vadde]

Hello,
I would like to know is it possible to stream what Hololens visualizes on my mobile? It may be a kind of spectator view or remote streaming.
Thanks in advance.

[djee-ms]

Yes, provided that what you want to see is only the content of your application inside HoloLens. For example, if you create a fullscreen exclusive app with Unity, you can stream its content with this library. The Unity sample VideoChatDemo does that thanks to the Mixed Reality Capture (MRC) feature.

Conversely however, you cannot stream an arbitrary HoloLens content showing the Pins Panel and all the slates from all open applications using a custom application you wrote; this requires some special UWP restricted capability that is almost impossible to be granted by Microsoft to third-party applications, for security and privacy reasons. See the screenDuplication restricted capabilities here for more details:
https://docs.microsoft.com/en-us/windows/uwp/packaging/app-capability-declarations#restricted-capabilities

The screenDuplication restricted capability allows apps to project the screen on another device.

This capability is required to use APIs in the DirectX namespace.

We don't recommend that you declare this capability in applications that you submit to the Microsoft Store. In most cases, the use of this capability won't be approved.

[bharath09vadde]

Hello,
I have opened the Microsoft.MixedReality.WebRTC.sln Visual Studio solution located at the root of the repository, When I opened the Unity project at libs\Microsoft.MixedReality.WebRTC.Unity and pressed play, I found these errors
WebRTC plugin initializing failed. See full log for exception details.
AggregationException: One or more errors occurred.
Exception: Microsoft.MixedReality.WebRTC.Native.dll
UnityEngine.Debug:LogError(Object)
Microsoft.MixedReality.WebRTC.Unity.PeerConnection:OnError_Listener(String) (at Assets/Microsoft.MixedReality.WebRTC.Unity/Scripts/PeerConnection.cs:552)
UnityEngine.Events.UnityEvent`1:Invoke(String)
Microsoft.MixedReality.WebRTC.Unity.<>c__DisplayClass23_1:b__1() (at Assets/Microsoft.MixedReality.WebRTC.Unity/Scripts/PeerConnection.cs:457)
Microsoft.MixedReality.WebRTC.Unity.PeerConnection:Update() (at Assets/Microsoft.MixedReality.WebRTC.Unity/Scripts/PeerConnection.cs:361)

And also when I tried copying the libraries there is an issue saying "Invalid parameter number". Can you help me solve this problem?

Thanks

[djee-ms]

Please see the Unity tutorial, and in particular the part about configuring the import settings for Unity. And check with git status that there is no change to the checked in .meta files; if Unity has deleted or changed them automatically, the changes must be reverted. If that doesn't work we can try fixing it after that.

[bharath09vadde]

But I opened the Unity project at "libs\Microsoft.MixedReality.WebRTC.Unity" nad pressed play, which according to the tutorial should display "WebRTC library being initialized successfully". This did not happen to me, the errors that I have attached are the errors shown while initializing the WebRTC. Can you tell me if I am missing something from the errors shown?

[bharath09vadde]

Which configuration should I set while building the "Microsoft.MixedReality.WebRTC.sln"?

[djee-ms]

You need to use x64 for being able to Play in the Unity editor. Debug or Release, as you prefer.

[bharath09vadde]

I am extremely sorry to bother you, I wanted to know how can I get the Peer ID of my Hololens device?

[djee-ms]

The peer ID is a concept that node-dss introduce. The IDs only need to be unique strings so that the node-dss server can recognize the 2 peers. You can chose any unique string you like, as long as the 2 peers have different ones and that they know each other's one.

[bharath09vadde]

I have used a unique peer ID for my Hololens and for the PC the ID that I got from the node-dss. When I deployed the application to Hololens it is showing "Cannot connect to destination Host". I have connected both the devices to the same Wifi network.

[djee-ms]

If it says "Cannot connect to destination Host" then it's a networking issue, not an issue with the peer IDs. Check your firewalls on both devices, sometimes they are a bit too agressive and block NodeJS or the app.

[djee-ms]

Any news @bharath09vadde? Did you manage to get your app working?

[bharath09vadde]

@djee-ms Thanks for asking. Actually no, And I couldn't figure out where did it go wrong. But the connection can not be established.

[bharath09vadde]

Network error trying to send data to http://192.xxx.xxx.xxx:3000/: Cannot connect to destination host
UnityEngine.Debug:LogError(Object)
Microsoft.MixedReality.WebRTC.Unity.<CO_GetAndProcessFromServer>d__15:MoveNext() (at Assets/Microsoft.MixedReality.WebRTC.Unity/Scripts/Signaling/NodeDssSignaler.cs:259)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

If I can solve this, then I think I will be done.

[cwoo3511]

Hi,
The Unity video demo (I followed the Unity tutorial) did not work when I switched the Unity platform to Android.
I want to implement video chat through Ar Foundation in the Unity mobile app.
Is it possible?
thank

[djee-ms]

Hi @cwoo3511,
Android is not currently supported. See #28 for details.

[zhuangce]

@djee-ms it's exciting to know there will be Support for Android deployment in Unity soon ,right now im confused about how to stream what Hololens visualizes on a android app ?

[djee-ms]

Yes for now it's not easy, you need to go back to Google's core implementation and roll your own Android peer client. If you really can't wait, hopefully that should be compatible with MixedReality-WebRTC since we're using Google's implementation too. For best chances I'd try for the m71 branch, which is the one we use currently. But this is a fair amount of work, so I'd suggest if you can wait until we support it that would make things much easier for you.

[djee-ms]

I believe everything was answered, so closing this question. See #28 for Android support via Unity.