Skip to content
This repository has been archived by the owner on Mar 22, 2022. It is now read-only.

Add video capture on Android #246

Closed
djee-ms opened this issue Mar 31, 2020 · 0 comments
Closed

Add video capture on Android #246

djee-ms opened this issue Mar 31, 2020 · 0 comments
Assignees
@djee-ms
Labels
bug Something isn't working platform:android Android specific issues
Milestone
2.0.0

Comments

@djee-ms
Copy link
Member

djee-ms commented Mar 31, 2020

This is a TODO task to add video capture on Android once #193 is merged to master.
@djee-ms djee-ms added the bug Something isn't working label Mar 31, 2020
@djee-ms djee-ms added this to the 2.0.0 milestone Mar 31, 2020
djee-ms added a commit that referenced this issue Mar 31, 2020
@djee-ms
Merge pull request #193 from AltspaceVR/eanders/android
9a50602 
Add experimental Android ARM64 support via Unity.

This changes adds support for compiling MixedReality-WebRTC for Android ARM64
for use inside a Unity project and deploy as a Unity build to an Android
device. Support for non-Unity Android apps is out of scope.

The Android build produces 2 archives:
- libwebrtc.aar: the core Google WebRTC implementation for Android ARM64.
- mrwebrtc.aar: the MixedReality-WebRTC API wrapping the Google implementation.

Both archives are copied to the `Assets/Plugins/arm64-v8a` folder of the Unity
sample project, and deployed to the device.

The current change builds over the main WebRTC Google repository instead of
WebRTC UWP, therefore represents a temporary diverging in the
MixedReality-WebRTC. For this reason, it is strongly recommended to use the
`branch-heads/71` Google branch to keep the code as close as possible to the
one used for other platforms (Windows Desktop and UWP). Building from Google's
`master` branch may work but is not supported.

This change is experimental. The `tools/build/libwebrtc` and
`tools/build/android` folders provides some utility scripts to help building
the Google repository for Android ARM64 and the MixedReality-WebRTC project,
respectively (see `tools/README.md`). However this process is involved, and
there is currently no precompiled binaries provided to avoid it, nor any CI in
place to validte changes. This will come in time.

Known limitations and issues:
- Video capture on Android device currently does not work. This requires
  specific interop code to open the device camera from Java, which is not yet
  available. (Bug: #246)
- The two Android archives are huge, much too big for production. Current
  testing shows a deploy at 326 MB on device, including Unity's project and
  without any optimization nor stripping (dev build). (Bug: #247)
- The build should produce a single archive, not two. (Bug: #248)
- Because Android and Windows use different Google repository commits, there is
  a small chance to get discrepancies in behavior and/or incompatibility bugs
  between those. No such bug has been observed so far, but please keep this in
  mind.

Android users are encouraged to try this change and provide feedback via GitHub
issues, be it on the build process itself, on the missing features not listed
above, or anything else related to Android support. The change has been tested
manually on Google Pixel 3A and Occulus Quest. Other Android devices should
work too, but have not been tested explicitly.

A huge thanks to Eric Anderson from AltspaceVR for contributing this change.

Bug: #28
@djee-ms djee-ms mentioned this issue Mar 31, 2020
Closed
22 tasks
@djee-ms djee-ms added the platform:android Android specific issues label Apr 1, 2020
@drejx drejx mentioned this issue May 22, 2020
Open
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue May 31, 2020
@djee-ms
Implement video capture for Android
5578eb1 
Add the missing binding to the native Java video capture on Android.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue May 31, 2020
@djee-ms
Implement video capture for Android
c697e65 
Add the missing binding to the native Java video capture on Android.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 2, 2020
@djee-ms
Implement video capture for Android
d3f5a4f 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 2, 2020
@djee-ms
Implement video capture for Android
b3317c1 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 2, 2020
@djee-ms
Implement video capture for Android
3d2f914 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 5, 2020
@djee-ms
Implement video capture for Android
40c25f1 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 22, 2020
@djee-ms
Implement video capture for Android
af7d74c 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
djee-ms added a commit to djee-ms/MixedReality-WebRTC that referenced this issue Jun 24, 2020
@djee-ms
Implement video capture for Android
5654066 
Add the missing binding to the native Java video capture on Android. The
binding is done with some Java class AndroidCameraInterop based on the
WebRTC project's UnityUtility.java. It enables interop between the Java
camera object and the C++ video track source object. This both avoids
having to build and merge UnityUtility.java, as well as allow extending
it with new functionality like video capture format selection.

Bug: microsoft#246
@djee-ms djee-ms mentioned this issue Jun 30, 2020
Merged
@djee-ms djee-ms self-assigned this Jul 2, 2020
@djee-ms djee-ms closed this as completed Jul 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@djee-ms djee-ms

Labels
bug Something isn't working platform:android Android specific issues
Projects
None yet
Milestone
2.0.0
Development

No branches or pull requests
1 participant
@djee-ms