Skip to content
This repository has been archived by the owner on Mar 22, 2022. It is now read-only.

Android support #193

Merged
merged 45 commits into from
Mar 31, 2020
Merged

Android support #193

merged 45 commits into from
Mar 31, 2020

Conversation

eanders-ms
Copy link
Contributor

@eanders-ms eanders-ms commented Feb 19, 2020

Add support for Android platform

New folders below /tools/build contain per-platform scripts to build MR-WebRTC, and to build the core Chromium libwebrtc base library:

  • /tools/build/libwebrtc: Scripts to build the Chromium base library for a given platform.
  • /tools/build/android: Contains a Gradle+CMake configuration to build MR-WebRTC for Android/arm64-v8a platform.
  • In the future possibly add more, even all, platform builds here? iOS build is similar to Android.
  • Interested to hear what you think of this organization. Happy to change it.

Status

  • libwebrtc build works for Android, with placeholder codepaths for future platforms. Build artifact for Android is libwebrtc.aar. It is automatically copied to the Unity sample project upon build success.
  • Android build works, though README instructions are incomplete. Build artifact is Microsoft.MixedReality.WebRTC.Native.aar, and is also copied to the Unity sample project automatically.
  • In testing, the DLL successfully loads and initializes on Android (Quest).
  • In a two-client test between Desktop/Editor and Quest, Desktop crashes after receiving the SDP answer.

TODO

  • Finish adding Android build instructions to READMEs.
  • Fix: VideoChatDemo crashes on Oculus Quest unless Local Video and Scene Capture are disabled.
  • Revert: Unsure how to fix this one: The change list shows I changed some things in webrtc-uwp-sdk. I should revert those changes.

@eanders-ms
Copy link
Contributor Author

eanders-ms commented Feb 19, 2020 via email

@djee-ms djee-ms self-assigned this Feb 19, 2020
@djee-ms djee-ms added the enhancement New feature or request label Feb 19, 2020
@djee-ms djee-ms added this to the 2.0.0 milestone Feb 19, 2020
@djee-ms djee-ms linked an issue Feb 19, 2020 that may be closed by this pull request
Copy link
Member

@djee-ms djee-ms left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No major issue, looks all good overall, amazing work! 👍

libs/Microsoft.MixedReality.WebRTC.Native/src/pch.h Outdated Show resolved Hide resolved
projects/android/webrtc-native/src/main/cpp/jni_onload.cpp Outdated Show resolved Hide resolved
projects/libwebrtc/.gitignore Outdated Show resolved Hide resolved
projects/libwebrtc/ENV-UNIX.md Outdated Show resolved Hide resolved
projects/libwebrtc/build.sh Outdated Show resolved Hide resolved
projects/libwebrtc/lib.sh Outdated Show resolved Hide resolved
@eanders-ms eanders-ms changed the title [wip] Android support Android support Feb 19, 2020
@astaikos316
Copy link

@eanders-ms from what I can see from node-dss and logcat, the quest never sends an answer message and the dss polling stops after an offer from the windows standalone is sent. No errors stand out in logcat when attached to the quest.

@djee-ms
Copy link
Member

djee-ms commented Mar 9, 2020

My 2c (didn't try) : network/signaling issues aside, not sending an answer is generally a sign of the offer failing to apply, either because of unsupported options (e.g. unsupported video codec), or because the offer arrives at the wrong time or the answer tries to be created too soon, both of which when the peer connection is in an invalid state (must be in "idle" to transition to "have-remote-offer" when applying the offer, and then be able to transition to "have-local-answer" when creating the answer). Typically as an example the latter fails in C# when forgetting to await the SetRemoteDescriptionAsync() call, and calling CreateAnswer() too soon.

# Functions

#-----------------------------------------------------------------------------
function usage() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just spent 4 hours setting up an Ubuntu VM with Hyper-V (╯°□°)╯︵ ┻━┻ and resizing the virtual disk to accommodate the gigantic repository (currently at 37GB total partition size) (╯°□°)╯︵ ┻━┻, to be told by the Google setup script that:

ERROR: The only supported distros are
 	Ubuntu 14.04 LTS (trusty)
 	Ubuntu 16.04 LTS (xenial)
 	Ubuntu 17.10 (artful)
 	Ubuntu 18.04 LTS (bionic)
 	Debian 8 (jessie) or later

Yes, I used Ubuntu 19.10 as suggested by default by Hyper-V. 😡

So I really think we should test ASAP in those scripts, preferably during config.sh:

  1. The user runs a valid distribution (see list above)
  2. The user has at least 40GB of disk space available

The actual check is in webrtc/build/install-build-deps.sh which is outdated on branch-heads/71 compared to recent Ubuntu releases (as linked on master it actually supports eoan 19.10). Ideally we'd grab that file and extract the supported distros from it, but it might be a bit complex to reach it via the chain of dependency (git branch => sub-repo chromium/build => script => grep on supported_codenames).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is terrible! Yes, we need to verify platform earlier.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed some basic checks as discussed (see d425548). Can you please have a look @eanders-ms? My bash is quite rusty... 😉

* Android Studio (https://developer.android.com/studio/index.html). Once installed, use the System Settings/Android SDK dialog to install the following packages:
* SDKs 9.0 (Pie), 8.1 (Oreo)
* CMake
* NDK 16.1.4479499
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no option to select a version of the NDK as I know. I can only see "NDK (side by side)", which as of today resolves to 21.0.6113669. I think that's OK but I can't remember the details about NDK versions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually that's not even enough, I had to uncheck "Hide obsolete packages", then install "NDK (Obsolete)" version 21.0.6113669, then click "Sync Project with Gradle Files" before I was finally allowed to Build > Make Project.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is light on detail, due entirely to the fact that I wasn't sure what the best practice is here. I wanted to provide steps that could be done entirely on the command line and not rely on Android Studio. I haven't gone back and worked out what those steps are yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can polish later. I want to commit to master an experimental support version, then we refine the instructions.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to build in Android Studio and getting this build error. any suggestions?

Build command failed.
Error while executing process /home/astaikos/Android/Sdk/cmake/3.10.2.4988404/bin/cmake with arguments {--build /home/astaikos/eanders/MixedReality-WebRTC/tools/build/android/webrtc-native/.externalNativeBuild/cmake/debug/arm64-v8a --target mrwebrtc}
[1/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/local_video_track_interop.cpp.o
[2/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/data_channel.cpp.o
[3/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/str.cpp.o
[4/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp.o
[5/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/mrs_errors.cpp.o
[6/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/audio_frame_observer.cpp.o
[7/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/external_video_track_source_interop.cpp.o
[8/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/peer_connection_interop.cpp.o
[9/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/pch.cpp.o
[10/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp.o
[11/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/global_factory.cpp.o
[12/17] Building CXX object CMakeFiles/mrwebrtc.dir/jni_onload.cpp.o
[13/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/sdp_utils.cpp.o
[14/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/interop_api.cpp.o
[15/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp.o
[16/17] Building CXX object CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/video_frame_observer.cpp.o
[17/17] Linking CXX shared library /home/astaikos/eanders/MixedReality-WebRTC/tools/build/android/webrtc-native/build/intermediates/cmake/debug/obj/arm64-v8a/libmrwebrtc.so
FAILED: /home/astaikos/eanders/MixedReality-WebRTC/tools/build/android/webrtc-native/build/intermediates/cmake/debug/obj/arm64-v8a/libmrwebrtc.so
: && /home/astaikos/Android/Sdk/ndk/21.0.6113669/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=aarch64-none-linux-android24 --gcc-toolchain=/home/astaikos/Android/Sdk/ndk/21.0.6113669/toolchains/llvm/prebuilt/linux-x86_64 --sysroot=/home/astaikos/Android/Sdk/ndk/21.0.6113669/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -std=c++17 -DANDROID_STL=c++_shared -DMRS_USE_STR_WRAPPER -DMR_SHARING_ANDROID -DWEBRTC_POSIX -DWEBRTC_ANDROID -O0 -fno-limit-debug-info -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -shared -Wl,-soname,libmrwebrtc.so -o /home/astaikos/eanders/MixedReality-WebRTC/tools/build/android/webrtc-native/build/intermediates/cmake/debug/obj/arm64-v8a/libmrwebrtc.so CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/external_video_track_source_interop.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/global_factory.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/local_video_track_interop.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/peer_connection_interop.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/interop/interop_api.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/audio_frame_observer.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/data_channel.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/mrs_errors.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/pch.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/sdp_utils.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/str.cpp.o CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/video_frame_observer.cpp.o CMakeFiles/mrwebrtc.dir/jni_onload.cpp.o -llog -landroid -ldl -lOpenSLES /home/astaikos/eanders/build_webrtc/webrtc/src/out/android/arm64/Release/obj/libwebrtc.a -latomic -lm && :
CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp.o: In function PeerConnection': /home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp:1370: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::TrackedObject(Microsoft::MixedReality::WebRTC::RefPtrMicrosoft::MixedReality::WebRTC::GlobalFactory, Microsoft::MixedReality::WebRTC::ObjectType)'
CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp.o: In function ~PeerConnection': /home/astaikos/eanders/MixedReality-WebRTC/tools/build/android/webrtc-native/src/main/cpp/../../../../../../../libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.h:65: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::~TrackedObject()'
CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/peer_connection.cpp.o:(.data.rel.ro._ZTIN9Microsoft12MixedReality6WebRTC14PeerConnectionE[_ZTIN9Microsoft12MixedReality6WebRTC14PeerConnectionE]+0x10): undefined reference to typeinfo for Microsoft::MixedReality::WebRTC::TrackedObject' CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp.o: In function ExternalVideoTrackSource':
/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp:287: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::TrackedObject(Microsoft::MixedReality::WebRTC::RefPtr<Microsoft::MixedReality::WebRTC::GlobalFactory>, Microsoft::MixedReality::WebRTC::ObjectType)' CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp.o: In function ~ExternalVideoTrackSource':
/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.h:79: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::~TrackedObject()' CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/external_video_track_source.cpp.o:(.data.rel.ro._ZTIN9Microsoft12MixedReality6WebRTC24ExternalVideoTrackSourceE[_ZTIN9Microsoft12MixedReality6WebRTC24ExternalVideoTrackSourceE]+0x10): undefined reference to typeinfo for Microsoft::MixedReality::WebRTC::TrackedObject'
CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp.o: In function LocalVideoTrack': /home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp:18: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::TrackedObject(Microsoft::MixedReality::WebRTC::RefPtrMicrosoft::MixedReality::WebRTC::GlobalFactory, Microsoft::MixedReality::WebRTC::ObjectType)'
/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp:27: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::~TrackedObject()' CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp.o: In function ~LocalVideoTrack':
/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp:35: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::~TrackedObject()' /home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp:35: undefined reference to Microsoft::MixedReality::WebRTC::TrackedObject::~TrackedObject()'
CMakeFiles/mrwebrtc.dir/home/astaikos/eanders/MixedReality-WebRTC/libs/Microsoft.MixedReality.WebRTC.Native/src/media/local_video_track.cpp.o:(.data.rel.ro._ZTIN9Microsoft12MixedReality6WebRTC15LocalVideoTrackE+0x28): undefined reference to `typeinfo for Microsoft::MixedReality::WebRTC::TrackedObject'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fixed that like 5 min ago, this was due to the merge of master, you're too fast... 😉

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also attempting to build for android, via android studio. but I'm coming across the following error:
In file included from C:\MixedRealityRTCLib\MixedReality-WebRTC\libs\Microsoft.MixedReality.WebRTC.Native\src\media\external_video_track_source.cpp:4:
C:\MixedRealityRTCLib\MixedReality-WebRTC\libs\Microsoft.MixedReality.WebRTC.Native\src\pch.h:44:10: fatal error: 'api/audio_codecs/builtin_audio_decoder_factory.h' file not found

is there some include directive that is missing? I've pulled from the 'master' branch.

* NDK 16.1.4479499
* JRE 1.8 (https://www.java.com/download/)

3. Add environment variable JAVA_HOME pointing to your JRE install directory. e.g.: `JAVA_HOME="C:\Program Files\Android\Android Studio\jre"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Linux that's apt install openjdk-8-jre and I think already installed by the Google dependency script. Shall we mention it for completeness?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noted later that this is also needed on Unity, although Unity will strongly suggest using the version shipped with the Unity Editor. Maybe we should recommend that too, or at least point it out.

Due to a bug in P/Invoke on Windows, which prevents loading DLLs by name
without extension when the filename contains some dot '.' character, and
in order to keep the Microsoft.MixedReality.WebRTC assembly fully
cross-platform, this change renames the native implementation DLL from
Microsoft.MixedReality.WebRTC.Native.dll into mrwebrtc.dll. This has the
added benefit to make it clear that this is not anymore the C++ library,
but an opaque internal implementation. The C++ library will, like the C#
one, build on top of that implementation DLL.
It seems the explicit reference was not needed before, but at least in
Unity 2019.3 the test assemblies will fail to build without this
reference.
Check during config.sh that the host machine pass some basic
requirements when configuring an Android build, to avoid starting a
lengthy task that cannot possibly complete:
- Host needs to run a particular Linux distro supported by the Google
  scripts. This can be ignored with the -u option.
- Host needs at least 40GB of disk space to build everything needed.
  This can be ignored with the -s option.
@djee-ms djee-ms merged commit 9a50602 into microsoft:master Mar 31, 2020
fibann added a commit that referenced this pull request Apr 14, 2020
No longer necessary. See #193 (comment)
for details.
@eanders-ms eanders-ms deleted the eanders/android branch April 14, 2020 21:24
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants