An experimental Chromium modification in the form of an app for Android that lets developers build Augmented Reality (AR) experiences using web technologies on top of Google's ARCore.
Clone or download
Pull request Compare This branch is 138 commits ahead, 147550 commits behind chromium:master.
Iker Jamardo
Latest commit 2441c86 May 8, 2018
Permalink
Failed to load latest commit information.
android_webview Removed the supported device check to enable WebARonARCore to run on … Feb 28, 2018
apk Removed the supported device check to enable WebARonARCore to run on … Feb 28, 2018
apps Move Chrome terminating notification out of //apps Jan 13, 2017
ash chromeos: Split laser tests into two files. Jan 20, 2017
base add TEAM/COMPONENT information Jan 20, 2017
blink/tools Move run_webkit_tests.* script into their new home. Oct 10, 2014
breakpad Remove GYP files in breakpad, build, and tools. Sep 12, 2016
build Suppress warnings for the new android_webview resources. Jan 21, 2017
build_overrides Make sanitizer suppressions and blacklists fully configurable. Dec 19, 2016
cc Remove unused transform node index from ScrollTree::OnScrollOffsetAni… Jan 20, 2017
chrome Incrementing VERSION to 57.0.2987.5 Jan 20, 2017
chrome_elf Use loader-based binding of GetInstallDetailsPayload for install_static. Jan 18, 2017
chromecast [Chromecast] Dump the runtime deps for all test targets. Jan 19, 2017
chromeos [CUPS] Implement the enterprise icon for printers in Print Preview Di… Jan 20, 2017
cloud_print Re-enable building both 32 and 64 bit version of the GCP port monitor. Dec 20, 2016
components Revert of [Sync] Make directory types registration explicit in ModelT… Jan 20, 2017
content add TEAM/COMPONENT information Jan 20, 2017
courgette [Courgette] Simple AssemblyProgram and Disassembler cleanups. Jan 13, 2017
crypto Remove the password parameter for ECPrivateKey::ExportEncryptedPrivat… Jan 4, 2017
dbus Migrate desktop Linux to histogram_macros.h include. Jan 19, 2017
device Verified that the anchor calculations are correct. Found that the mai… Nov 29, 2017
docs Android: Update android studio docs Jan 19, 2017
extensions Remove usage of AsyncExtensionFunction::results_ in system_network_ap… Jan 20, 2017
gin [Gin] Change the Ignition experiment to test Ignition+TurboFan. Jan 19, 2017
google_apis Update ConnectionEventTracker to check if there is an in progress att… Jan 16, 2017
google_update Remove GYP files. Sep 14, 2016
gpu Removed some logs. Nov 9, 2017
headless [headless] Ensure devtooled tests call Disable/RemoveObserver. Jan 19, 2017
infra Up TSAN bot CQ experiment to 50% Jan 19, 2017
ios Adjust disabling of synchronization and add wait to test. Jan 20, 2017
ipc Add more mojo perf tests. Jan 14, 2017
jingle Add xmllite and xmpp sources to third_party/ Jan 4, 2017
mash mash: Do not use InProcessContextFactory in tests. Jan 20, 2017
media MSE: Fix moar mp4 parsing security bugs. Jan 20, 2017
mojo Make mojom-generated C++ string constants really constant. Jan 19, 2017
native_client_sdk [NaCl SDK] Add more support and fix FakePepperInterfaceGoogleDriveFs Dec 23, 2016
net Update cert_verify_tool to use new CertNetFetcher interface Jan 20, 2017
pdf Add hooks to PDFium PostScript code Jan 18, 2017
ppapi Autogenerated changes when executing the command buffer build python … Jan 21, 2017
printing [CUPS] Implement the enterprise icon for printers in Print Preview Di… Jan 20, 2017
remoting Cleanup media depndencies in the android remoting client Jan 20, 2017
rlz Convert RLZ to a buildflag header Dec 13, 2016
sandbox Record a crash key when crashing due to a Seccomp SIGSYS failure. Jan 18, 2017
sdch Remove GYP files from some smaller toplevel dirs. Aug 30, 2016
services Revert of Fix unused lambda captures in //device and //services/media… Jan 19, 2017
skia Allow multiple calls to SetDefaultSkiaFactory on Windows. Jan 19, 2017
sql [tracing] Remove MemoryAllocatorDump::process_memory_dump Dec 8, 2016
storage Remove ScopedVector from ContentBrowserClient. Jan 13, 2017
styleguide Update Python style guide to point out exceptions only apply to legac… Jan 19, 2017
testing Revert of Enable BrowserScheduler.RedirectHistoryService experiment o… Jan 20, 2017
third_party Removed the supported device check to enable WebARonARCore to run on … Feb 28, 2018
tools add TEAM/COMPONENT information Jan 20, 2017
ui Revert "Put the close button of a notification into focus chain." Jan 20, 2017
url Reduce the perf overhead of our dangling markup experiments. Jan 19, 2017
.clang-format Add more macros to .clang-format's MacroBlock{Begin,End} options. Jul 17, 2015
.git-blame-ignore-revs Add missing comment reflow changes to .git-blame-ignore-revs Oct 10, 2016
.gitattributes Force Python LF checkout in .gitattributes. Dec 12, 2016
.gitignore Add hitTest implementation to match WebARKit logic Aug 17, 2017
.gn Remove all //blimp code. Jan 18, 2017
AUTHORS Fix aggregated WebKit license Jan 17, 2017
BUILD.gn Remove all //blimp code. Jan 18, 2017
CODE_OF_CONDUCT.md [CoC] Update code-of-conduct Aug 26, 2016
CONTRIBUTING.md Changed the app label to 'WebARCore'. Added the correct license (apac… Aug 23, 2017
DEPS Publish DEPS for Chromium 57.0.2987.5 Jan 20, 2017
LICENSE Updates the license file. May 7, 2018
LICENSE.chromium Changed the app label to 'WebARCore'. Added the correct license (apac… Aug 23, 2017
LICENSE.chromium_os Fix third-party license information for a number of projects Jul 27, 2012
OWNERS Update OWNERS for //.gn to include //build/OWNERS Jan 17, 2017
PRESUBMIT.py Encourage folks to use user-select instead of -webkit-user-select Jan 19, 2017
PRESUBMIT_test.py Add presubmit check for Google support URL format. Jan 12, 2017
PRESUBMIT_test_mocks.py Presubmit: Skip third_party for fwd decl warning Dec 12, 2016
README.md Changed the order of some of the build commands to adapt to the lates… Apr 8, 2018
WATCHLISTS Added eugenebut@chromium.org to ios_web watchlist. Jan 20, 2017
advancedlogcat.sh The script that allows to clear the logcat and the console and execut… Feb 26, 2017
backupwebar.sh Added VRMarker to the backup script. Jul 24, 2017
build_install_run.sh Added a simple progressbar to show the page load progress (#8) Sep 15, 2017
codereview.settings Update CL description by adding CQ_EXTRA_TRYBOTS for Telemetry changes. Apr 27, 2015
rebase2tag.sh Added a new bash script to ease the process of rebasing to a new tag. Feb 8, 2017

README.md

WebARonARCore

An experimental browser for Android that lets developers create Augmented Reality (AR) experiences using web technologies. An iOS version is also available.

Spawn-at-Camera exampleSpawn-at-Surface example

Note: This is not an official Google product. Nor is it a fully-featured web browser. Nor are the enabling JavaScript APIs standards, or on the standardization path. WebARonARCore is only meant to enable developer experimentation.

Getting started

1. Install the ARCore APK Developer Preview 1

WebARonARCore is still built on top of the Android ARCore Developer Preview 1 APK and because of that, only a certain number of devices are supported officially: Google Pixel 1 and 2 (both standard and XL) and Samsung Galaxy S8. The check to only detect these devices at runtime has been removed but the behavior on other ARCore models (supported by newer ARCore versions) is undefined.

Install the ARCore APK, either directly from a device:

  • Visit this link from a web browser on your Android device to download and install the ARCore Developer Preview 1 APK.

...or by using ADB:

  • Download the ARCore Developer Preview 1 APK to your computer from here and install the APK on your device:
    • $ adb install -r path/to/arcore_preview.apk

2. Install the WebARonARCore APK

Directly from a device:

  • Visit this link from a web browser on your Android device to download and install the WebARonARCore APK.

...or by using ADB:

  • Download the WebARonARCore APK.
  • Install the APK to your device:
    • $ adb install -r /path/to/WebARonARCore.apk

Alternatively, the WebARonARCore APK can be built and installed from source.

3. Viewing examples

A list of example scenes compatible with WebARonARCore and WebARonARKit are available at developers.google.com.

4. Building your own scenes

To build AR web experiences that work with WebARonARCore (or WebARonARKit for iOS), we recommend three.ar.js, a helper library that works with the popular three.js WebGL framework. Three.ar.js provides common AR building blocks, such as a visible reticle that draws on top of real world surfaces, and example scenes.

5. Debugging

WebARonARCore uses WebViews, which is a similar debugging process to debugging Chrome for Android tabs. Check out the prereqs for your device at Get Started with Remote Debugging Android Devices, and learn more about Remote Debugging WebViews by opening chrome://inspect in the desktop browser while your device is connected via USB.

Building the WebARonARCore APK from source

WebARonARCore can optionally be built and installed from source. Instructions for cloning and building Chromium are available at chromium.org

Prerequisites:

  • Linux machine
  • GIT
  • Python

We recommend the following steps:

  1. Open a terminal window
  2. Install depot_tools. You can follow this tutorial or simply follow these 2 steps:
  • git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
  • export PATH=$PATH:/path/to/depot_tools
  1. Create a folder to contain chromium and move to it: $ mkdir ~/chromium && cd ~/chromium
  2. Checkout the Chromium repo: ~/chromium$ fetch --nohooks android. Note: This process may take a long time (an hour?)
  3. Enter the src folder: $ cd src.
  4. Download the dependencies: $ gclient sync. Note: This process may take some time too.
  5. Install the build dependencies: ~/chromium/src$ build/install-build-deps-android.sh
  6. Setup the environment: ~/chromium/src$ . build/android/envsetup.sh
  7. Add this git repo as a remote (we are going to call it 'github'): git remote add github https://github.com/google-ar/WebARonARCore.git
  8. Fetch the newly added remote: git fetch github
  9. Checkout the webarcore branch from the github remote: git checkout --track github/webarcore_57.0.2987.5
  10. Synchronize the dependencies with this command: ~/chromium/src$ gclient sync --disable-syntax-validation. Note: This process may take some time too.
  11. Create a folder where to make the final product compilation: ~/chromium/src$ mkdir -p out/build.
  12. Create and edit a new file out/build/args.gn. Copy and paste the following content in the args.gn file:
  target_os = "android"
  target_cpu = "arm64"
  is_debug = false
  is_component_build = true
  enable_webvr = true
  proprietary_codecs = false
  ffmpeg_branding = "Chromium"
  enable_nacl = false
  remove_webcore_debug_symbols = true
  1. Prepare to build: ~/chromium/src$ gn args out/build. Note: once the command is executed, the vi editor will show you the content of the args.gn file just edited a few steps before. Just exit by pressing ESC and typing colon and x.
Build, install and run

The line below not only compiles Chromium but also installs the final APK on to a connected device and runs it, so it is convenient that you to connect the device via USB before executing it. The project that will be built by default is the Chromium WebView project, the only one that has been modified to provide AR capabilities.

~/chromium/src$ ./build_install_run.sh

You can review the content of the script to see what it does (it is a fairly simple script) but if you would like to compile the final APK on your own you could do it by executing the following command:

~/chromium/src$ ninja -C out/build android_webview_apk

The final APK will be built in the folders ~/chromium/src/apk and ~/chromium/src/out/build/apks.

How WebARonARCore works

WebARonARCore is built of two essential technologies: ARCore and Chromium. We also extend the WebVR 1.1 API, which gives us much of what we need for augmented reality, with a few more essentials, such as motion tracking, rendering of the camera's video feed, and basic understanding of the real world. For details, see WebVR API extension for smartphone AR

Known issues

  • The current implementation of WebARonARCore is built on top of the Chromium WebView flavor. This has some implementation advantages but some performance and use disadvantages. We are working on making the implementation on a full version of Chromium.
  • The current implementation of WebARonARCore is built on top of the ARCore Developer Preview version 1. This may lead to uncertain behavior in some devices that were not supported by that ARCore version.
  • Pausing/resuming/switching away from the app causes screen to turn black. This is a consequence of having built the implementation on top of the WebView flavor of Chromium. A proper implementation on full Chromium or a rebase to a more recent Chromium WebView version (>57.0.2987.5) might solve this problem.
  • The Web Speech API is a standard web API for text to speech and speech to text conversion that is available in Chromium. As WebARonARCore is built on top of the WebView version of Chromium, does not provide this functionality by default. There is a solution though, using a polyfill we provide, but in order to use it, you need to either a) include the three.ar.js library before making any use of the Web Speech API or b) include the ARSpeechRecognition.js file also before making any reference to the Web Speech API. Only speech recognition is suspported, not speech synthesis for now.

Future work

  • Add more AR-related features.
  • Adapt the implementation to the WebXR spec proposal.
  • Implement the prototype on full Chromium (not on the WebView flavor) and to a newer tag version (>57.0.2987.5).
  • Improve the VRPassThroughCamera rendering pipeline either making it obscure for the developer or by using regular WebGL textures and shader samplers without having to use the external image texture extension.
  • Adapt the prototype to the latest ARCore.

License

Apache License Version 2.0 (see the LICENSE file inside this repo).