ARCore SDK for Unreal v1.7.0

Engine and plugin changes

ARCore Unreal SDK v1.7.0 is based on Unreal 4.21.2 release. The Unreal Engine branch with the 1.7.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.21-arcore

New APIs

• New UGoogleARCoreFrameFunctionLibrary::GetTrackingFailureReason C++/Bluerpint function that returns the reason for AR tracking failure when tracking state/quality is NotTracking.
• New UGoogleARCoreFrameFunctionLibrary::TransformARCoordinates2D C++/Bluerpint function that transforms a list of 2D coordinates from one 2D coordinate system to another 2D coordinate system.
• New EGoogleARCoreCameraFacing CameraFacing and EGoogleARCoreAugmentedFaceMode AugmentedFaceMode properties in UGoogleARCoreSessionConfig enable new capabilities:
• Front Camera & Augmented Faces
  • Apps can now enable Augmented Faces with the front-facing (selfie) camera by setting the CameraFacing and AugmentedFaceMode properties in UGoogleARCoreSessionConfig.
  • EARSessionType::Face session type in UARSessionConfig is also supported when using ARCore. When running on ARCore supported devices, it will configure the session to use front camera with AugmentedFaceMode set to EGoogleARCoreAugmentedFaceMode::PoseAndMesh.
  • New UGoogleARCoreAugmentedFace type which inherents from UARFaceGeometry class.
  • New UGoogleARCoreSessionFunctionLibrary::GetAllAugmentedFaces and UGoogleARCoreFrameFunctionLibrary::GetUpdatedAugmentedFaces C++/Blueprint functions which can be used to get the UGoogleARCoreAugmentedFace objects that ARCore manages.
  • New UGoogleARCoreFaceMeshComponent class, which is a C++/Blueprint component that creating face mesh for rendering purpose.

Deprecations

• UGoogleARCoreSessionFunctionLibrary::GetPassthroughCameraImageUV is now deprecated. Please use UGoogleARCoreFrameFunctionLibrary::TransformARCoordinates2D(EGoogleARCoreCoordinates2DType::Viewport, ..., EGoogleARCoreCoordinates2DType::Texture, ...) instead.

Bug fixes

• Fixed the crash when using UGoogleARCoreServicesFunctionLibrary::RemoveCloudARPin to remove an invalid UCloudARPin.
• Fixed the issue that the texture returned by UGoogleARCoreFrameFunctionLibrary::GetCameraTexture couldn't be rendered in the correct color space when using a surface material.
• Issue #47 Fixed a bug where ARCore Plugin prevented camera Pitch and Roll movement even when ARSession wasn't running.

GoogleARCore sample changes

• New AugmentedFaces sample project that shows an example of how to use the new Augmented Faces capabilities.
• ComputerVision sample now uses the new UGoogleARCoreFrameFunctionLibrary::TransformARCoordinates2D to calculate UVs to render the CPU image.

ARCore SDK for Unreal v1.6.0 (2018-12-06)

Engine and plugin changes

ARCore Unreal SDK v1.6.0 is based on Unreal 4.20.3 release. The Unreal Engine branch with the 1.6.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.20-arcore

New APIs

• ARCore Augmented Images are now supported in the cross-platform API in UE4 Augmented Reality Module.

Deprecations

• UGoogleARCoreAugmentedImageDatabase class has been deprecated. Please use the UARCandidateImage list on UARSessionConfig instead.
• UGoogleARCoreAugmentedImageDatabase::AddRuntimeAugmentedImageFromTexture() has been deprecated. Please use UARBlueprintLibrary::AddRuntimeCandidateImage() instead.
• UGoogleARCoreAugmentedImageDatabase::AddRuntimeAugmentedImage() has been deprecated. Please use UGoogleARCoreSessionFunctionLibrary::AddRuntimeCandidateImageFromRawbytes() instead.
• FGoogleARCoreAugmentedImageDatabaseEntry struct has been deprecated. Please use UARCandidateImage instead.
• UGoogleARCoreAugmentedImage::GetCenter() has been deprecated without replacement since it always return (0,0,0).
• UGoogleARCoreAugmentedImage::GetExtent() has been deprecated. Please use UARTrackedImage::GetEstimatedSize() instead.
• UGoogleARCoreAugmentedImage::GetImageIndex() has been deprecated. Please use UARTrackedImage::GetDetectedImage() to get the UARCandidateImage object instead.
• UGoogleARCoreAugmentedImage::GetImageName() has been deprecated. Please use UARCandidateImage::GetFriendlyName() instead.

Bug fixes

• Fixed the build issues in UE4 4.20 when targeting Android x86/x64.
• Fixed the issue that Actor relative transform doesn't replicate correctly.

GoogleARCore sample changes

• Updated the AugmentedImages sample to use the cross-platform API. Now the sample could work on both iOS and Android. (Need to use GoogleARCore plugin v1.6.0)
• Updated the CloudARPin sample with a new CloudARPin syncing flow to demonstrate the expected use case.
• Updated the edge detection code in ComputerVision sample to use a copied image buffer to workaround the slow image buffer read issue on Exynos precessors.

ARCore SDK for Unreal v1.5.0 (2018-09-28)

Engine and plugin changes

ARCore Unreal SDK v1.5.0 is based on Unreal 4.20.2 release. The Unreal Engine branch with the 1.5.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.20-arcore

New APIs

• New BP/C++ function UGoogleARCorePointCloud::GetPointId(…) which will return the unique identifier of a point in PointCloud that could persistent across frame.
• New BP/C++ function UGoogleARCorePointCloud::GetCameraTexture() which will return the UTexture that GoogleA RCore Plugin uses to render the passthrough camera.
• New BP/C++ functions UGoogleARCoreAugmentedImageDatabase::AddRuntimeAugmentedImageFromTexture(…) and UGoogleARCoreSessionConfig::SetAugmentedImageDatabase(…) to support adding Augmented Images at runtime.
• Google ARCore Plugin now uses FARSessionStatus.AdditionalInfo to provide additional session status information.

Bug fixes

• Fixed the issues that UARPlaneGeometry::GetExtent() and UGoogleARCoreAugmentedImage::GetExtent() return negative value.

GoogleARCore sample changes

• All GoogleARCore samples now show an error pop-up when the ARSession fails to start due to some error.
• HelloAR sample now places a blue Andy when the line trace hit a Feature Point with valid surface normal.

ARCore SDK for Unreal v1.4.0

ARCore SDK for Unreal v1.4.0 (2018-08-02)

Engine and plugin changes

ARCore Unreal SDK v1.4.0 is based on Unreal 4.20.0 release. The Unreal Engine branch with the 1.4.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.20-arcore

New APIs

• ARCore Camera Configuration: All supported ARCore camera configurations can now be queried and set through the OnConfigCamera callback in ARCoreEventManager. The callback will be called when ARCore session is about to start.
• ARCore camera now support auto focus. You can enabled it in your ARSessionConfig.
• New BP/C++ method ARCoreLineTraceRay that performs a line trace with an arbitrary ray constructed from the given start and end point.

Behavioral changes

• The AugmentedImageDatabase cooking process now print more information to the Unreal Editor log.

Bug fixes

• Fixed the crash in UGoogleARCoreServicesFunctionLibrary::RemoveCloudARPin when the passed in UCloudARPin was already removed.
• AugmentedImageDatabase cooking process correctly handles engine and project paths that contain spaces
• Fixed crash when starting ARCore session if AugmentedImageDatabase failed to cook correctly.

GoogleARCore sample changes

• ARCore "ComputerVision" sample now supports options to configure the CPU image resolution.

ARCore Unreal SDK v1.3.0

ARCore SDK for Unreal v1.3.0 (2018-06-21)

Engine and plugin changes

The Unreal Engine branch with the 1.3.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.19-arcore

New APIs

• Camera intrinsics data may now be queried in GoogleARCore plugin.

Behavioral changes

• Calling StartARSession or StartARCoreSession with a different ARSessionConfig while session is running will reconfigure the session without pausing and resuming it.
• GoogleARCoreServices plugin now provides two fields for Cloud Anchor API keys (AndroidAPIKey and IOSAPIKey) on each platform, so that developers can use restricted API keys separately.

Other changes

• GoogleARCore plugin now can be used when Android ndk-level is set to 19+.

Bug fixes

• Functions in GoogleARCore plugin now returns proper values when calling while ARCore session is paused.
• Fixed the issue that UARTrackedGeometry on ARCore platform is not set to EARTrackingState::NotTracking tracking state when AR session is paused.
• Fixed the issue that the Extent value in UARPlaneGeometry isn't set correctly in GoogleARCore plugin.

GoogleARCore sample changes

• CloudARPin sample now shows an error pop-up if the input host session IP address is invalid.
• Computer Vision sample now displays the camera GPU texture and CPU images intrinsics.

ARCore SDK for Unreal v1.2.1

Engine and plugin changes

The Unreal Engine branch with the 1.2.1 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.19-arcore

Changes

• Update the ARCore Android/iOS SDK to 1.2.1. See ARCore SDK for Android Release Notes and ARCore SDK for iOS Release Notes.

GoogleARCore sample changes

Bug fixes

• Fixed Augmented Images sample passthrough camera being rendered in linear space by disabling mobile HDR.

ARCore SDK for Unreal v1.2.0

Updated on 2018-05-11 to remove temporary APK sideload instructions.

Engine and plugin changes

Breaking changes

• The ARSessionConfig object now use two bools to set the session plane selection mode instead of using the EARPlaneDetectionMode enum.
• Renamed some values in EGoogleARCoreAvailability enum to fix typos.

New APIs

GoogleARCore plugin

• Augmented Images API. Allows ARCore apps to detect and track images.

GoogleARCoreServices plugin

A new plugin that surfaces the cross-platform ARCore features in Unreal Engine 4. This plugin provides a single unified API that works with both ARCore and ARKit when used with GoogleARCore Plugin (Android) and AppleARKit Plugin (iOS).

• CloudARPin API. Allows for hosting and resolving CloudARPin using the ARCore cloud anchors API. Hosting a CloudARPin assigns a cloud ID to it, which can then be shared with another ARCore user with an app using the same client API key. This second user can then use the ID to resolve a CloudARPin at the same position and orientation as the original hosted CloudARPin.

New capabilities

• Vertical plane detection. ARCore now detects both horizontal and vertical planes.

Deprecations

None.

Known Issues

• If the API key specified for authenticating with the ARCore Cloud Anchor service is invalid, the final cloud state of the CloudARPin will be ErrorInternalError instead of ErrorNotAuthorized. This is a known issue and will be fixed in an upcoming release.

Behavioral changes

• The default ARSessionConfig now detects both horizontal and vertical planes. Previously only horizontal planes where detected.

Other changes

• Upgraded 4.19-arcore branch to Unreal 4.19.1.
• Added a new function UGoogleARCorePointCloud::GetPointInTrackingSpace to provide point cloud position in tracking space.

Bug fixes

• Fixed the color space issue in ARCore passthrough camera rendering when building for OpenGL ES3.1. The camera image should have the same brightness for both OpenGL ES2 and OpenGL ES3.1 builds.

GoogleARCore sample changes

New samples

• New CloudARPin sample that shows how to share CloudARPins between devices on both Android and iOS.
• New AugmentedImages sample that shows how to configure images for detection and tracking in ARCore.

Existing samples

• HelloAR sample now detects both horizontal and vertical planes.

ARCore SDK for Unreal v1.1.0

ARCore 1.1 Release

ARCore enables AR applications to track a phone’s motion in the real world, detect planes in the environment, and understand lighting in the camera scene.

ARCore 1.1 provides two new APIs:

• API for synchronously (in the same frame) acquiring the image from a camera frame (without manually copying the image buffer from the GPU to the CPU). This lets you run your own image processing algorithms on the images from the camera.
• API for getting color correction information for the images captured by the camera. This lets you render your virtual objects in a way that matches the real-world lighting conditions.

GoogleARCore Plugin Changes

The Unreal Engine branch with the 1.1.0 version of GoogleARCore plugin is based on Unreal Engine 4.19.0. The branch is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.19-arcore
Note that the ARCore plugin in Epic's Unreal Engine 4.19 branch is still on v1.0.0.

Changes from Unreal 4.18 to Unreal 4.19

Unreal 4.19 provide a Unified AR Framework for building Augmented Reality (AR) apps for both Apple and Google handheld platforms. GoogleARCore plugin in 4.19 implemented this Unified AR Framework and provides additional functionality that is ARCore platform specific.

For information on the Unified Unreal AR Framework, see AugmentedReality API reference and ARAugmentedReality Blueprint reference.

Changes from v1.0.0 to v1.1.0 in Unreal 4.19

• Added the RGBScaleFactor filed in FGoogleARCoreLightEstimate struct for color correction.
• Added a new type UGoogleARCoreCameraImage and the C++/Blueprint function.
• UGoogleARCoreFunctionLibrary::AcquireCameraImage for providing camera frame CPU access.

Bug Fixes

• Fixed the issue that UGoogleARCorePointCloud::GetPoint() returns point in Unreal AR tracking space instead of world space.
• Fixed the issue that red and blue channel of the passthrough camera image is swapped when building for GLES 3.1.
• Fixed the issue that LineTraceTrackedObjects or ARCoreLineTrace doesn’t returns all ARTraceResult when multiple Plane channel is selected.

GoogleARCore Sample Changes

• Updated HelloARUnreal to Unreal 4.19.0 with GoogleARCore plugin 1.1.0.
• Added ambient color correction in HelloARUnreal sample.
• Added a new ComputerVision sample to demonstrate how to use new Camera Image API.

Supported Devices

This release is only supported on, and should only be installed on, qualified devices running Android N and later. See the list of ARCore supported devices for specific device models.

ARCore SDK for Unreal v1.0.0

ARCore 1.0 Release

ARCore enables AR applications to track a phone’s motion in the real world, detect planes in the environment, and understand lighting in the camera scene. ARCore 1.0 introduces the placement of anchors on textured surfaces (like posters or plant leaves) using oriented points. The Android Emulator in Android Studio 3.1 Beta supports ARCore 1.0.

ARCore 1.0 is available for use in production apps on supported Android devices. ARCore 1.0 introduces the concept of AR required and AR optional apps. Apps should be classified as required or optional to ensure the ARCore service install is properly handled by the Play Store.

Unreal Engine Release

The Unreal Engine branch with the 1.0.0 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.18-arcore

Note that the ARCore plugin in Epic's Unreal Engine 4.18 branch is still on the first preview release.
If you are on Epic's Unreal 4.18 branch, you can merge our 4.18-arcore branch to get the latest GoogleARCore plugin.

Supported Devices

• This release is only supported on, and should only be installed on, qualified devices running Android N and later. See the list of ARCore supported devices for specific device models.
• Apps built with ARCore Developer Preview or ARCore Developer Preview 2 are not supported on with ARCore 1.0.

Known Issues

Lifecycle issues

• ARCore does not support picture-in-picture mode.

ARCore SDK for Unreal Preview 2

ARCore SDK Preview 2

2018-01-30 EDIT: Added link to ARCore supported devices.

This is a developer preview SDK that enables prototyping AR experiences using ARCore on supported Android devices.

ARCore enables AR applications to track a phone’s motion in the real world, detect planes in the environment, and understand lighting in the camera scene.

This developer preview release is not intended for use in production apps that are shipped to customers. Future versions of ARCore might introduce breaking changes to the API.

Unreal Engine Release

The Unreal Engine branch with the preview2 version of GoogleARCore plugin is located here:
https://github.com/google-ar-unreal/UnrealEngine/tree/4.18-arcore-sdk-preview2

Note that the ARCore plugin in Epic's Unreal Engine 4.18 branch is still on the previous preview release.
If you are on Epic's Unreal 4.18 branch, you can merge our 4.18-arcore-sdk-preview2 branch to get the latest GoogleARCore plugin.

Supported Devices

This release is only supported on, and should only be installed on, qualified devices running Android N and later. See the list of ARCore supported devices for specific device models.

Installing this release on phones other than the supported devices might break other software on the device.

Notes

Multiple Versions of ARCore

• Apps built with the original ARCore SDK Developer Preview are only compatible with the original arcore-preview.apk. Apps built with ARCore SDK Developer Preview 2 are only compatible with arcore-preview2.apk.
• A developer can install versions of ARCore provided in both the original ARCore SDK Developer Preview and ARCore SDK Developer Preview 2 at the same time to facilitate using apps built with either SDK. The original arcore-preview.apk will be visible as “Tango Core” in the Android Apps settings whereas arcore-preview2.apk will be visible as “ARCore” in the Android Apps settings.

Known Issues

• ArSession_configure() does not apply settings changes if called while in a resumed state.

What’s New In Developer Preview 2

New SDK

• Developer Preview 2 features an Android NDK (C API)

API Changes

• The API interfaces have undergone significant revision and changes. All key functionality from the original Developer Preview remains, but the methods and function calls have changed.
• The API now natively supports attaching Anchors to Planes.

Lifecycle Improvements

• ARCore apps may recover planes and anchors if the session is paused and then resumed, provided the user is in roughly the same location and the environment or lighting has not changed significantly. This enables scenarios where AR content is not lost when the user briefly switches apps.

Fixes

• Many stability and performance improvements.