Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 

README.md

AR Environmental Lighting

Tested on Unity 2018.1.0f2

Captures the camera video frame and uses it for spherical environment mapping. ARKit 1.5 and ARCore 1.2 supported.

The main stage are:

  1. Capture the Camera Background

The camera background is blitted to a RenderTexture. This RenderTexture will be smaller then the screen size. This makes the following operations more performant and gives us some cheap filtering when it is downscaled. The blit must be done in a CommandBuffer so that the external camera sampler is access correctly. The texture is accessed in a platform specific way. ARCore relies on an external sampler OpenGLES extension, so requires a GLSL shader. ARKit uses a YCrCb image format with multiple buffers. The shaders reconstructs the RGB color.

  1. Blur the Background

The lighting is an approximation, so we blur the camera blur to avoid bright, flickering highlights.

  1. Lighting

The background is wrapped around the scene much like a MatCap or spherical environment map. This is not strictly accurate as the camera edges are wrapped around to the front of the object, but gives a "good enough" result in most AR scenes.

The shader calculates the reflection vector and then converts that into a UV coordinate.

There are two ways to light the scene. The first uses Unity's real-time GI system by combining a skybox with reflection and light probes in the scene. This works with any existing materials. The view direction must be corrected in the skybox shader to account for the pass for each skybox face.

The second uses the "finalcolor" function in a shader, so it requires a customer shader. This method is more performant.

Usage

Tap on the screen to set the position of the objects in the world.

Video

Robot Kyle Walking

About

A demonstration of using the AR camera background for lighting a scene.

Resources

License

Releases

No releases published

Packages

No packages published