Halide OpenGL/GLSL backend
==========================

Halide's OpenGL backend offloads image processing operations to the GPU by
generating GLSL-based fragment shaders.

Compared to other GPU-based processing options such as CUDA and OpenCL, OpenGL
has two main advantages: it is available on basically every desktop computer
and mobile device, and it is generally well supported across different
hardware vendors.

The main disadvantage of OpenGL as an image processing framework is that the
computational capabilities of fragment shaders are quite restricted. In
general, the processing model provided by OpenGL is most suitable for filters
where each output pixel can be expressed as a simple function of the input
pixels. This covers a wide range of interesting operations like point-wise
filters and convolutions; but a few common image processing operations such as
histograms or recursive filters are notoriously hard to express in GLSL.


Writing OpenGL-Based Filters
----------------------------

To enable code generation for OpenGL, include "opengl" in the target specifier
passed to Halide. Since OpenGL shaders are limited in their computational
power, you must also specify a CPU target for those parts of the filter that
cannot or should not be computed on the GPU. Examples of valid target
specifiers are

  host-opengl
  x86-opengl-gpu_debug

Adding "gpu_debug", as in the second example, adds additional logging output
and is highly recommended during development.

By default, filters compiled for OpenGL targets run completely on the CPU.
Execution on the GPU must be enabled for individual Funcs by appropriate
scheduling calls.

GLSL fragment shaders implicitly loop over two spatial dimensions x,y and the
color channel

  Func f;
  Var x, y, c;
  f(x, y, c) = ...;
  f.bound(c, 0, 3);
  f.glsl(x, y, c);

You have to specify explicit bounds for the color channel before calling
glsl() since OpenGL does not support loops over a variable number of color
channels.


JIT Compilation
---------------

To execute JIT-compiled filters, Halide tries to load a dynamic library called
'HalideOpenGLRuntime', which contains platform-specific code for interacting
with the system's OpenGL implementation. A simple default implementation of
this library can be found in 'bin/HalideOpenGLRuntime.{so,dylib,dll}'.

Examples for JIT execution of OpenGL-based filters can be found in test/opengl.


AOT Compilation
---------------

When AOT (ahead-of-time) compilation is used, Halide generates OpenGL-enabled
object files that can be linked to and called from a host application. In
general, this is fairly straightforward, but a few things must be taken care
of.

On Linux and OS X, Halide creates its own OpenGL context unless the current
thread already has an active context.  On other platforms you have to link
implementations of the following two functions with your Halide code:

  extern "C" int halide_opengl_create_context(void *) {
    return 0; // if successful
  }

  extern "C" void *halide_opengl_get_proc_addr(void *, const char *name) {
    ...
  }

Halide allocates and deletes textures as necessary.  Applications may manage
the textures by hand by setting the 'dev' field in buffer_t; this is most
useful for reusing image data that is already stored in textures. Some
rudimentary checks are performed to ensure that externally allocated textures
have the correct format, but in general that's the responsibility of the
application.


Limitations
-----------

GLSL shaders can only express a subset of the Halide language.  The current
implementation of the OpenGL backend targets OpenGL ES 2.0, which has
additional restrictions, but is widely available on both mobile devices and
traditional computers. Some limitations include:

  * Only uint8 textures are properly supported so far. uint16 buffers work in
    principle, but are truncated to 8 bits by many OpenGL
    implementations. Support for float and half-float textures is planned for
    the near future.

  * Only 2D images with 1, 3, or 4 color channels can be scheduled;

  * Textures are either read-only or write-only, never read-write;

  * Reductions cannot be implemented in GLSL and must be run on the CPU.

The maximum texture size in OpenGL is GL_MAX_TEXTURE_SIZE, which is often
smaller than the image of interest; on mobile devices, for example,
GL_MAX_TEXTURE_SIZE is commonly 2048. Tiling must be used to process larger
images.