Merge branch 'master' into ib/fix-warnings

.eslintrc

@@ -15,6 +15,7 @@
     "no-inline-comments": 0,
     "camelcase": 0,
     "tree-shaking/no-side-effects-in-initialization": 0,
+    "max-statements": 0,
     "luma-gl-custom-rules/check-log-call": 1
   }
 }

dev-docs/RFCs/README.md

@@ -22,6 +22,8 @@ These are early ideas not yet associated with any release
 | RFC | Author | Status | Description |
 | --- | --- | --- | --- |
 | **WIP/Draft** | | | |
+| [**Automatic Shader Module Injection**](vNext/glsl-function-replacement-rfc.md) | @ibgreen | **Draft** | Replace functions and inject attributes in shaders |
+| [**Automatic Shader Module Injection**](vNext/dynamic-shader-regeneration-rfc.md) | @ibgreen | **Draft** | Automatically recompile shader modules when "props" are updated. |
 | [**Automatic Shader Module Injection**](vNext/automatic-shader-module-injection-rfc.md) | @ibgreen | **Draft** | Automatically inject code required by a shader module |
 | [**Dist Size Reduction**](vNext/reduce-distribution-size-rfc.md) | @ibgreen | **Draft** | Reduce luma.gl impact on app bundle size |
 

dev-docs/RFCs/vNext/dynamic-shader-regeneration-rfc.md

@@ -0,0 +1,17 @@
+# RFC: Dynamic Shader Regeneration
+
+Being able to specify props that affect the shader source code means that either:
+* Changes to these props need to be detected, shaders recompiled and programs relinked
+* Or we need to specify in documentation that shaders are only generated on creation
+
+Apart from the work/extra complexity required to accurately detect when shaders should be recompiled, the big issue is that recompilation and relinking of GLSL shaders tends to be slow (sometimes extremely slow, as in several seconds, or even half a minute or more) and happens synchronously on the main WebGL thread, freezing all rendering or even the entire app.
+
+If we support dynamic recompilation of shaders based on changing props, we therefore need to be extra careful to avoid (and detect/warn for) "spurious" recompiles. The user might not realize that the supplied props are triggering constant recompiles causing app performance to crater.
+
+For ideas around working around slow shader compilation, see:
+* [Deferring linking till first draw](http://toji.github.io/shader-perf/)
+* [KHR_parallel_shader_compile extension](https://www.khronos.org/registry/webgl/extensions/KHR_parallel_shader_compile/)
+* [KHR_parallel_shader_compile discussion](https://github.com/KhronosGroup/WebGL/issues/2690)
+* [Intel webGL 2018 presentation](https://docs.google.com/presentation/d/1qeD2xio2dgkqWGQucZs6nezQPePOM4NtV9J54J7si9c/htmlpresent) search for KHR_parallel_shader_compile
+
+

dev-docs/RFCs/vNext/glsl-function-replacement-rfc.md

@@ -0,0 +1,145 @@
+# RFC: GLSL Function Replacement And Attribute Injection
+
+* Authors: Ib Green
+* Date: March 2019
+* Status: **Draft**
+
+This RFC is part of the [shadertools roadmap](dev-docs/roadmaps/shadertools-roadmap.md).
+
+
+## Summary
+
+This RFC proposes a system for shader module users to replace GLSL functions (by supplying a map of function names to strings fragments of GLSL code) as well as injecting new attributes and uniforms.
+
+This supports use case such as:
+* Completely or partially skip CPU side attribute generation
+* Work directly on supplied binary data (e.g. binary tables).
+* Do additional GPU processing on data, allowing performant custom effects.
+
+
+## Overview
+
+Let's start with a couple of API examples to illustrate what we want to achieve:
+
+
+### API Example: Using Columnar Binary Data Sirectly
+
+Columnar binary data systems (like Apache arrow) will often store primitive values (e.g. a Float32) in separate columns, rather than as a `vec2` or `vec3`. This means that even if we exposed these columns as attributes in GLSL, we cannot use them directly as the expected `vec3` attribute for positions. However, if we could just define a snippet of GLSL code to override how the shader extracs a position from attributes (`getPosition`), we could still use these values rather than pre-processing them in JavaScript:
+
+```
+new Model({
+  vs,
+  fs,
+
+  // inject new attributes
+  glsl: {
+    attributes: {
+      x: 'float',
+      y: 'float'
+    },
+  },
+
+  // Define a custom glsl accessor (string valued accessor will be treated as GLSL)
+  getPosition: 'return vec3(x, y, 0.);',
+
+  // Supply values to the injected attributes
+  attributes: {
+    x: new Float32Array(...),
+    y: new Float32Array(...)
+  },
+})
+```
+
+The input vertex shader would be defined as follows:
+
+```
+in vec3 instancePosition;
+
+// Shadertools would e.g. do a line-based replacement of getPosition
+vec3 getPosition() { return instancePosition; }
+
+main() {
+  const position = getPosition();
+  // Note: shader does not assume that `position = instancePositions`, but calls an overridable GLSL function!
+}
+```
+
+Shadertools would resolve this to:
+
+```
+in vec3 instancePosition; // Note: no longer used, would be nice if this could be removed but not critical.
+
+in float x;
+in float y;
+
+// Shadertools has done a line-based replacement of getPosition
+vec3 getPosition() { return vec3(x, y, 0.);; }
+
+main() {
+  const position = getPosition();
+  // Note: layer no longer assumes `position = instancePositions`, but calls overridable GLSL function
+}
+```
+
+
+### API Example: Adapting Data to fit layer's Requirement
+
+Let's say that we load binary data for a point cloud that only contains a single value per point (e.g. `reflectance`), but the shader we want to use only supports specifying an RGBA color per point. While we could certainly write a JavaScript function to generate a complete RGBA color array attribute, this extra time and memory could be avoided:
+
+```
+new Model({
+  vs: POINT_CLOUD_VS,
+  fs: POINT_CLOUD_FS,
+
+  // inject new attributes and uniforms
+  glsl: {
+    attributes: {
+   	  reflectance
+    },
+    uniforms: {
+   	  alpha
+    }
+  },
+
+  // Define a custom glsl accessor (string valued accessor will be treated as GLSL)
+  getColor: 'return vec4(reflectance, reflectance, reflectance, alpha);',
+
+  // Supply actual values to the injected attributes and uniforms
+  attributes: {
+    reflectance: new Float32Array(...)
+  },
+  uniforms: {
+    alpha: 0.5
+  }
+})
+```
+
+
+## Design Discussions
+
+### Shader Buy-In
+
+While it might be possible to use the function replacement system to replace of arbitrary functions in a shader, this can lead to brittle overrides as the base shader is updated.
+
+The ideal setup is one where the shader is written with the intention of having certain functions be overridable. Often, the readout of attributes would be a typical place to add an overridable function, allowing the shader to be used with different attributes.
+
+Apart from attribute access, a shader could potentially define additional overridable functions that it would call at the right moments. These would then normally be separately documented.
+
+
+## Open Issues
+
+### Naming Conflicts
+
+Define/reinforce naming conventions/prefixes for GLSL functions and variables in shader code to maximize predictability/minimize conflicts with user's GLSL accessor code.
+
+Since we do not have a true syntax aware parser/string replacement system we also risk replacing unrelated strings. This risk is becoming bigger the more extensive our shader replacement support becomes.
+
+
+### Fragment Shader Support
+
+This RFC as written does not offer any way for users to modify fragment shaders.
+
+If it is desirable to offer the ability to redefine GLSL functions in the fragment shader, the GLSL accessor system outlined in this system does not extend well.
+* attributes are not available in the fragment shader, and such data needs to be explicitly plumbed through as `varyings` that "eat into" a very limited bank (often only 8 varyings or `vec4`s).
+* It may be necessary to implement a "varying compressor" and additonal props with more complicated semantics to describe the data flow.
+Naturally, if user-defined GLSL functions in the fragment shaders can only use the data already available, the system should be simpler.

docs/api-reference/lights/point-light.md

@@ -28,3 +28,7 @@ const pointLight = new PointLight({color, intensity, position});
 * `color` - (*array*,)  RGB color of point light source, default value is `[255, 255, 255]`.
 * `intensity` - (*number*) Strength of point light source, default value is `1.0`.
 * `position` - (*array*,)  Location of point light source, default value is `[0, 0, 1]`.
+* `attenuation` - (*array*,)  Attenuation of point light source based on distance.
+In order of Constant, Linear, Exponential components.
+For details see [this tutorial](http://ogldev.atspace.co.uk/www/tutorial20/tutorial20.html).
+Use `[1, 0, 0]` for no attenuation. Default value is `[0, 0, 1]`.

docs/api-reference/webgl/texture-2d.md

@@ -1,6 +1,6 @@
 # Texture2D
 
-2D textures hold basic "single image" textures (although technically they can contain multiple mimap levels). They hold image memory of a certain format and size, determined at initialization time. They can be read from using shaders and written to by attaching them to frame buffers.
+2D textures hold basic "single image" textures (although technically they can contain multiple mipmap levels). They hold image memory of a certain format and size, determined at initialization time. They can be read from using shaders and written to by attaching them to frame buffers.
 
 Most texture related functionality is implemented by and documented on the [Texture](/docs/api-reference/webgl/texture.md) base class. For additional information, see [OpenGL Wiki](https://www.khronos.org/opengl/wiki/Texture).
 

docs/api-reference/webgl/texture-3d.md

@@ -1,15 +1,13 @@
 # Texture3D (WebGL2)
 
-A `Texture3D` holds a number of textures of the same size and format. The entire array can be passed to the shader which uses an extra texture coordinate to sample from it. A core feature of `Texture3D` is that the entire stack of images can passed as a single uniform to and accessed in a GLSL shader, and sampled using 3D coordinates.
-
-3D textures are typically used to store volumetric data or for 3D lookup tables in shaders.
+3D textures hold basic volumetric textures and can be thought of 3-dimentional arrays with a width, height and depth. They hold image memory of a certain format and size, determined at initialization time. They can be sampled in shaders using the `texture` function with a 3D texture coordinate.
 
 Most texture related functionality is implemented by and documented on the [Texture](/docs/api-reference/webgl/texture.md) base class. For additional information, see [OpenGL Wiki](https://www.khronos.org/opengl/wiki/Texture).
 
 
 ## Usage
 
-Create a new texture array
+Create a new 3D texture
 ```js
 if (Texture3D.isSupported()) {
   texture3D = new Texture3D(gl, {...});
@@ -21,15 +19,15 @@ if (Texture3D.isSupported()) {
 
 * `handle` - The underlying `WebGLTexture`
 * `target` - Always `GL.TEXTURE_3D`
-* `depth` - the number of texture layers
-* `width` - width of the layer textures
-* `height` - height of the layer textures
-* `format` - format of the layer textures
+* `width` - width of texture
+* `height` - height of texture
+* `depth` - depth of the texture
+* `format` - format of texture
 
 
 ## Methods
 
-`Texture3D` is a subclass of the [Texture](texture.md) and [Resource](resource.md) classes and inherit all methods and members of those classes.
+`Texture3D` is a subclass of the [Texture](texture.md) and [Resource](resource.md) classes and inherit all methods and members of those classes. Note that `setSubImageData` is not currently supported for 3D textures.
 
 
 ### Texture3D.isSupported(gl)
@@ -41,11 +39,23 @@ Returns true if the context supports creation of `Texture3Ds`.
 
 `new Texture3D(gl, {parameters})`;
 
+```
+const texture = new Texture3D(gl, {
+  width: TEXTURE_DIMENSIONS,
+  height: TEXTURE_DIMENSIONS,
+  depth: TEXTURE_DIMENSIONS,
+  data: textureData,
+  format: gl.RED,
+  dataFormat: gl.R8
+});
+```
+
 * `gl` (WebGLRenderingContext) - gl context
 * `data`=`null` (*) - See below.
 * `width`=`0` (*Number*) - The width of the texture.
 * `height`=`0` (*Number*) - The height of the texture.
-* `mipmaps`=`GL/ (*Enum*, default false) - `n`th mipmap reduction level, 0 represents base image
+* `depth`=`0` (*Number*) - The depth of the texture.
+* `mipmaps`=`true` (*Boolean*) - whether to generate mipmaps
 * `format` (*enum*, default `GL.RGBA`) - internal format that WebGL should use.
 * `type` (*enum*, default is autodeduced from format) - type of pixel data (GL.UNSIGNED_BYTE, GL.FLOAT etc).
 * `dataFormat` (*enum*, default is autodeduced from `format`) - internal format that WebGL should use.