Full shader transpilation (#1342)

docs/api-reference/engine/model.md

@@ -175,15 +175,14 @@ when not provided will be deduced from `geometry` object.
 
 The constructor for the Model class. Use this to create a new Model.
 
-The following props can only be specified on construction:
-
 * `vs` - (VertexShader|*string*) - A vertex shader object, or source as a string.
 * `fs` - (FragmentShader|*string*) - A fragment shader object, or source as a string.
 * `varyings` (WebGL 2) - An array of vertex shader output variables, that needs to be recorded (used in TransformFeedback flow).
 * `bufferMode` (WebGL 2) - Mode to be used when recording vertex shader outputs (used in TransformFeedback flow). Default value is `gl.SEPARATE_ATTRIBS`.
 * `modules` - shader modules to be applied.
 * `program` - pre created program to use, when provided, vs, ps and modules are not used.
-* `shaderCache` - (ShaderCache) - Compiled shader (Vertex and Fragment) are cached in this object very first time they got compiled and then retrieved when same shader is used. When using multiple Model objects with duplicate shaders, use the same shaderCache object for better performance.
+* `programManager` - `ProgramManager` to use for program creation and caching.
+* `transpileShaders` - Transpile vertex and fragment shaders to GLSL 1.0.
 
 
 ### delete() : Model

docs/api-reference/engine/program-manager.md

@@ -78,6 +78,7 @@ Get a program that fits the parameters provided. If one is already cached, retur
 * `defines`: Object indicating `#define` constants to include in the shaders.
 * `modules`: Array of module objects to include in the shaders.
 * `inject`: Object of hook injections to include in the shaders.
+* `transpileShaders`: Transpile shaders to GLSL 1.0.
 
 ### `addDefaultModule(module: Object)`
 

docs/api-reference/shadertools/assemble-shaders.md

@@ -14,6 +14,7 @@ Takes the source code of a vertex shader and a fragment shader, and a list of mo
 * `modules`=`[]` (Array) - list of shader modules (either objects defining the module, or names of previously registered modules)
 * `inject`=`{}` (Object) - map of substituions,
 * `hookFunctions`=`[]` Array of hook functions descriptions. Descriptions can simply be the hook function signature (with a prefix `vs` for vertex shader, or `fs` for fragment shader) or an object with the hook signature, and a header and footer that will always appear in the hook function. For example:
+* `transpile`: force transpilation to GLSL ES 1.0 (see below)
 
 ```js
 [
@@ -122,5 +123,30 @@ new Model(gl, {
 });
 ```
 
+## Transpilation
+
+If the `transpile` option is used, `assembleShaders` will attempt to transpile shaders to GLSL ES 1.0. This is a limited text replacement and requires that certain conventions be followed:
+- Statements are written one per line.
+- Only one fragment shader output is supported.
+- GLSL 3.0-only features, such as 3D textures are not supported.
+
+Text transformations are performed according to the following tables:
+
+Vertex Shaders
+
+| 3.00 ES         | 1.00 ES     | Comment         |
+| ---             | ---         | ---             |
+| `in`            | `attribute` |                 |
+| `out`           | `varying`   |                 |
+
+Fragment Shaders
+
+| 3.00 ES         | 1.00 ES        | Comment |
+| ---             | ---            | ---     |
+| `in`            | `varying`      |         |
+| `out vec4 <varName>` | `gl_FragColor` | `<varName>` declaration is removed and usage in the code are replaced with `gl_FragColor`|
+| `texture`       | `texture2D`    | `texture` will be replaced with `texture2D` to ensure 1.00 code is correct. See note on `textureCube` below. |
+| `textureCube` * | `textureCube`  | `textureCube` is not valid 3.00 syntax, but must be used to ensure 1.00 code is correct, because `texture` will be substituted with `texture2D` when transpiled to 100. Also `textureCube` will be replaced with correct `texture` syntax when transpiled to 300. |
+
 
 

docs/developer-guide/shader-modules.md

@@ -154,38 +154,7 @@ For example:
 }
 ```
 
-## GLSL Syntax Conversion Reference
+## GLSL Versions
 
-Where possible, the shader assembler replaces keywords based on the version of the shader into which a module is inserted. While not all GLSL ES 3.0 features can be emulated, this allows many modules to be used across versions.
+Shader modules will undergo some basic text transformations in order to match the GLSL version of the shaders they are injected into. These transformations are generally limited to the naming of input variables, output variables and texture sampling functions. See [assembleShaders](/docs/api-reference/shadertools/assemble-shaders) documentation for more information.
 
-Syntax replacement tables are provided below:
-
-Vertex Shaders
-
-| 3.00 ES         | 1.00 ES     | Comment         |
-| ---             | ---         | ---             |
-| `in`            | `attribute` |                 |
-| `out`           | `varying`   |                 |
-
-Fragment Shaders
-
-| 3.00 ES         | 1.00 ES        | Comment |
-| ---             | ---            | ---     |
-| `in`            | `varying`      |         |
-| `out`           | `gl_FragColor` |         |
-| `out`           | `gl_FragData`  |         |
-| `texture`       | `texture2D`    | `texture` will be replaced with `texture2D` to ensure 1.00 code is correct. See note on `textureCube` below. |
-| `textureCube` * | `textureCube`  | `textureCube` is not valid 3.00 syntax, but must be used to ensure 1.00 code is correct, because `texture` will be substituted with `texture2D` when transpiled to 100. Also `textureCube` will be replaced with correct `texture` syntax when transpiled to 300. |
-| `gl_FragDepth`  | `gl_FragDepthEXT` | WebGL 1: **EXT_frag_depth** |
-
-
-| 3.00 ES             | 1.00 ES                | Comment |
-| ---                 | ---                    | --- |
-| `texture2DLod`      | `texture2DLodEXT`      | WebGL 1: **EXT_shader_texture_lod** |
-| `texture2DProjLod`  | `texture2DProjLodEXT`  | WebGL 1: **EXT_shader_texture_lod** |
-| `texture2DProjLod`  | `texture2DProjLodEXT`  | WebGL 1: **EXT_shader_texture_lod** |
-| `textureCubeLod`    | `textureCubeLodEXT`    | WebGL 1: **EXT_shader_texture_lod** |
-| `texture2DGrad`     | `texture2DGradEXT`     | WebGL 1: **EXT_shader_texture_lod** |
-| `texture2DProjGrad` | `texture2DProjGradEXT` | WebGL 1: **EXT_shader_texture_lod** |
-| `texture2DProjGrad` | `texture2DProjGradEXT` | WebGL 1: **EXT_shader_texture_lod** |
-| `textureCubeGrad`   | `textureCubeGradEXT`   | WebGL 1: **EXT_shader_texture_lod** |

modules/engine/src/lib/model.js

@@ -49,9 +49,29 @@ export default class Model {
     this._programManagerState = -1;
     this._managedProgram = false;
 
-    const {program = null, vs, fs, modules, defines, inject, varyings, bufferMode} = props;
-
-    this.programProps = {program, vs, fs, modules, defines, inject, varyings, bufferMode};
+    const {
+      program = null,
+      vs,
+      fs,
+      modules,
+      defines,
+      inject,
+      varyings,
+      bufferMode,
+      transpileShaders
+    } = props;
+
+    this.programProps = {
+      program,
+      vs,
+      fs,
+      modules,
+      defines,
+      inject,
+      varyings,
+      bufferMode,
+      transpileShaders
+    };
     this.program = null;
     this.vertexArray = null;
     this._programDirty = true;
@@ -143,8 +163,28 @@ export default class Model {
   }
 
   setProgram(props) {
-    const {program, vs, fs, modules, defines, inject, varyings, bufferMode} = props;
-    this.programProps = {program, vs, fs, modules, defines, inject, varyings, bufferMode};
+    const {
+      program,
+      vs,
+      fs,
+      modules,
+      defines,
+      inject,
+      varyings,
+      bufferMode,
+      transpileShaders
+    } = props;
+    this.programProps = {
+      program,
+      vs,
+      fs,
+      modules,
+      defines,
+      inject,
+      varyings,
+      bufferMode,
+      transpileShaders
+    };
     this._programDirty = true;
   }
 
@@ -376,8 +416,26 @@ export default class Model {
     if (program) {
       this._managedProgram = false;
     } else {
-      const {vs, fs, modules, inject, defines, varyings, bufferMode} = this.programProps;
-      program = this.programManager.get({vs, fs, modules, inject, defines, varyings, bufferMode});
+      const {
+        vs,
+        fs,
+        modules,
+        inject,
+        defines,
+        varyings,
+        bufferMode,
+        transpileShaders
+      } = this.programProps;
+      program = this.programManager.get({
+        vs,
+        fs,
+        modules,
+        inject,
+        defines,
+        varyings,
+        bufferMode,
+        transpileShaders
+      });
       if (this.program && this._managedProgram) {
         this.programManager.release(this.program);
       }

modules/engine/src/lib/program-manager.js

@@ -49,7 +49,15 @@ export default class ProgramManager {
   }
 
   get(props = {}) {
-    const {vs = '', fs = '', defines = {}, inject = {}, varyings = [], bufferMode = 0x8c8d} = props; // varyings/bufferMode for xform feedback, 0x8c8d = SEPARATE_ATTRIBS
+    const {
+      vs = '',
+      fs = '',
+      defines = {},
+      inject = {},
+      varyings = [],
+      bufferMode = 0x8c8d,
+      transpileShaders = false
+    } = props; // varyings/bufferMode for xform feedback, 0x8c8d = SEPARATE_ATTRIBS
 
     const modules = this._getModuleList(props.modules); // Combine with default modules
 
@@ -75,7 +83,9 @@ export default class ProgramManager {
 
     const hash = `${vsHash}/${fsHash}D${defineHashes.join('/')}M${moduleHashes.join(
       '/'
-    )}I${injectHashes.join('/')}V${varyingHashes.join('/')}H${this.stateHash}B${bufferMode}`;
+    )}I${injectHashes.join('/')}V${varyingHashes.join('/')}H${this.stateHash}B${bufferMode}${
+      transpileShaders ? 'T' : ''
+    }`;
 
     if (!this._programCache[hash]) {
       const assembled = assembleShaders(this.gl, {
@@ -84,7 +94,8 @@ export default class ProgramManager {
         modules,
         inject,
         defines,
-        hookFunctions: this._hookFunctions
+        hookFunctions: this._hookFunctions,
+        transpile: transpileShaders
       });
 
       this._programCache[hash] = new Program(this.gl, {

modules/engine/test/lib/model.spec.js

@@ -21,6 +21,32 @@ const DUMMY_FS = `
   void main() { gl_FragColor = vec4(1.0); }
 `;
 
+const VS_300 = `#version 300 es
+
+  in vec4 positions;
+  in vec2 uvs;
+
+  out vec2 vUV;
+
+  void main() {
+    vUV = uvs;
+    gl_Position = positions;
+  }
+`;
+
+const FS_300 = `#version 300 es
+  precision highp float;
+
+  in vec2 vUV;
+
+  uniform sampler2D tex;
+
+  out vec4 fragColor;
+  void main() {
+    fragColor = texture(tex, vUV);
+  }
+`;
+
 test('Model#construct/destruct', t => {
   const {gl} = fixture;
 
@@ -244,7 +270,7 @@ test('Model#program management - getModuleUniforms', t => {
   t.end();
 });
 
-test('getBuffersFromGeometry', t => {
+test('Model#getBuffersFromGeometry', t => {
   const {gl} = fixture;
 
   let buffers = getBuffersFromGeometry(gl, {
@@ -303,3 +329,27 @@ test('getBuffersFromGeometry', t => {
 
   t.end();
 });
+
+test('Model#transpileShaders', t => {
+  const {gl} = fixture;
+
+  let model;
+
+  t.throws(() => {
+    model = new Model(gl, {
+      vs: VS_300,
+      fs: FS_300
+    });
+  }, "Can't compile 300 shader with WebGL 1");
+
+  t.doesNotThrow(() => {
+    model = new Model(gl, {
+      vs: VS_300,
+      fs: FS_300,
+      transpileShaders: true
+    });
+  }, 'Can compile transpiled 300 shader with WebGL 1');
+
+  t.ok(model.program, 'Created a program');
+  t.end();
+});

modules/engine/test/lib/program-manager.spec.js

@@ -19,6 +19,32 @@ void main(void) {
 }
 `;
 
+const VS_300 = `#version 300 es
+
+  in vec4 positions;
+  in vec2 uvs;
+
+  out vec2 vUV;
+
+  void main() {
+    vUV = uvs;
+    gl_Position = positions;
+  }
+`;
+
+const FS_300 = `#version 300 es
+  precision highp float;
+
+  in vec2 vUV;
+
+  uniform sampler2D tex;
+
+  out vec4 fragColor;
+  void main() {
+    fragColor = texture(tex, vUV);
+  }
+`;
+
 test('ProgramManager#import', t => {
   t.ok(ProgramManager !== undefined, 'ProgramManager import successful');
   t.end();
@@ -309,3 +335,37 @@ test('ProgramManager#release', t => {
 
   t.end();
 });
+
+test('ProgramManager#transpileShaders', t => {
+  const {gl} = fixture;
+
+  const pm = new ProgramManager(gl);
+
+  t.throws(() => {
+    pm.get({
+      vs: VS_300,
+      fs: FS_300
+    });
+  }, "Can't compile 300 shader with WebGL 1");
+
+  t.doesNotThrow(() => {
+    pm.get({
+      vs: VS_300,
+      fs: FS_300,
+      transpileShaders: true
+    });
+  }, 'Can compile transpiled 300 shader with WebGL 1');
+
+  const programTranspiled = pm.get({vs, fs, transpileShaders: true});
+  const programUntranspiled = pm.get({vs, fs});
+  const programTranspiled2 = pm.get({vs, fs, transpileShaders: true});
+
+  t.equals(programTranspiled, programTranspiled2, 'Transpiled programs match');
+  t.notEquals(
+    programTranspiled,
+    programUntranspiled,
+    'Transpiled program does not match untranspiled program'
+  );
+
+  t.end();
+});