WebGL 2 and WebGL Extensions Roadmap

[pjcozzi]

Extensions in Canary

• WEBGL_debug_shaders and WEBGL_debug_renderer_info - no longer require privileged access.
• OES_element_index_uint - already used by createVertexArrayFromGeometry, but could also be used by billboards and polylines that use vertex buffer directly.
• OES_texture_half_float
• OES_texture_float_linear and OES_texture_half_float_linear for completeness.
• WEBGL_compressed_texture_s3tc - for glTF.
• EXT_frag_depth (still draft) - opaque ray-traced geometry, e.g., ellipsoids, sensors, etc.

Draft WebGL Extensions

• WEBGL_color_buffer_float and EXT_color_buffer_half_float for HDR (I think).
• WEBGL_compressed_texture_atc and WEBGL_compressed_texture_pvrtc - for glTF
• WEBGL_draw_buffers - first for combing color and pick passes in a single pass. Later, with the post-processing framework, we can combine other passes like glow and velocity. Shadow and reflection passes will still be separate since they are rendered with different view parameters.
• WEBGL_shared_resources - for multiple canvases, but not web workers?

Proposed WebGL Extensions

• ANGLE_timer_query, for profiling tools.
• WEBGL_compressed_texture_etc1 - for glTF.
• WEBGL_dynamic_texture - for video textures.

OpenGL ES 3.0 Features

• Multisampled Renderbuffers - AA without using FXAA.
• Half-float vertex attributes, i.e., GL_OES_vertex_half_float.
• Uniform buffer objects - for backing uniforms with buffers based on update frequency. Also see Optimize setting uniforms #649.
• Sampler objects - the Cesium abstractions already look like this to some extent.
• Immutable textures - reduce driver validation (EXT_texture_storage).
• Instancing, i.e., ANGLE_instanced_arrays, for batching geometry and models.
• Primitive restart - batch render line strips instead of individual lines.
• Transform feedback - perhaps for particle systems.
• [ ] Program binaries - to reduce shader compile times. (Not in WebGL 2.0)

More on ES 3.0

• OpenGL ES 3.0 at SIGGRAPH 2012
• OpenGL ES 3.0 – Halti has arrived

More on WebGL 2.0

• What's coming in WebGL 2.0
• WebGL 2 Specification
• Mozilla Platform/GFX/WebGL2

WebGL 2 Demos

• http://toji.github.io/webgl2-particles/

Done

• OES_vertex_array_object - Geometry and Appearances #766.

[pjcozzi]

Uniform buffer objects

The renderer could always expose UBOs and emulate them with traditional uniforms or uniform arrays if they are not supported. This will require some GLSL macros or shader patching for the uniform/uniform-buffer declarations.

Automatic uniforms can be grouped into uniform buffers based on update frequency: per frame and per object (model-matrix).

Each command can provide two uniform buffer: one for uniforms that don't change often and one for uniforms that do. This maps to technique (don't change) and material (do change) parameters. Or one uniform buffer might be fine since the command creator won't have to figure out how to organize them.

[pjcozzi]

Cesium Implementation Status (after #3094)

• OES_standard_derivatives is disabled. dFdx, dFdy and fwidth are not recognized when compiling the shader.
  • There should be one test failure for materials because the PolylineArrowMaterial uses standard derivatives.
• EXT_frag_depth is disabled. gl_FragDepth is not recognized when compiling the shader (neither is gl_FragDepthEXT).
• OES_texture_float is disabled. Could not create a texture with PixelDatatype.FLOAT.
• WEBGL_depth_texture is disabled. Could not attach a depth-stencil texture to a FBO.

[pjcozzi]

• WebGL 2.0 talk from SIGGRAPH Asia 2015 - https://docs.google.com/presentation/d/1Orx0GB0cQcYhHkYsaEcoo5js3c5-pv7ahPniIRIzzfg/edit?usp=sharing

[pjcozzi]

• It will be easy enough to do by hand, but here's a tool to convert WebGL 1 to WebGL 2 shaders: https://github.com/Jam3/glsl-100-to-300/

[pjcozzi]

WebGL 2 Samples

• https://github.com/WebGLSamples/WebGL2Samples
• https://floooh.github.io/oryol-webgl2/

[pjcozzi]

• http://webgl2fundamentals.org/

[pjcozzi]

WebGL 2 articles on the RTR blog:

• WebGL 2 Basics
• WebGL 2: New Features

[pjcozzi]

Texture Array: we could use TEXTURE_2D_ARRAY in some cases where texture atlases are used. It will be simpler and great for dynamic updates when the size doesn't change, #2319

[pjcozzi]

• MDN has doc on WebGL 2 - see the Firefox 51 release

[pjcozzi]

• WebGL 2 lands in Firefox

[pjcozzi]

• WebGL 2 from WebGL 1
• 4.1 Backwards Incompatibility

[lilleyse]

@mramato 👍

That'll be on me an @YoussefV once we start the WebGL 2 / MSAA work.

[lilleyse]

In WebGL 2 the camera zooms towards the globe instead of towards the geometry that the mouse is hovering over. This seems like a bug in either ScreenSpaceCameraController or scene.pickPosition.

Sandcastle

WebGL 1

WebGL 2

[dennisadams]

@lilleyse
When ScreenSpaceCameraController needs a pickPosition, it calls both scene.pickPositionWorldCoordinates and globe.pickWorldCoordinates and chooses the closer of the two, assuming both are defined.
In our case, scene.pickPositionWorldCoordinates returns undefined and so ScreenSpaceCameraController uses the globe pick instead.
I played around a little and the problem seems to be that the depth texture is empty.
With the help of this error, I tried hard-coding a NEAREST filtering and it worked alright (even the error was gone). So I guess it's some problem with linear minification/magnification filtering.
I tried it with the changes from 8969, so that can also be a factor.

[lilleyse]

@dennisadams awesome, that fixes it for me as well. Just to confirm you added nearest filtering to the depth-stencil texture?

  globeDepth._depthStencilTexture = new Texture({
    context: context,
    width: width,
    height: height,
    pixelFormat: PixelFormat.DEPTH_STENCIL,
    pixelDatatype: PixelDatatype.UNSIGNED_INT_24_8,
    sampler: Sampler.NEAREST // Fixes the zoom
  });

[baothientran]

@dennisadams @lilleyse That also fixes the Silhouette post processing as well. Also I look it up, and in WebGL 2, it seems depth texture cannot have linear filtering. That may be a potential problem with other depth textures used by Cesium as well

[dennisadams]

Just to confirm you added nearest filtering to the depth-stencil texture?

I went on a safe bet - hard-coded the Texture constructor.

[lilleyse]

Labels look a bit different in WebGL 2. Click the images to see the difference.

WebGL 2

WebGL 1

Sandcastle

[lilleyse]

The AO post process has artifacts: Local sandcastle.

[lilleyse]

Polyline arrows start to get squashed. Local sandcastle

[lilleyse]

Cesium Inspector is extremely show when "Show Frustums" is checked. Local sandcastle

[lilleyse]

Contour lines are much thinner in WebGL 2. Local sandcastle.

First thing I would look into is GL_OES_standard_derivatives which is another built-in define that's part of WebGL 1 but maybe not WebGL 2 (I haven't checked). GL_EXT_frag_depth is another one we should investigate.

[lilleyse]

Alright that's all I've uncovered so far by going through each sandcastle example with WebGL2 enabled. @baothientran I recomend starting with #797 (comment) and seeing what other bugs are fixed in the process.

[lilleyse]

@baothientran since the number of bugs fixes was starting to grow too much I consolidated them into a single CHANGES.md entry: 65757f8

[lilleyse]

Also I updated the checklist above: #797 (comment)

[lilleyse]

• modernizeShader is too slow. This is something we should address before defaulting to WebGL 2.
  • This may mean upgrading all shaders to GLSL ES 3.0 with some DSL (aka @pjcozzi's comment WebGL 2 and WebGL Extensions Roadmap #797 (comment))
  • Or see WebGL2.0. When i add 500 entities, there will be a few seconds of severe lag. But in WebGL1.0, it's ok. #9201 for a workaround

[ebogo1]

Suggestions to improve modernizeShader from @genosage in #9743:

A better way [than using replaceInSourceString/Regex] may be assigning an identifier and use it as the keyword to avoid calling replace functions each draw call.

// ShaderCache.js
  var vertexShaderText = vertexShaderSource.createCombinedVertexShader(
    this._context
  );
  var fragmentShaderText = fragmentShaderSource.createCombinedFragmentShader(
    this._context
  );

  var keyword =
    vertexShaderText + fragmentShaderText + JSON.stringify(attributeLocations);

[ptrgags]

When implementing EXT_mesh_features in CesiumJS, one WebGL 1 hurdle that impacts performance is the lack of gl_VertexID.

Implicit feature ID attributes can be computed by offset + floor(gl_VertexID / repeat). However, in WebGL 1, we have no choice but to do this computation on the CPU and then upload an extra attribute

[ptrgags]

Another WebGL 1 limitation that limits our implementation of EXT_mesh_features: In this table in the MDN docs, it seems that even a highp int is only required to have 17 bits. So for feature IDs, I'll have to use a highp float to squeeze out a couple more bits. Also there's no uint type. In a shader,uint32 would be more ideal, but only guaranteed in WebGL 2

[lilleyse]

CloudCollection and voxels would benefit from 3D textures

[lilleyse]

Point cloud properties (e.g. classification) may have precision artifacts in WebGL 1 if accessed in the frag shader. WebGL 2 may fix this with the flat qualifier. See CesiumGS/3d-tiles-samples#55 (comment) for more details.

[ggetz]

Given the work done in #10894, that should cover most of the actionable items in this issue. I'm going to close this and any specific items should be listed in their won issue, such as #649 and #630.