Single-channel float data texture with WebGL2 context

[flekschas]

Is it possible to create float data textures with PIXI v5? If so, how would I have to configure the BaseTexture?

I've tried the following but it doesn't work:

const dataArray = new Float32Array([
  1.0, 0.1, 0.1,
  0.1, 0.75, 0.1,
  0.1, 0.1, 0.5,
]);

const createDataTexture = (data, width, height) => {
  const resource = new PIXI.resources.BufferResource(dataArray, { width, height });
  return new PIXI.BaseTexture(resource, {
    format: 0x822e, // gl.R32F 
    type: PIXI.FLOAT
  })
}

const dataTexture = createDataTexture(dataArray, 3, 3);

E.g., in this demo nothing is rendered, i.e., I assume the texture didn't get loaded: https://jsfiddle.net/flek/pct2qugr/136/

However, if I blow-up the data and create a 4-channel float texture it works:

const array = new Float32Array([
  1.0, 0.1, 0.1, 1.0,  0.1, 0.1, 0.1, 1.0,  0.1, 0.1, 0.1, 1.0,
  0.1, 0.1, 0.1, 1.0,  0.75, 0.1, 0.1, 1.0,  0.1, 0.1, 0.1, 1.0,
  0.1, 0.1, 0.1, 1.0,  0.1, 0.1, 0.1, 1.0,  0.5, 0.1, 0.1, 1.0,
]);

const texture = PIXI.Texture.fromBuffer(array, 3, 3);

Notice how the first channel is identical.

Here's the updated demo, which works as expected: https://jsfiddle.net/flek/pct2qugr/137/

• pixi.js version: 5.1.6
• Browser & Version: Chrome 80
• OS & Version: macOS 10.14.6
• Running Example: https://jsfiddle.net/flek/pct2qugr/126/

[ivanpopelyshev]

Hello!

In WebGL2 there are cases when internalFormat is not the same as format.

BaseTexture is context-independent, it stores format.

TextureSystem creates a GLTexture for it, and determines internalFormat

Resource does the texImage2D call using baseTexture format and glTexture internalFormat.

Of course we, pixijs core devs, dont know all the ways people want to use texImage2D params, we just typed general ones: thus that exist both in WebGL and WebGL2. This is our magic:
https://github.com/pixijs/pixi.js/blob/67ff50884ba0b8c42a1011598e2319ab3039cd1e/packages/core/src/textures/TextureSystem.ts#L252

We really do not want to put all WebGL2-specific formats. Say hello to line 1454 of 3309 in one of playcanvas files: https://github.com/playcanvas/engine/blob/64410ef87d2d641019b5609f724118a89d7052b6/src/graphics/device.js#L1456-L1461

We, PixiJS core team, understand that we cannot possibly predict all the ways people want to use texImage2D and its params. So, we allow to write your own logic for it!

BufferResource: https://github.com/pixijs/pixi.js/blob/67ff50884ba0b8c42a1011598e2319ab3039cd1e/packages/core/src/textures/resources/BufferResource.ts#L17

How to make your own resource:
https://pixijs.io/examples/#/textures/gradient-resource.js

More info can be found at https://www.html5gamedevs.com/topic/44689-how-bind-webgl-texture-current-to-shader-on-piximesh - that's the latest thread about texture resources.

==============
SUMMARY:

Just CTRL+C BufferResource, remove static methods, set internalFormat for glTexture there and use it for texImage2D.

After all, single line will create a texture:

new PIXI.BaseTexture(new MyAwesomeResource(data))

[flekschas]

Thanks a lot for the quick reply and detailed response!

I ended up extending BufferResource and adjusted the format and internalFormat in upload(), which did the trick!

This is what I ended up doing:

class CustomBufferResource extends PIXI.resources.Resource {
  constructor(source, options) {
    const { width, height, internalFormat, format, type } = options || {};

    if (!width || !height || !internalFormat || !format || !type) {
      throw new Error(
        'CustomBufferResource width, height, internalFormat, format, or type invalid'
      );
    }

    super(width, height);

    this.data = source;
    this.internalFormat = internalFormat;
    this.format = format;
    this.type = type;
  }
    
  upload(renderer, baseTexture, glTexture) {
    const gl = renderer.gl;

    gl.pixelStorei(
      gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL,
      baseTexture.alphaMode === 1 // PIXI.ALPHA_MODES.UNPACK but `PIXI.ALPHA_MODES` are not exported
    );

    glTexture.width = baseTexture.width;
    glTexture.height = baseTexture.height;

    gl.texImage2D(
      baseTexture.target,
      0,
      gl[this.internalFormat],
      baseTexture.width,
      baseTexture.height,
      0,
      gl[this.format],
      gl[this.type],
      this.data
    );

    return true;
  }
}

const resource = new CustomBufferResource(dataArray, {
  width: 3,
  height: 3,
  internalFormat: 'R32F',
  format: 'RED',
  type: 'FLOAT'
});

Here's the updated demo: https://jsfiddle.net/flek/pct2qugr/175/

I wonder if you could expose internalFormat as that seems to be the only limitation of BufferResource.

[ivanpopelyshev]

For now, I added it here: https://github.com/pixijs/pixi.js/wiki/v5-Hacks#texture-formats

Yes, exposing internalFormat might be a good idea when we get more people who need this :)

[ivanpopelyshev]

Clarification, why cant we just add this thing without questions:

1. Its WebGL2, not for real production. In case of production it needs a fallback - more IF's and re-structuring the array.
2. In case its not production but a demo or academic purposes - its better to leave code in the demo for others to understand that this low-level is actually part of the demo. Thus, people will learn WebGL itself and not pixijs magic.

[blakjak44]

Hi all,

I'm very new at this but I'm trying to use this same strategy to render 16 bit unsigned integer data texture. However, it doesn't look like Pixi supports the "usampler2d" uniform. Is there a way to support this uniform type so that I don't need to convert to 32 bit float? If not, are there plans to add support?