Unreal Bloom and Renderer Transparency issue

[fire67]

Description of the problem

I am currently using the Unreal Bloom post with a transparent renderer.

renderer = new THREE.WebGLRenderer({ alpha: true });
renderer.setClearColor(0xFF0000, 0);

Unfortunately, the Unreal Bloom pass doesn't seem to take in account the alpha channel of the coming texture when doing the blur and it apparently comes from getSeperableBlurMaterial in js\postprocessing\UnrealBloomPass.js.

The fragment's alpha channel is always set to 1.0 which results in a complete removal of the previous alpha values when doing the additive blending at the end.

This is happening in the latest version of Three.js r92 and you can experiment about this issue using the Unreal Bloom example

[WestLangley]

In getSeperableBlurMaterial()you can try a work-around:

gl_FragColor = vec4( diffuseSum / weightSum, 0.1 );\n\

Remember, it is an example, so you are free to modify it.

[fire67]

Thanks for the answer @WestLangley !
Yeah this would help keeping the transparency but would destroy the bloom effect (which is really cool btw) around the object. I think a more cleaner solution would be to apply weight on alpha channel too.

Any solution would help me a lot :)

[WestLangley]

/ping @spidersharma03

[fire67]

I achieved to find a solution for this but I would like to know your opinion @spidersharma03 and @WestLangley.

void main()
{
	vec2 invSize = 1.0 / texSize;
	float fSigma = float(SIGMA);
	float weightSum = gaussianPdf(0.0, fSigma);
	float alphaSum = 0.0;
	vec3 diffuseSum = texture2D(colorTexture, vUv).rgb * weightSum;
	for( int i = 1; i < KERNEL_RADIUS; i ++ )
	{
		float x = float(i);
		float weight = gaussianPdf(x, fSigma);
		vec2 uvOffset = direction * invSize * x;
		
		vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
		float weightAlpha = sample1.a * weight;
		diffuseSum += sample1.rgb * weightAlpha;
		alphaSum += weightAlpha;
		weightSum += weight;
		
		vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
		weightAlpha = sample2.a * weight;
		diffuseSum += sample2.rgb * weightAlpha;
		alphaSum += weightAlpha;
		weightSum += weight;
		
	}
	alphaSum /= weightSum;
	diffuseSum /= alphaSum; // Should apply discard here if alphaSum is 0
	gl_FragColor = vec4(diffuseSum.rgb, alphaSum);
}

[WestLangley]

Sorry, I have no opinion on this.

[spidersharma03]

Sorry @fire67 , was away for a few days. Yeah, what you have done seems correct. I didn't handle the transparency.

[fire67]

No problem, thank you for the answer @spidersharma03 !
While testing the effect I noticed a visual difference when pushing some values very high.

Do you have any idea where does this comes from ?

[spidersharma03]

@fire67 , Not sure actually. Do you have the changed code in some branch? I will try to take a look.
Thanks!

[fire67]

@spidersharma03 No, unfortunately the code is not available on a branch. Do you want me to do it ?
Just to let you know, there's nothing more than the main function in the previous post.

[jcyuan]

@fire67 @spidersharma03 hi, I tried to use your main() code to replace the old one in example but all bloom effect gone although the background takes effect it's transparent.

this.renderer = new THREE.WebGLRenderer({
        canvas: view,
        alpha:true
});

//and pass
this.$unrealBloom = new UnrealBloomPass(new THREE.Vector2(this.app.stageWidth, this.app.stageHeight), 3.2, .6, 0.7 );
this.$composer.addPass(this.$unrealBloom);

this.$blurPass = new ShaderPass(HorizontalBlurShader);
this.$blurPass.uniforms['h'].value = 0;
this.$composer.addPass(this.$blurPass);

this.$blurPass.renderToScreen = true;

please help, thanks.

[jcyuan]

ok so the problem is caused by my setting:
this.$renderer.context.enable(this.$renderer.context.SAMPLE_ALPHA_TO_COVERAGE);
the alpha channel of user RTT will not be handled by lowlevel for A2C in WebGL1.0 (it does in WebGl2.0)

the changed shader works well actually, thank you very much @fire67

[robhawkes]

@fire67 Did you manage to get this working with the bloom keeping the colour of the object? As soon as I try to use the changed shader the bloom just goes pure white, rather than taking the object colour as before.

I also tried disabling rendering.alpha but the bloom was still pure white.

Any ideas?

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float weight = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    float weightAlpha = sample1.a * weight;\
    diffuseSum += sample1.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    weightAlpha = sample2.a * weight;\
    diffuseSum += sample2.rgb * weightAlpha;\
    alphaSum += weightAlpha;\
    weightSum += weight;\
  }\
  alphaSum /= weightSum;\
  diffuseSum /= alphaSum;\
  gl_FragColor = vec4(diffuseSum.rgb, alphaSum);\n\
}

Before:

After:

[robhawkes]

I've been mulling this over for the past few days and decided to attempt a slightly different approach that takes the original (non-transparent) colour calculations, leaves them unmodified and then adds on top the new alpha calculations from @fire67 with a couple of minor tweaks to utilise the same weights as the colour calculations. I don't know if this is the right way to do it, and the calculations could probably be tidied up, but it works.

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    weightSum += 2.0 * w;\
    float weightAlpha = sample1.a * w;\
    alphaSum += weightAlpha;\
    weightAlpha = sample2.a * w;\
    alphaSum += weightAlpha;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

Previous approach (posted earlier in this issue):

New approach combining original logic with updated logic posted earlier in the issue:

[robhawkes]

Tidy things up a bit and it seems to work just as well:

void main() {\n\
  vec2 invSize = 1.0 / texSize;\
  float fSigma = float(SIGMA);\
  float weightSum = gaussianPdf(0.0, fSigma);\
  float alphaSum = 0.0;\
  vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;\
  for( int i = 1; i < KERNEL_RADIUS; i ++ ) {\
    float x = float(i);\
    float w = gaussianPdf(x, fSigma);\
    vec2 uvOffset = direction * invSize * x;\
    vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);\
    vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);\
    diffuseSum += (sample1.rgb + sample2.rgb) * w;\
    alphaSum += (sample1.a + sample2.a) * w;\
    weightSum += 2.0 * w;\
  }\
  gl_FragColor = vec4(diffuseSum/weightSum, alphaSum/weightSum);\n\
}

I've noticed some little (single pixel) artefacts that appear once in a while but seem to be random or based on something that only occurs on some frames as they disappear on the next frame. Not sure if it's caused by the changes I made or something unrelated to this issue.

[robhawkes]

I've done some more testing and can confirm that the artefacts that I was seeing are unrelated to Three.js and the changes that I made to the bloom shader, so that's good.

[mrdoob]

/ping @spidersharma03

[spidersharma03]

I will see the results with a colored scene and come back.

[spidersharma03]

@robhawkes , I see that you initialize the alphaSum to 0. Did you simply try making something like this::

vec4 diffuseSum = texture2D( colorTexture, vUv)* weightSum;\ ... ... diffuseSum /= weightSum;

[spidersharma03]

@robhawkes , basically something like this::

                                    float fSigma = float(SIGMA);
				float weightSum = gaussianPdf(0.0, fSigma);
				vec4 diffuseSum = texture2D( colorTexture, vUv) * weightSum;
				for( int i = 1; i < KERNEL_RADIUS; i ++ ) {
					float x = float(i);
					float w = gaussianPdf(x, fSigma);
					vec2 uvOffset = direction * invSize * x;
					vec4 sample1 = texture2D( colorTexture, vUv + uvOffset);
					vec4 sample2 = texture2D( colorTexture, vUv - uvOffset);
					diffuseSum += (sample1 + sample2) * w;
					weightSum += 2.0 * w;
				}
				gl_FragColor = vec4(diffuseSum/weightSum);

I need to test it though

[netgfx]

I've created a full example. Thanks to help from: #14104 (comment)

Disclaimer it's in typescript:

• Example
• Source Code

Any idea if this exists for r3f or is there another way to achieve transparent background and bloom effect in r3f nowadays?

[manmohanbishnoi]

I've created a full example. Thanks to help from: #14104 (comment)

Disclaimer it's in typescript:

• Example
• Source Code

Solution provided by @mbalex99 worked for me with r136

[DOSputin]

Typescript 🤢
Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

[mrdoob]

Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

Lovely! Could you share the JS version? 🤓

[DOSputin]

Reverted @mbalex99's implementation back to pure JS and it does work fine with alpha, also r136

Lovely! Could you share the JS version? 🤓

I'm sort of new here in the Three.js scene. How would you suggest might be the best way to share? As in, what's the preference?

[DOSputin]

@mrdoob; I uploaded my reimplementation (JS version) in this gist: UnrealBloomPass.js. Should these changes be integrated into the original UnrealBloomPass and do a #14104 branch PR?

[mrdoob]

A new PR would be great actually.
https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request

[VanderSP]

Okay, i´ve found what happens ->

the bloom alpha was finally fixed on 137.5
the bloom alpha fix was broken in 137.3

I´ve tested in 137.2 and 136.0 it worked (pasting the file on node_modules)

it´s funny when something breaks an old thing fix just before the fix release :D

Also, i wonder what setClearColor does? when i got it working with i described above or not, setClearColor does nothing always.

[Mugen87]

I've decided to having a look at this issue since the PR #23356 does not work for me. Here is my branch with the suggested change from above: Mugen87@551e7b5

UnrealBloomPass now supports transparent backgrounds however the bloom does not blend correctly with the HTML background image. The same artifacts appear like mentioned in #14104 (comment) (see the three screenshots at the end of the post).

You can easily verify this by opening the following URL and set the bloom strength and radius to maximum.

https://raw.githack.com/Mugen87/three.js/dev60/examples/webgl_postprocessing_unreal_bloom.html

Looking at the suggested fix, a simple enhancement in the fragment shader is not sufficient.

[luca-viggiani]

Hi, I've been using this tweak with 128 and worked for long time very well.
Today I switched to 142 and the tweak doesn't work anymore. I looked at comment above and saw that the issue should be fixed in 137.5 so I removed the tweak from my code but it still renders a black background with 142.
Here is my code:

const BLOOM_PARAMS = {
    resolution: new THREE.Vector2(1024, 1024),
    strength: 2,
    radius: .65,
    threshold: .85
}

const CAMERA_PARAMS = {
    fov: 50,
    pixelRatio: window.devicePixelRatio,
    nearClippingPlane: .1,
    farClippingPlane: 10000
}

this.#scene = new THREE.Scene();
this.#renderer = new THREE.WebGLRenderer({
    antialias: true,
    alpha: true,
    format: THREE.RGBAFormat,
    logarithmicDepthBuffer: true
});

this.#renderer.shadowMap.type = THREE.PCFSoftShadowMap;
this.#renderer.outputEncoding = THREE.sRGBEncoding;
this.#renderer.setPixelRatio(window.devicePixelRatio);
this.#renderer.setClearColor(0x000000, 0.0);

this.#shadowRoot.appendChild(this.#renderer.domElement);

this.#camera = new THREE.PerspectiveCamera(...Object.values(CAMERA_PARAMS));

this.#passes = {
    render: new RenderPass(this.#scene, this.#camera),
    bloom: new UnrealBloomPass(...Object.values(BLOOM_PARAMS))
}

this.#effectComposer = new EffectComposer(this.#renderer);
Object.values(this.#passes).forEach(pass => this.#effectComposer.addPass(pass));

Where I do wrong? Thanks!

[Mugen87]

saw that the issue should be fixed in 137.5

The issue is actually not fixed. Sorry for the confusion in this thread.

Besides, I don't think the issue can be properly fixed since there is no way to correctly additively blend the bloom with a HTML background.

I think we should close the issue and state that UnrealBloomPass requires a non-transparent scene background.

[lviggiani]

saw that the issue should be fixed in 137.5

The issue is actually not fixed. Sorry for the confusion in this issue.

Besides, I don't think the issue can be properly fixed since there is no way to correctly additively blend the bloom with a HTML background.

I think we should close the issue and state that UnrealBloomPass requires a non-transparent scene background.

Thanks for the feedback! With 128 + the tweak it actually works and I get transparent background as desired. Unfortunately it stop working with more recent releases. Mixing 142 and 128 for the UnrealBloomPass maybe is not a good idea but I'll give it a try

[lviggiani]

Ok, the mix doesn't work. I'll stick to 128 + frag shader for now

[CountAsWun]

Ah i was so hopeful when i saw that this was fixed but maybe we are back at square one. I don't know much about modifying this library and the underlying WebGL tech but i've spent 10 hours trying to get background transparency working as it is essential to my use case.

What changed from the previous versions that now break background transparency?

What is it about RenderPass that handles background transparency with ease that breaks in UnrealBloom?

I don't have the background to fully understand how to implement this, but the solution here seems simple as this allows bloom to work with WebGL, I just need to understand how to implement it with threejs.UnrealBloom

https://stackoverflow.com/questions/53422916/how-to-draw-transparent-background-in-webgl-on-canvas

[johnnyshankman]

+1 Why haven't we fixed this? What is the final solution? @mrdoob let's get something merged we have plenty of PRs and Gists in this thread but nothing someone can just import and use right now.

[johnnyshankman]

Also what is up with the breaking changes from 127 to 128? Codepen: https://codepen.io/johnnyshankman/pen/MWGMqYZ I can't seem to figure out how to upgrade to 128 successfully.

[donmccurdy]

This remains open because no working, stable fix has been found. You might try https://github.com/pmndrs/postprocessing/, it may support bloom + canvas transparency.

I'd recommend using the forums (https://discourse.threejs.org/) for help with the upgrade to r128.

[Mugen87]

Voting again for closing this issue since I don't think it can be fixed, see #14104 (comment).

[xiaohaook]

//fragmentShader
 void main() {
				vec2 invSize = 1.0 / texSize;
				float fSigma = float(SIGMA);
				float alphaNum = 0.0; 
				float weightSum = gaussianPdf(0.0, fSigma);
				vec3 diffuseSum = texture2D( colorTexture, vUv).rgb * weightSum;
				for( int i = 1; i < KERNEL_RADIUS; i ++ ) {
					float x = float(i);
					float w = gaussianPdf(x, fSigma);
					vec2 uvOffset = direction * invSize * x;
					vec3 sample1 = texture2D( colorTexture, vUv + uvOffset).rgb;
					vec3 sample2 = texture2D( colorTexture, vUv - uvOffset).rgb;
					diffuseSum += (sample1 + sample2) * w;
					weightSum += 2.0 * w;
					alphaNum += (texture2D( colorTexture, vUv + uvOffset).a + texture2D( colorTexture, vUv - uvOffset).a) * w;
				}
	             
				gl_FragColor = vec4(diffuseSum/weightSum, alphaNum/weightSum);
			}


//212 line
render( renderer, writeBuffer, readBuffer, deltaTime, maskActive ) {

	renderer.getClearColor( this._oldClearColor );
	this.oldClearAlpha = renderer.getClearAlpha();
	const oldAutoClear = renderer.autoClear;
	renderer.autoClear = false;

	renderer.setClearColor( this.clearColor, 0 );

	if ( maskActive ) renderer.state.buffers.stencil.setTest( false );



	if ( this.renderToScreen ) {

		this.fsQuad.material = this.basic;
		this.basic.map = readBuffer.texture;

		renderer.setRenderTarget( null );
		//Todo
		renderer.clear(false,true,true);
		//this.fsQuad.render( renderer );

	}

The three version is 0.146.0
This is the effect before I changed UnrealBloomPass.js. The background is opaque. To make the background transparent, I modified the code of the slice source shader in UnrealBloomPass.js, but it cannot make my scene transparent. So I changed renderer. clear() to renderer clear(false,true,true); And commented out this.fsQuad render( renderer );
The display is as follows. The background is transparent, but the display is very false. I need official help

[ButzYung]

@Mugen87 Recently I tried to implement UnrealBloom in my app but encountered the same transparency problem. I came across this thread, seeing all attempts to modify the blur pass in the middle but none seemed to yield favorable results. After trying some experiments by myself, I suggest a new approach which basically doesn't need to modify UnrealBloom.js.

I use the selective bloom example as a base. In the fragment shader of the final pass where the bloom texture and the screen texture is merged,

gl_FragColor = ( texture2D( baseTexture, vUv ) + vec4( 1.0 ) * texture2D( bloomTexture, vUv ) );

which is practically a simple additive operation, I replace it with the following.

vec4 base_color = texture2D(baseTexture, vUv);
vec4 bloom_color = texture2D(bloomTexture, vUv);

float lum = 0.21 * bloom_color.r + 0.71 * bloom_color.g + 0.07 * bloom_color.b;
gl_FragColor = vec4(base_color.rgb + bloom_color.rgb, max(base_color.a, lum));

The concept is fairly simple. I generate an alpha value for the final bloom texture by using its luminosity as a base (it can be something else other than luminosity, which is open for discussion). The final result is the original base_color + bloom_color as color, plus the max of base_color.a and the new bloom alpha as the final alpha. For fully opaque parts, the result is completely the same as the default approach.

The actual result is fairly decent, at least according to my testings. Below are some screenshots using the same bloom parameters. As you can see, the default one with no transparency (i.e. black background) looks almost identical with the new approach using a HTML black background.

The new approach also looks fairly decent on a pure white background, as well as a complex background.

Of course, this is not a perfect solution. There is no perfect solution when there is no way to actually mix WebGL with HTML background with additive blending. At least, this is a fairly reasonable solution for those who really needs a transparent background.

[simongloor]

Wow, the result is amazing, congratz!