CubeTexture orientation

[fabienrohrer]

In my app, I would like to rotate the CubeTexture that defines the scene background and the environment map reflections by 180° along the vertical y-axis.
My rotation is defined by the VRML and X3D specifications about the background fields.

I tried to modify several parameters like the mapping, flipY and rotation Texture fields. However I didn't found any good way to do so.

• Is there some obvious way to do so that I missed?
• Any chance to see this coming in three.js?
• If I would like to implement it, what parameters would you like to see (for example Texture.flipX and CubeTexture.flipZ)? Would you be interested in this feature?

[mrdoob]

You mean being able to rotate scene.background? If so... yes, definitely interested.

We probably need a new API though...

scene.background = new THREE.Background( cubeTexture );`
scene.background.rotation.y = Math.PI / 2;

[looeee]

I'm struggling with exactly this right now. Trying to get baked ground shadows to match up to the sun direction from a cubemap.
It seems like somewhere along the way my cubemap is being flipped 180 degrees. It would be great if I could just fix that here rather than re-rendering my cubemap multiple times and trying to figure out where that's happening.

[WestLangley]

@fabienrohrer Try

scene.rotation.y = Math.PI;

If that is not working, please provide a live example to demonstrate exactly what you are doing.

[mrdoob]

Oh, that works? 😮 The camera must be a child of the scene though?

[WestLangley]

The camera must be a child of the scene though?

I don't think that matters.

[fabienrohrer]

@mrdoob this is exactly what I need too :-)

@WestLangley:

Rotating the overall scene is overkilled and may cause many side effects in my app.

I would like to be able to rotate the background only, and for sure, the environment maps should work the same way.

A live demo is not possible to do, because the API is missing. It's a matter to take any example having a background cubemap (like https://threejs.org/examples/#webgl_materials_cubemap) and be able to rotate only the background around the y-axis. Or more generally, to apply any rotation to the background.

Being able to apply a custom rotation matrix / quaternion to any cubemap seems to be the most generic solution for me.

[WestLangley]

three.js is careful to ensure that material reflections are consistent with the material envMap, which is defined in the world space coordinate system. Allowing users to rotate the scene background would result in a background that is inconsistent with material reflections.

Why don't you create the correct world-space cube map in your app?

[mrdoob]

The camera must be a child of the scene though?

I don't think that matters.

I tried doing scene.rotation.y ++ in the console in webgl_materials_standard and the gun rotates. Then I tried doing scene.add( camera ) and then if I do scene.rotation.y ++ it does what we're after, but then the camera controls get messes up 😁

If we introduce a THREE.Background object, we could add API to handle this, plus in the renderer we can also introduce the idea of:

var envMap = material.envMap;
if ( scene.background && scene.background.isBackground && envMap === null ) {
    envMap = scene.background.texture;
}

[WestLangley]

I said

Try scene.rotation.y = Math.PI; If that is not working,...

That means it may not work. :-)

It depends on the use-case. I tried with OrbitControls and it worked fine.

unrelated: the webgl_materials_standard example should be using orbit controls, anyway IMO.

[looeee]

You mean being able to rotate scene.background? If so... yes, definitely interested.

It makes more sense to do this at the texture level, otherwise any IBL lighting will not match the background, I assume.

We allow rotation of normal textures, is there any reason why we wouldn't want to do the same for cube textures? Is it too complex to implement?

Why don't you create the correct world-space cube map in your app?

This is very time consuming. I've been working on generating cube maps over the last couple of days, and making changes then re-rendering the cube map takes me about 15-20 minutes for each change.

Another potential interesting use case for this is that you could change the direction of environmental lighting in real time. I'm currently combining a PMREM environment map with a directional light for real-time shadows, and using the map as a skybox, which gives a fairly decent daylight effect.
Obviously there's a limit to what you could do without changing the color of the environment map, but you could simulate at least a couple of hours of daylight passing by rotating the map in sync with the directional light while adjusting the .envMapIntensity.

[fabienrohrer]

My concern is also about performance. Currently I implemented a hack about to rotate the top and bottom texture and swap the other ones. It takes about 10ms to rotate a 1024x1024 texture in JS.

[omichel]

Alternatively, could it be possible to have only two options for the CubeTexture orientation, the one you currently have and another one (with the 180 degree rotation) that would be friendly with the other standard largely used in 3D formats (X3D, VRML, etc.). We could probably easily adapt the consistency with material reflections to handle both formats?

[makc]

This sounds familiar #11103

[fabienrohrer]

If I would implement CubeMap.rotation = matrix3, would you merge in r105?

[WestLangley]

My concern is also about performance. Currently I implemented a hack about to rotate the top and bottom texture and swap the other ones. It takes about 10ms to rotate a 1024x1024 texture in JS.

And how often are you doing this in your app?

[fabienrohrer]

@WestLangley I just give you a detailed answer here: #16507 (comment)

And how often are you doing this in your app?

Not much. Once at loading per client, and once each time the background is changed (very rare case). A DEFINE would also do the job.

[WestLangley]

My concern is also about performance ... It takes about 10ms.

And how often are you doing this in your app?

Not much. Once at loading per client, and once each time the background is changed (very rare case).

Sorry, I am not following the logic of that.

[fabienrohrer]

Unfortunately, @WestLangley doesn't want to merge my modification about the CubeTexture.rotation because of the runtime overhead (a supplementary mat3 multiplication). In this case, I'm running out of ideas to solve this cleanly in threejs.
If threejs contributors won't solve this, please close this issue.

[VanderSP]

actually im using a hack lol, but my hack is pre fixed value, maybe if i know how to feed uniforms onBeforeCompile... maybe ask gpt, at least my hacked fragment rotates correctly by 3 axis, after so much attempts that used to distort before.

[PeteMatterfield]

This is quite an old, but fundamental, feature of rendering. It's been punted over the years to the next revision but some of us need the feature for various upcoming deliverables. Happy to put up some serious sponsorship cash, immediately, to fast track its implementation. Or, for someone to at least post the working shader code so we can implement it locally.

[VanderSP]

@PeteMatterfield paste this in any part of your js:
but this is without using pmrem... pmrem u need to change other file... anyway im not into pmrem... now i dunno if it was because in the past when i changed pmrem file, i was doing distorted one and i was thinkin was a pmrem problem.. maybe one day i test, but i think they will implement it soon :)

ShaderChunk['cube_uv_reflection_fragment'] = `
#ifdef ENVMAP_TYPE_CUBE_UV

	#define cubeUV_minMipLevel 4.0
	#define cubeUV_minTileSize 16.0

	// These shader functions convert between the UV coordinates of a single face of
	// a cubemap, the 0-5 integer index of a cube face, and the direction vector for
	// sampling a textureCube (not generally normalized ).
	float getFace( vec3 direction ) {
		vec3 absDirection = abs( direction );

		float face = - 1.0;

		if ( absDirection.x > absDirection.z ) {

			if ( absDirection.x > absDirection.y )

				face = direction.x > 0.0 ? 0.0 : 3.0;

			else

				face = direction.y > 0.0 ? 1.0 : 4.0;

		} else {

			if ( absDirection.z > absDirection.y )

				face = direction.z > 0.0 ? 2.0 : 5.0;

			else

				face = direction.y > 0.0 ? 1.0 : 4.0;

		}

			return face;

	}

	// RH coordinate system; PMREM face-indexing convention
	vec2 getUV( vec3 direction, float face ) {

		vec2 uv;

		if ( face == 0.0 ) {

			uv = vec2( direction.z, direction.y ) / abs( direction.x ); // pos x

		} else if ( face == 1.0 ) {

			uv = vec2( - direction.x, - direction.z ) / abs( direction.y ); // pos y

		} else if ( face == 2.0 ) {

			uv = vec2( - direction.x, direction.y ) / abs( direction.z ); // pos z

		} else if ( face == 3.0 ) {

			uv = vec2( - direction.z, direction.y ) / abs( direction.x ); // neg x

		} else if ( face == 4.0 ) {

			uv = vec2( - direction.x, direction.z ) / abs( direction.y ); // neg y

		} else {

			uv = vec2( direction.x, direction.y ) / abs( direction.z ); // neg z

		}

		return 0.5 * ( uv + 1.0 );

	}

	vec3 bilinearCubeUV( sampler2D envMap, vec3 direction, float mipInt ) {

		float face = getFace( direction );

		float filterInt = max( cubeUV_minMipLevel - mipInt, 0.0 );

		mipInt = max( mipInt, cubeUV_minMipLevel );

		float faceSize = exp2( mipInt );

		highp vec2 uv = getUV( direction, face ) * ( faceSize - 2.0 ) + 1.0; // #25071

		if ( face > 2.0 ) {

			uv.y += faceSize;

			face -= 3.0;

		}

		uv.x += face * faceSize;

		uv.x += filterInt * 3.0 * cubeUV_minTileSize;

		uv.y += 4.0 * ( exp2( CUBEUV_MAX_MIP ) - faceSize );

		uv.x *= CUBEUV_TEXEL_WIDTH;
		uv.y *= CUBEUV_TEXEL_HEIGHT;

		#ifdef texture2DGradEXT

			return texture2DGradEXT( envMap, uv, vec2( 0.0 ), vec2( 0.0 ) ).rgb; // disable anisotropic filtering

		#else

			return texture2D( envMap, uv ).rgb;

		#endif

	}

	// These defines must match with PMREMGenerator

	#define cubeUV_r0 1.0
	#define cubeUV_v0 0.339
	#define cubeUV_m0 - 2.0
	#define cubeUV_r1 0.8
	#define cubeUV_v1 0.276
	#define cubeUV_m1 - 1.0
	#define cubeUV_r4 0.4
	#define cubeUV_v4 0.046
	#define cubeUV_m4 2.0
	#define cubeUV_r5 0.305
	#define cubeUV_v5 0.016
	#define cubeUV_m5 3.0
	#define cubeUV_r6 0.21
	#define cubeUV_v6 0.0038
	#define cubeUV_m6 4.0

	float roughnessToMip( float roughness ) {

		float mip = 0.0;

		if ( roughness >= cubeUV_r1 ) {

			mip = ( cubeUV_r0 - roughness ) * ( cubeUV_m1 - cubeUV_m0 ) / ( cubeUV_r0 - cubeUV_r1 ) + cubeUV_m0;

		} else if ( roughness >= cubeUV_r4 ) {

			mip = ( cubeUV_r1 - roughness ) * ( cubeUV_m4 - cubeUV_m1 ) / ( cubeUV_r1 - cubeUV_r4 ) + cubeUV_m1;

		} else if ( roughness >= cubeUV_r5 ) {

			mip = ( cubeUV_r4 - roughness ) * ( cubeUV_m5 - cubeUV_m4 ) / ( cubeUV_r4 - cubeUV_r5 ) + cubeUV_m4;

		} else if ( roughness >= cubeUV_r6 ) {

			mip = ( cubeUV_r5 - roughness ) * ( cubeUV_m6 - cubeUV_m5 ) / ( cubeUV_r5 - cubeUV_r6 ) + cubeUV_m5;

		} else {

			mip = - 2.0 * log2( 1.16 * roughness ); // 1.16 = 1.79^0.25
		}

		return mip;

	}

	vec3 rotateVectorX(vec3 v, float angle) {
        float s = sin(angle);
        float c = cos(angle);
        return vec3(v.x, c * v.y - s * v.z, s * v.y + c * v.z);
    }

    vec3 rotateVectorY(vec3 v, float angle) {
        float s = sin(angle);
        float c = cos(angle);
        return vec3(c * v.x - s * v.z, v.y, s * v.x + c * v.z);
    }


    vec3 rotateVectorZ(vec3 v, float angle) {
        float s = sin(angle);
        float c = cos(angle);
        return vec3(c * v.x - s * v.y, s * v.x + c * v.y, v.z);
    }

	vec4 textureCubeUV( sampler2D envMap, vec3 sampleDir, float roughness ) {

		float mip = clamp( roughnessToMip( roughness ), cubeUV_m0, CUBEUV_MAX_MIP );

		float mipF = fract( mip );

		float mipInt = floor( mip );

    sampleDir = rotateVectorZ(sampleDir, -0.1);

    sampleDir = rotateVectorY(sampleDir, -0.8);

    sampleDir = rotateVectorX(sampleDir, -0.8);

		vec3 color0 = bilinearCubeUV( envMap, sampleDir, mipInt );

		if ( mipF == 0.0 ) {

			return vec4( color0, 1.0 );

		} else {

			vec3 color1 = bilinearCubeUV( envMap, sampleDir, mipInt + 1.0 );

			return vec4( mix( color0, color1, mipF ), 1.0 );

		}

	}

#endif
`

im hardcoding these:
sampleDir = rotateVectorZ(sampleDir, -0.1);
sampleDir = rotateVectorY(sampleDir, -0.8);
sampleDir = rotateVectorX(sampleDir, -0.8);

but i suppose it can feed via some uniforms in onbeforecompile, gpt said something about that, but i not tested, so can realtime update... because i think they will implement it soon...

[PeteMatterfield]

@PeteMatterfield paste this in any part of your js: but this is without using pmrem... pmrem u need to change other file... anyway im not into pmrem... now i dunno if it was because in the past when i changed pmrem file, i was doing distorted one and i was thinkin was a pmrem problem.. maybe one day i test, but i think they will implement it soon :)

Thanks @VanderSP but I'll wait for pmrem support as that's what is being used by default.

[Mugen87]

I've added an experimental Scene.backgroundRotation property to test a potential change with cube, equirectangular and cubeUV maps. You can use the following two links and enable rotation with the GUI:

https://rawcdn.githack.com/Mugen87/three.js/053bbee3597e55b88e1da5be44dd0b1e81b70cce/examples/webgl_materials_envmaps.html
https://rawcdn.githack.com/Mugen87/three.js/053bbee3597e55b88e1da5be44dd0b1e81b70cce/examples/webgl_tonemapping.html

webgl_tonemapping uses PMREM when the background blurriness is greater than 0.

@VanderSP @PeteMatterfield Does this look correct to you?

Like mentioned earlier in the discussion, it would be good to have this feature for environment maps with normal meshes (not just the background skybox) as well.

[VanderSP]

Looks like it rotates... but i neve use back just envmap reflections, my manual solution works, not tested with the pmrem file but probably its possible, as i tried before but with wrong rot calculations, now i just need to ask gpt how to feed uniform values on onbeforecompile so i can change in realtime, but yes actually im overriding the shader variable fragment

[PeteMatterfield]

@Mugen87 Yes, this looks good so far!

[joezappie]

@VanderSP, I'm trying to figure out rotating my scene.enviornment as I have a Z up scene. Does your code work for that? I tried to use it, but I get Fragment shader is not compiled. Never done shader stuff before, so I'm a bit confused. I tried to run renderer.compile(scene, camera) after creating them.

[VanderSP]

hi @jrj2211 no need to compile , just paste code in any part of js, it will override it

[joezappie]

Hmm, if I just paste it in I get the "Fragment shader is not compiled.". I am unfortunately stuck on an older version of Threejs so I'm guessing that's the reason.

[VanderSP]

@jrj2211 Maybe you get luck first copying the fragment from your actual version, then copying the relevant parts I added!

[joezappie]

Thanks for the suggestion! I'll give that a shot

[Mugen87]

@nickyvanurk I suggest you ask your question at the forum.

GitHub is no the right place for help requests. When you do so, demonstrate the issue with a live example. Without seeing your code, it is not possible to provide feedback.

Edit: Answer at the forum: https://discourse.threejs.org/t/cubetexture-docs-axis-confusing-wrong/61513/4