Feature Request: 180 degree 3d camera presets for VR [$100 awarded]

[Deadwing888]

Currently the environment camera does a good job of generating panoramic scenes in 360 degree view for VR. With a camera offset of a ~65mm interpupillary distance, two panoramic renders can be stitched together top-bottom style in any given image editing program to generate convincing 360 degree 3D in VR. It would be nice if there was a setting to automatically bake this into one 1:1 aspect ratio image.

What's really missing at the moment is a 180 degree 3D VR preset, or at least some way to input numbers to be able to get one. I'm relatively new to VR rendering, but I believe the image would be a 2:1 aspect ratio image with the left eye's 180 degrees of vision on the left half of the image and the right eye's 180 degrees of vision on the right half of the image. I believe this should have the barrel distortion post production flagged to enable by default.

After skimming the source code, I notice there is a "stereo" camera type that blendluxcore doesn't know about. This seems to have a framework for 180 degree VR already in it, though the default field of view is too small and seems to be unaffected by the Field of View section of the scene binary.

I'm putting a $100 bounty on any workable implementation of a 180 degree VR preset at the scene binary level. Blender integration and configurable interpupillary distance would be a nice bonus. If this is already possible with the current tools I'm open to advice.

These links may provide useful reading for 3D resolutions and standards:

https://creator.oculus.com/blog/encoding-high-resolution-360-and-180-video-for-oculus-go/?locale=en_US
https://developer.oculus.com/documentation/native/pc/dg-render/?locale=en_US

---

The $100 bounty on this issue has been claimed at Bountysource.

[Dade916]

Currently the environment camera does a good job of generating panoramic scenes in 360 degree view for VR. With a camera offset of a ~65mm interpupillary distance, two panoramic renders can be stitched together top-bottom style in any given image editing program to generate convincing 360 degree 3D in VR. It would be nice if there was a setting to automatically bake this into one 1:1 aspect ratio image.

This should be easy to add.

What's really missing at the moment is a 180 degree 3D VR preset, or at least some way to input numbers to be able to get one. I'm relatively new to VR rendering, but I believe the image would be a 2:1 aspect ratio image with the left eye's 180 degrees of vision on the left half of the image and the right eye's 180 degrees of vision on the right half of the image. I believe this should have the barrel distortion post production flagged to enable by default.

However, here, you talk about 180 degree while, in previous statement, you were talking of 360 degree views. Which one you are looking for ? Both ?

You are also adding the request of barrel distortion support. This can be quite tricky. Does Oculus provide a post-processing tool to apply barrel distortion to any image/video ? It sounds like something they should offer (and it would avoid the work to directly support barrel distortion).

[Deadwing888]

I'm really looking for the 180 degree. I was just mentioning how close the 360 degree panoramic is to already facilitating it. I believe rendering a 180 degree view may be as simple as slicing the middle square of pixels in a 2:1 panoramic.

There's already a bool for barrel distortion in the code and when flagged it operates as expected. I think I was incorrect about the 180 degree barrel distortion being needed on the 180 degree renderings though. Upon further research it seems that it is applied at the headset level. I've crossed that sentence out in the initial comment.

[Dade916]

Ok, so it is something like the current stereo camera but with 2 x 180 degree panoramic views. I'm not sure when I will have the time to work on it but it should be quite easy to add and I should add it soon.

[Deadwing888]

Terrific. I have an oculus quest headset to test on whenever it makes it in.

[Deadwing888]

Did some 180 degree renderings yesterday using the panoramic camera by just letting half the rendered pixels hit the floor. The middle square of a 2:1 aspect ratio panorama view is the correct behavior for a 180 degree view. Offset by 65mm with no change to yaw/pitch/roll achieves a good stereoscopic depth effect. Nothing really new here, just confirming things work as we thought they did.

[Dade916]

I added the support for 360 degree view to stero camera.

It requires only to add to the scene a new camera property:

scene.camera.stereo.type = environment

Or (the default for normal 2x perspective cameras):

scene.camera.stereo.type = perspective

This is a rendering with perspective stereo camera (like 2x perspective cameras):

And this is the same scene with 360 degree camera (like 2x environment cameras):

I can add the support for 180 degree cameras too if of any use.

[Deadwing888]

For 180 degree VR it's two 1:1 aspect panoramas with the left eye on the left and the right eye on the right for 2:1 image aspect ratio total.

For 360 degree images two 2:1 panoramas are stacked vertically with the left eye on the top and the right eye on the bottom for 1:1 image aspect ratio total.

If I'm interpreting what I'm seeing correctly, what you've done will work fine if the cameras are changed to be 180 instead of 360. This is the format that is important to me personally.

A 360 degree version would be a logical option as well, since both formats are used; however the frames will need to stack vertically as described earlier.

[Dade916]

I added the supported for 180 (horizontal stacked) and 360 degree (vertical stacked) stereo camera.

With "scene.camera.stereo.type = perspective":

With "scene.camera.stereo.type = environment_180":

And with "scene.camera.stereo.type = environment_360":

[Deadwing888]

At a glance the 180 looks correct, I'll have a chance to look in an actual headset later today.

For the 360, at a glance, the aspect ratio of the eyes seems a bit wide. After the two 2:1 aspect ratio images are stacked the final image resolution should be square at 1:1.

[Dade916]

The image resolutions are the one of the original (no stereo image): 1280 x 800

If you render at the correct resolution, you should get the correct output (800x800):

[Deadwing888]

Cool! Last couple days have been hectic but I'll be trying this on a headset soon. At a glance things are looking good.

[Deadwing888]

I got the latest version of Luxcore 2.5 alpha compiled today. Using blender to export the scene, everything works fine if I export a scene binary; however if I export the scene as text, luxcoreUI no longer sees my largest scene object (4.9 meg of geometry). It reports .3k tris instead of the 49.9k tris it reports when rendering the .bcf

Because blender does not yet know about this new camera type, I am unable to render my test scene to look at in VR.

Any thoughts on why a blender text export and binary export would not produce identical behavior in luxcoreui?

Edit: Luxcore 2.4 (release) throws error:
bad lexical cast: source type value could not be interpreted as target.

Edit 2: A version of 2.4 where I manually merged in the commit that addresses the bad lexical cast that blender tries to make results in the bug manifesting in the text but not binary version of the scene export just as 2.5 alpha does. Is this worthy of being its own issue in this github? Would the blendluxcore github be a better place for it?

[Dade916]

Can you post a test .blend scene showing the problem and the exported scene text files ? So I can check them here.

[Deadwing888]

Certainly. Here is the blender scene in question as well as a zip file of a .bcf and text export of the scene. If I had to take a wild guess, it's not interpreting the clip end value that excludes far away objects correctly.

Scene VR camera test2_LuxCore.zip

Scene VR camera test2.zip

[Dade916]

It is still a localization related problem, if you open the mesh-00002.ply file, you will see:

comment Created by LuxRays v2.4
element vertex 148,426
property float x
property float y
property float z
property float nx
property float ny
property float nz
element face 49,506
property list uchar uint vertex_indices
end_header

Notice the "," inside "148,426" and "49,506". Notice also the "LuxRays v2.4" in the first line. It think you have not correctly updated your BlendLuxCore installation.

If I load your .blend and export the scene, I get:

comment Created by LuxRays v2.5beta1
element vertex 148426
property float x
property float y
property float z
property float nx
property float ny
property float nz
element face 49506
property list uchar uint vertex_indices
end_header

and it works fine.

Try to update your BlendLuxCore installation (may be, delete the BlendLuxCore directory first so you are sure to do a clean installation).

[Deadwing888]

I figured a scene binary would be forward compatible. Will hop on blendluxcore 2.5 later and see what happens. Thanks!

[Deadwing888]

I had a bunch of stuff come up and still haven't had a chance to look at this. You can go ahead and collect the bounty it looks accurate enough in the attached screenshots. Thanks for your work!

[Dade916]

Ok, closing this.