Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

re_renderer: implement depth cloud renderer #1415

Merged
merged 21 commits into from
Mar 2, 2023
Merged

Conversation

teh-cmc
Copy link
Member

@teh-cmc teh-cmc commented Feb 27, 2023

This implements a new Renderer within re_renderer that takes a depth texture and a set of intrinsics as input, and draws a point cloud as output.
It also adds a new example which compares doing backprojection into a point cloud manually vs. using the new depth-cloud renderer.

From a rendering standpoint, this work very similarly to the existing point renderer, except that the vertex shader backprojects a depth texture rather than reading from a position/color texture.
The depth texture is uploaded as-in and just-in-time, no caching involved.
Alpha and shading are both turned off.

This PR does not include a bunch of things on purpose as it's already way too big as-is (though most of it is boilerplate).
All of these will come in (hopefully much smaller) follow up PRs:

  • integration with main Rerun app
  • support for picking which primitive to render (image vs. point cloud)
  • support for (gpu) color maps
  • support for albedo textures

Picking is ignored altogether, until we have a generalized solution for that.


23-02-27_11.05.43.patched.mp4
23-03-01_22.21.49.patched.mp4

@teh-cmc teh-cmc added the 🔺 re_renderer affects re_renderer itself label Feb 27, 2023
@teh-cmc teh-cmc marked this pull request as ready for review February 27, 2023 10:08
@teh-cmc
Copy link
Member Author

teh-cmc commented Feb 27, 2023

This doesn't show anything on the web @Wumpf (see screenshot) which I'm sure is yet another case of WebGL alignment shenanigans... If you've got an idea I'd love to hear it 👀

image

@Wumpf
Copy link
Member

Wumpf commented Feb 27, 2023

alignment issues are very unlikely at this point for web-only problems. Also when I had issues with that on the web it was a crash. Anything useful in the log? Will have a closer look myself later!

@Wumpf
Copy link
Member

Wumpf commented Feb 27, 2023

this checks in very large binary files, we need to generate some synthetic data instead

@emilk
Copy link
Member

emilk commented Feb 27, 2023

this checks in very large binary files, we need to generate some synthetic data instead

Yeah, let's avoid checking in binary blobs 😬

@teh-cmc
Copy link
Member Author

teh-cmc commented Feb 27, 2023

I'll make a procedural one.

@Wumpf Wumpf self-requested a review February 27, 2023 13:03
@Wumpf
Copy link
Member

Wumpf commented Feb 27, 2023

Anything useful in the log?

No, because it doesn't yet have #1385 which reduces a lot of log spam 😅

Copy link
Member

@Wumpf Wumpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice start!

I'm a bit worried about the amount of boiler plate needed in some places, but can (should) take that separately.

Main issue is about Mat3 as already pointed out on Slack.
Other issues are mostly about naming and suggestion for code-reuse.

Best candidate for WebGL issue is the use of D16 sampling which I found somewhat exotic (but after getting past my first "shock" is totally watertight with the spec and the correct way to go about it as I believe now!).

crates/re_renderer/src/renderer/depth_cloud.rs Outdated Show resolved Hide resolved
crates/re_renderer/examples/depth_cloud.rs Show resolved Hide resolved
crates/re_renderer/shader/utils/quad.wgsl Outdated Show resolved Hide resolved
crates/re_renderer/shader/utils/camera.wgsl Show resolved Hide resolved
crates/re_renderer/shader/depth_cloud.wgsl Outdated Show resolved Hide resolved
crates/re_renderer/src/wgpu_buffer_types.rs Outdated Show resolved Hide resolved
crates/re_renderer/src/renderer/depth_cloud.rs Outdated Show resolved Hide resolved
crates/re_renderer/examples/depth_cloud.rs Outdated Show resolved Hide resolved
@teh-cmc
Copy link
Member Author

teh-cmc commented Feb 28, 2023

See also gfx-rs/wgpu#3537

@teh-cmc teh-cmc requested a review from Wumpf March 1, 2023 21:23
@teh-cmc
Copy link
Member Author

teh-cmc commented Mar 1, 2023

Should be good to go!

Copy link
Member

@Wumpf Wumpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's do this! :)

@teh-cmc teh-cmc merged commit 73187e0 into main Mar 2, 2023
@teh-cmc teh-cmc deleted the cmc/renderer/depth_clouds branch March 2, 2023 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🔺 re_renderer affects re_renderer itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants