How to calculate focus area for DoF in Bokeh effect

[gigablox]

Have an interesting question for you about calculating depth of field.

Let's compare 2 scenes.

1. A camera is looking at a ball that has a 1m radius, 10m away.

2. A camera is looking at a ball that has a 10m radius, 20m away.

Is there a generic equation I can use to figure out how to focus on each ball in these scenes instead of tweaking parameters until it "looks good".

I understand that there are some creative preferences that go into deciding what a "fit" is.

I understand that focus can have different levels of drop off and intensity. But how do I use an equation to parameterize these things with more intuition?

There is a common use case im running into for this transitioning between different objects and characters in a scene. It's becoming very hard to adjust settings for each one of these camera transitions without a more dynamic solution.

It also makes the current solution very brittle to change - like say I want to move one of those objects to a different location, suddenly my parameters no longer keep the object in view.

Thinking out loud, maybe some sort of generic helper class to get a thing in focus, then high level parameters control the type of focus and creative aspects? Would be massively helpful.

[vanruesc]

Is there a generic equation I can use to figure out how to focus on each ball in these scenes

Yes, with RealisticBokehEffect it could probably be done with this code: https://github.com/mrdoob/three.js/blob/master/examples/webgl_postprocessing_dof2.html#L440-L463

Instead of bokeh_uniforms[ 'focalDepth' ].value = ldistance; you'd adjust bokehEffect.uniforms.get("focus").value = ldistance. The focalDepth calculations are the same in this library, so it should work the same way.

Using logarithmic depth might also affect the calculations, but I don't know the specifics.

maybe some sort of generic helper class to get a thing in focus [would] be massively helpful

I agree; that would be nice to have. I'm currently busy with other projects, but I'll add this to the list of things to add for version 7.

[vanruesc]

I tried out the code from the DoF2 example myself and couldn't get it to work. The focus didn't behave as expected, so I went over the shader code again and found a bug that was messing with the focal depth. This bug went unnoticed for so long because it only breaks things with certain parameter configurations.

Anyway, a fix is on the way.

There are also some other differences that I'm looking into. For instance, RealisticBokehEffect uses a simple depth texture while the DoF2 example uses a custom depth shader. The functions linearize, saturate and smoothstep that are used to calculate the focal depth in the example seem to be necessary only because of the custom depth values.

I'll report back when I find out more.

[vanruesc]

I think it would be best to scrap and salvage both the BokehEffect and the RealisticBokehEffect to make one good bokeh effect.

Having two effects that try to do the same thing is bad to begin with, but there's more issues with the existing bokeh effects that should be addressed.

The abrupt transition from blurred foreground to sharp background looks bad:

The depth buffer contains aliasing artifacts that show up in the final image:

The performance of this effect largely depends on how the blurring is done. Both existing effects perform the blur directly in the main shader on the full resolution image. This could easily be improved by pre-blurring to a down-scaled render target with an option to use fast blur, disc-based blur or pentagonal blur.

And lastly, the effect should allow the user to easily and reliably focus specific objects, even with logarithmic depth. Well, that's the plan so far.

[vanruesc]

Update: last week I figured out how to use linear depth with SSAO:

In this screenshot you can see how the effect is rendered up to a distance of 35 world units. The camera near plane is set to 1 and the far plane is set to 1000. The camera is at [0, 0, 35] and looks at the center of the cornell box at [0, 0, 0]. The SSAO distance cutoff is set to approx. 0.035035035 which corresponds to the near and far plane settings.

While the SSAOEffect worked well before, it didn't linearize depth which made it harder to tweak the distance and proximity cutoffs with a normal depth buffer. This wasn't much of a problem with logarithmic depth, but I'm now trying to move away from using a logarithmic depth buffer to treat it more like a special case instead of the other way around.

I'll start working on the BokehEffect when I have more time. I think I'll use https://pixelmischiefblog.wordpress.com/2016/11/25/bokeh-depth-of-field/ as a reference to improve the effect. If anyone has any tips or inputs on the subject, feel free to drop them here.

[gigablox]

I think it would be best to scrap and salvage both the BokehEffect and the RealisticBokehEffect to make one good bokeh effect. - When using these two in THREE I always felt this way too.

I took a read through the article you linked, really cool resource. Guy seems to know what he's talking about. Going to reach out to him see if he's interested in joining the conversation here.

[gigablox]

He said he'd be willing to help with implementation specific questions as they come up at a higher level - algorithms and strategy - but does not know JS.

[vanruesc]

Alright, that's good to know.

I haven't had a chance to work on this yet, but I should be able to start soon. Since we're going to replace two effects, this change will be included in the next major release. I still need to look into WebGL 2 support and color space output options for things like proper gamma correction before I can release that one, though.

[vanruesc]

Sorry for the delay; I'm finally able to start working on Bokeh 😅

[gigablox]

Very excited for this!

[vanruesc]

Quick update: I decided to get color space support out of the way first since that affects pretty much everything. That took a good amount of time and I also caught a cold a while ago which delayed things further.

I'm pretty happy with the color space encoding now and also got some first results with the new depth of field effect:

That's the circle of confusion texture stored with 8bits. I'll have to experiment a bit with this to see if we can get away with that or if we do need a half float buffer after all.

This week, I need to focus on other things again. Hopefully, I'll finish the effect next weekend.

[gigablox]

Ohhh WOW! This is SO COOL! Very excited to study your shader, it's so impressive you've implemented this from scratch. Did you use the strategy linked in the article you referenced? Is the black essentially whats in focus? Is focus dynamically calculated by passing in a Z vector?

[vanruesc]

Thanks for the encouraging words 😊

Yes, I'm relying on the article for the most part. In the above image, positive CoC values are green, negative values are red and, as you guessed, black represents the sharp area around the focal plane.

The focus distance is a normalized value in the range [0.0, 1.0] that determines the distance of the focal plane to the camera. This parameter can either be adjusted manually or with the help of a method that calculates the focus distance to a given object. Setting the focus distance to 0 would move the black focus area to the camera near plane while setting it to 1 would place it right at the far plane. Every value in between behaves in a linear fashion.