Compositing
Compositing is the big magic. It's the part of an OnAirTap setup that brings everything together into a coherent, believable image.
There are two pathways available for compositing: Shaders and Manual.
Each of those is further divided into three methods based on which textures are used.
There's three different ways to do compositing, based on if you're using just the Optimised texture, the Foreground and Background textures, or all three.
This is the setup you'll probably be using. An extra camera is used to render the scene two additional times. The foreground texture can then be used as a premultiplied image, so that emissive (glowing) objects and fog will correctly appear in front of the model.
There are some extra steps needed to limit the influence of the foreground, so that areas of the background not covered by your model don't experience doubled glow. That can be done either with an Advanced Mask filter or some adjustments to blend modes.
Uses Plugin, Uses Groups
To composite this config:
Uses Plugin, No Groups
To composite this config:
No Plugin, No Groups
To composite this config:
If there are no emissive objects between you and the camera, this is the way to go. The Optimised texture has the RGB channels of a background texture, but with the Alpha channel of a foreground texture. It only adds one render, and is therefore quite efficient.
The OBS layout for compositing Optimised configs is very simple.
That's it. You don't even need any filters.
This config is not very efficient. You're rendering the active scene three extra times. Avoid, if you can.
In Beat Saber, the alpha channel is overwritten during post-processing. For reasons unknown, Beat Saber is missing the code needed to save and re-apply the original alpha channel at the end of the frame, so our Foreground render does not not produce a usable premultiplied texture. Luckily, the Optimised texture still works as intended, and can be used to fix the problem.
To composite this config:
You only need a single filter here. The Foreground source must be masked to the Alpha channel of the VNyan source, so that foreground objects don't have double-strength glow.
My settings for that mask are this:
For some versions of some games, just the setup above is still not enough. Instead of a Straight Optimised source, you need an Opaque Foreground source masked to the alpha of a Straight Optimised source.
If you don't know what premultiplication is, watch this video. It's about compositing in general, but does explain premultiplication, as part of that.
OBS seems to prefer straight alpha. OBS' plugin API gives direct enough control for a plugin source to set blending correctly for premultiplication, but adding almost any effect filter will set it back to straight alpha. That means that things like glow and fog will disappear.
I've seen this with an empty shaderfilter, the render delay filter, and colour correction.
If you need to apply a filter to a premultiplied source, either find a way to apply it to a group further up, or separate the source into two with different blend modes.
With Shaders, you apply a single shaderfilter to an opaque background source, and feed the other sources in as inputs. Your other spout sources should be set to premultiplied. The output of that filter is the finished composite, so your background source needs to be in front of the others.
For misbehaving games that don't have a usable foreground.
This is based on my urpbroken scene, so it will also deal with the special games that have unusable optimised color data.
OnAirTap-All3.shader
uniform string Important_Notice<
string widget_type = "info";
> = "The sources below are fetched without any filters. If you need to apply filters to them before this compositing, add a Source Clone and set that as the target instead.";
uniform texture2d ForegroundImage<
string label = "Foreground Source";
>;
uniform texture2d OptimisedImage<
string label = "Optimised Source";
>;
uniform texture2d ModelImage<
string label = "VNyan Source";
>;
uniform string Description<
string widget_type = "info";
> = "'All3' Compositing Shader for OnAirTap by milkydelta. Apply to Background Spout/PipeWire source";
//END OF INPUT VARIABLES
float4 mainImage(VertData v_in) : TARGET
{
float4 base = image.Sample(textureSampler, v_in.uv);
float4 model = ModelImage.Sample(textureSampler, v_in.uv);
float4 op = OptimisedImage.Sample(textureSampler, v_in.uv);
float4 fg = ForegroundImage.Sample(textureSampler, v_in.uv);
float thresholdModelAlpha = model.a;
if (thresholdModelAlpha >0.01){ thresholdModelAlpha = 1;}
float4 comp = base;
comp.a = 1;
comp.rgb *= 1-model.a;
comp.rgb += model.rgb;
comp.rgb = saturate(comp.rgb);
comp.rgb *= 1-(op.a * thresholdModelAlpha);
comp.rgb += fg.rgb * thresholdModelAlpha;
comp.rgb = saturate(comp.rgb);
return comp;
}
The shader to use for most games.
OnAirTap-FGBG.shader
uniform string Important_Notice<
string widget_type = "info";
> = "The sources below are fetched without any filters. If you need to apply filters to them before this compositing, add a Source Clone and set that as the target instead.";
uniform texture2d ForegroundImage<
string label = "Foreground Source";
>;
uniform texture2d ModelImage<
string label = "VNyan Source";
>;
uniform string Description<
string widget_type = "info";
> = "'FGBG' Compositing Shader for OnAirTap by milkydelta. Apply to Background Spout/PipeWire source";
//END OF INPUT VARIABLES
float4 mainImage(VertData v_in) : TARGET
{
float4 base = image.Sample(textureSampler, v_in.uv);
float4 model = ModelImage.Sample(textureSampler, v_in.uv);
float4 fg = ForegroundImage.Sample(textureSampler, v_in.uv);
float thresholdModelAlpha = model.a;
if (thresholdModelAlpha >0.01){ thresholdModelAlpha = 1;}
float4 comp = base;
comp.a = 1;
comp.rgb *= 1-model.a;
comp.rgb += model.rgb;
comp.rgb = saturate(comp.rgb);
comp.rgb *= 1-(fg.a * thresholdModelAlpha);
comp.rgb += fg.rgb * thresholdModelAlpha;
comp.rgb = saturate(comp.rgb);
return comp;
}
Because Optimised uses only one texture, I have not made a shader for it. See the Manual section.