fix(nodes): refined fixes for refiner inpainting #6197
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Two fixes here. Both contribute noticeably to the results when inpainting with refiner.
fix(nodes): do not set seed on output latents from denoise latents
LatentsField
objects have an optionalseed
field. This should only be populated when the latents are noise, generated from a seed.DenoiseLatentsInvocation
needs a seed value for scheduler initialization. It's used in a few places, and there is some logic for determining the seed to use with a series of fallbacks:LatentsField
object)LatentsField
object - normally it won't have a seed)0
as a final fallbackIn
DenoisLatentsInvocation
, we set the seed in theLatentsOutput
, even though the output latents are not noise.This is normally fine, but when we use refiner, we re-use the those same latents for the refiner denoise. This causes that characteristic same-seed-fried look on the refiner pass.
Simple fix - do not set the field in the output latents.
Example outputs
These include the doubly-noised latents fix from below. The output is much worse without that.
Initial image:

fix(nodes): doubly-noised latents
When using refiner with a mask (i.e. inpainting), we don't have noise provided as an input to the node.
This situation uniquely hits a code path that wasn't reviewed when gradient denoising was implemented.
That code path does two things wrong:
We don't need to add noise in
latents_from_embeddings
because we do it just a lines later inAddsMaskGuidance
.add_noise
seed
a required arg. We never call the function without seed anyways. If we refactor this in the future, it will be clearer that we need to look at how seed is handled.AddsMaskGuidance
. The created noise tensor is now only used in that function, no need to create it every time.Note: Whether or not having both noise and latents as inputs on the node is correct is a separate conversation. This change just fixes the issue with the current setup.
Example outputs
Initial image:

Related Issues / Discussions
https://discord.com/channels/1020123559063990373/1149513625321603162/1227798332567715953
QA Instructions
Do some inpainting w/ refiner (and non-refiner). Should work great.
Merge Plan
n/a
Checklist