Severe pixelation in places very far from the mosaic area

[Milincho]

I think this is one of the biggest issues with Lada/Jasna: there are severe pixelation in places very far from the mosaic area in many videos.

This issue is also present in LADA, and is perfectly documented here: https://codeberg.org/ladaapp/lada/issues/107

There is already a solution posted in that thread, that also improves the demosaic result, but it hasn't been included in the main branch yet: https://codeberg.org/ladaapp/lada/issues/107#issuecomment-8357643

"Summary of Implementation
The pixelation issue in areas far from mosaic boundaries (issue #107) has been resolved with a safe, error-free solution compatible with watch (detection on/off) and export modes.
Key Changes Made:

Dynamic Clip Sizing (mosaic_detector.py):
• Added get_max_width_height() method to Scene class
• Implemented VRAM-aware clip size selection (256-1024px based on mosaic area)
• Reduces downscaling for large mosaics, minimizing pixelation
Adaptive Border Reduction (scene_utils.py):
• Halved border expansion for mosaics >512px to avoid excessive pixelation-causing downscaling
Dynamic Buffer Resizing (basicvsrpp/inference.py):
• Buffers now resize per inference call to match variable clip sizes
• Fixed RuntimeError by ensuring CPU/GPU buffer compatibility
Enhanced Model Loading:
• Added max_clip_size=1024 parameter to load_model() for larger buffer allocation
Results:
• Logs show successful initialization with tiered sizes [256, 384, 512, 768, 1024] based on 12GB VRAM
• Buffer resizing works correctly (e.g., from 1024x1024 to 384x384 without errors)
• Solution prevents OOM with safety caps and fallbacks
• Maintains backward compatibility and works across all modes
The artifacts (hair distortion, pixel borders) in areas far from mosaics are now eliminated through reduced downscaling and better blending isolation.
Same picture extended to show how good is the result (demosaic result is also became a bit better in terms of image quality):

Picture made from v0.8.2 release for comparison:

[Kruk2]

Yep, thats something in my backlog. Good that you created an issue for it.
It doesn't bother me that much since in my content I usually don't see that pixelation but it's still something I would like to fix.
The first solution proposed in that thread is not correct one in my opinion
a) huge vram usage
b) restoration model wasn't really trained on other res than 256x256
c) hides the issue instead of fixing how we blend restored images

There was another attempt to fix it in lada (I think it has PR active), much better but still not something that worked well in my tests.
As a workaround for now: use 2nd restoration model (tvai or rtx)

[Milincho]

I do a lot of top quality remasters and this is like my number 1 issue. Happens in a lot of my videos.

The posted result from the test scene ((KV-109 and timecode ~15 min) looks perfect to me... 🤔
There is no pixelation at all in any area there, the image retains all the quality, and the author says it even improves the demosaic result.

2nd restoration model doesn't fix this issue:

Original:

TVAI:

Fix from the LADA thread:

[cgkings]

Yep, thats something in my backlog. Good that you created an issue for it. It doesn't bother me that much since in my content I usually don't see that pixelation but it's still something I would like to fix. The first solution proposed in that thread is not correct one in my opinion a) huge vram usage b) restoration model wasn't really trained on other res than 256x256 c) hides the issue instead of fixing how we blend restored images

There was another attempt to fix it in lada (I think it has PR active), much better but still not something that worked well in my tests. As a workaround for now: use 2nd restoration model (tvai or rtx)

After my actual testing, using Jasna v0.5.0-alpha5, with the second scheme, the best practice is to use RTX 2x high-quality upscaling, and then perform denoising after secondary repair in advanced processing to achieve the best repair effect.

[cgkings]

The repair effect of HZGD-233 is very poor, whether it's Lada or jasna, I can't always find a good method, possibly because non-repairable objects, such as hands or tongues, entered the mosaic
Additionally, almost all methods, whether lada or jasna, inevitably extend or bold the original objects in certain scenarios
I have an idea to create multiple models, not to restore them, but to compare them and replace the original with the closest model. This might be clearer, perhaps

[Kruk2]

@Milincho if I would implement fix proposed in that thread you wouldn't be able to run Jasna, it would eat like 40gb vram. It uses 4x larger restoration resolution and in my opinion is workaround.

[Milincho]

@Milincho if I would implement fix proposed in that thread you wouldn't be able to run Jasna, it would eat like 40gb vram. It uses 4x larger restoration resolution and in my opinion is workaround.

I don't understand... how did suphamster run his fix then? there are no 40GB VRAM consumer GPUs... 🤔

He even listed:
• Implemented VRAM-aware clip size selection

I understand it would eat more performance and VRAM. That makes sense. But being impossible to run? suphamster results already proved otherwise...

[Milincho]

I see you are already in the conversation about it since months ago here: https://codeberg.org/ladaapp/lada/pulls/113

suphamster latest version of his fix looks even better, exactly like the original frame.

At least this fix could be optional to the user.
If this works, I'd happily take a 200% performance hit any day. Lada and Jasna are already very fast. JAVplayer was hell in comparision, but at least it didn't have this pixelation issue at all.

[Kruk2]

https://codeberg.org/ladaapp/lada/pulls/113#issuecomment-8391414
There is a reason why this wasn't added in Lada as well - it's AI workaround, not a fix. In my opinion it's solvable in a better way, like this: https://codeberg.org/ladaapp/lada/pulls/278
In Lada it causes VRAM spikes and Jasna is already VRAM hungry so thats why I said 40gb vram.

I'm not negating that this change isn't fixing the issue or that issue doesn't exist. Just saying that this is not the fix that I want to see in Jasna.

[Milincho]

Ok, but that other 'better way' is also from months ago...

I uploaded the small clip ( 10MB ) for testing purposes: https://mega.nz/file/C6p0BYzY#pC5KQAUPhuufVsl0k83RG3iFhupEFTN7QC2AHgnHBhA

[Kruk2]

Well, I run this project for free, on limited time so I'm prioritizing things that I enjoy to implement/are bothering me.
That's why I'm not saying that this will be completed by X day since I don't have such obligations.
But I'm also not saying this won't be fixed at some point.
Also contributions are welcomed since its open source project.

[Milincho]

Well, I run this project for free, on limited time so I'm prioritizing things that I enjoy to implement/are bothering me. That's why I'm not saying that this will be completed by X day since I don't have such obligations. But I'm also not saying this won't be fixed at some point. Also contributions are welcomed since its open source project.

You are totally right and I fully understand. Done.

[Kruk2]

Now I feel a bit of pressure, thanks for support :)
Still the VRAM issue has higher priority but then who knows, pixelation?

[Milincho]

I agree. Having VRAM related freezes and crashes myself trying to use TVAI outside small clips, no matter the settings.

If at any point you like me to do any specific tests let me know. Happy to help where I can.

[Kruk2]

@Milincho Check v0.6.0-alpha1. It's not that easy to find balance in blending logic so it works for all combinations:

• small/large mosaic areas
• 480p/4k videos
• imperfections of mosaic detection mask

If anyone finds example where v0.5.0 gives better blending than v0.6.0-alpha1 let me know JAV code + timestamp.

[suphamster]

From quick test looks very good. There are some room for improvement for close to mosaics borders and inside restored mosaics areas since there are some small pixelation and noise/blur still exists. But for sure this is much better than was in jasna older versions.