Ram fills up

[Rozin14]

Ram fills up when playing 1080p videos on 2560*1600 screen. At whatever quality setting. Device: Laptop with 32GB and 3070 gpu

[emoose]

E: (in the slim chance any NV engineers might see this after my report, the initial build found at https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.0 should display the issue, or to build from source use a684e0a)

Confirmed on my side too, memory usage grows slowly after playback begins, but after 30 seconds or so suddenly ramps up to GBs of use, disabling SuperRes in NVCP stops it happening. Seems really consistent too, one video I got would shoot up to 10+GB ram usage at the exact same moment every time I played it.

Do you know what codec the video you tried with uses? it seemed to only happen with h265 for me, h264 1080p worked fine without memory issue.
(too bad browsers don't support h265 files, would have been interesting to check with them)

MPC-HC also crashes if I try closing after the memory issue, and mentions nv_disp having an error, might be the issue is on NV's side, possible they didn't test all of h265 since browsers don't really support all of it afaik.

E: could be something to do with P010 decoding, only seeing the memory issue happen with videos that use that format anyway, sadly disabling that in MPC-VR properties stops it from using D3D11 video processing for it at all, haven't seen anything that can convert it to a different format first...

[clsid2]

You can disable P010 output in LAV Video Decoder. You also need to change hardware decoder from Native to Copyback.

Some suggested code:

if (!m_bInterlaced && !m_bHdrDisplayModeEnabled && !SourceIsHDR() && params.cformat == CF_NV12) {
    // SetSuperRes code
    // I am assuming it doesn't work properly with interlaced stuff and HDR input/output
}

In addition to limiting the colorspace format, you may need to check which DXGI_FORMAT values are working good. You can select 8/10/16 bit texture format in the settings to test.

[emoose]

You can disable P010 output in LAV Video Decoder. You also need to change hardware decoder from Native to Copyback.

Aha looks like that did the trick, video that ballooned to 10GB+ RAM now stays at 300MB or so, using NV12 instead of P010, thanks for the info!

Guess this could be worth reporting to NV, not sure how amused they'll be about a bug report for this though, but hopefully they'll at least try looking into it. (E: submitted to their feedback form)

Some suggested code:

Ah good idea, will look into adding it soon.

---

P01X workaround:

You can convert P01X & make VSR apply to all videos by accessing the LAV Video decoder settings in MPC-HC's Options > Internal Filters section, in the Output Formats section uncheck all of them except NV12, and change Hardware decoder to DXVA2 (copy-back)

Didn't have much luck with MPC-BE's internal filter, but you can install LAV Filters from https://github.com/Nevcairiel/LAVFilters/releases and follow the same steps in https://www.youtube.com/watch?v=GaQvHLRVxgs to add them into MPC-BE, and use the LAV Video Configuration program that was installed to set the formats as above.

Note that this will downsample any 4:4:4/4:2:2 content down to NV12's 4:2:0, resulting in chroma subsampling prior to being scaled by VSR - apparently isn't that noticeable to most people but is still a slight downgrade.

Alternatively you could just only clear the 4:2:0 row (except NV12) and keep the other rows enabled, should make it so only 4:2:0-native videos will be converted to NV12 and passed to VSR, while 4:4:4 / 4:2:2 will keep their color format and be skipped by VSR instead.

Hopefully NV might consider supporting more color formats in future as more non-browser apps start making use of it.

[clsid2]

D3D11 decoder uses Native by default. But you can let it do Copyback by selecting a hardware device below it.

Test if it works in your browser with HDR videos on Youtube. Those files use 10-bit VP9 or AV1 video.

Additionally, you can also use this for vendor specific code segments:
m_VendorId == PCIV_NVIDIA

[emoose]

Hm not noticing much with HDR videos, GPU usage stays flat with them, couldn't notice any difference toggling it neither, maybe they disable those in browsers somehow.

Additionally, you can also use this for vendor specific code segments:

Added some checks for that earlier in 41c757e :)

What I'd really like to find is some way to tell if SuperRes is actually active, since atm the code there would display Intel-VPE / Nvidia-VSR in stats overlay even if it's not actually being processed, or user turned it off in NVCP etc...

Doesn't look like VideoProcessorSetOutputExtension gives any useful result code, might try messing with GetOutputExtension to see what that returns, otherwise maybe will change that text to "Nvidia-VSR-possible" or something like that.

[Rozin14]

YES. confirmed issue only with h265 videos. Fix of clsid2 does solve it

[emoose]

@clsid2 added your suggestions in 4e2e160 - seemed VSR was still working on some interlaced clips I tested though so removed the interlaced check.

Made it only check for P010/P016 too since we know those are likely faulty, if it's found that other formats are also having issues we can make it only allow NV12 though.

[clsid2]

Can you guys test with YUY2 and RBG32 input?
Simply uncheck all but one output format in LAV Video Decoder to force a specific colorspace.

[emoose]

Oh good idea, with YUY2 it still requests VSR but doesn't look like it's active, low power draw & no difference toggling in NVCP.

RGB32 didn't seem to request it, doesn't look like MPCVR is using VP for that since it mentions using shaders, wonder why...

[clsid2]

Also test the effect texture format setting. I suspect it might only work with 8-bit (or auto for 8-bit input video). The Ctrl+J overlay shows the texture formats used.

Your release notes mentioned that dithering should be disabled. Is that really true? Any idea why this breaks VSR? You might have used 10-bit video when testing this.

[emoose]

Your release notes mentioned that dithering should be disabled. Is that really true? Any idea why this breaks VSR? You might have used 10-bit video when testing this.

I noticed it was stopping VSR from working early on but not sure what video I was trying that with, just gave it another try and seemed to work fine with each video I tried now, guess that part can be removed to simplify the install.

Also in early tests 10/16-bit formats didn't seem to be working, the release page did mention changing it to 8-bit initially, but after trying some other videos it seems to be fine with any of the texture formats MPC-VR allows choosing, so removed that part from it.

[clsid2]

Next step to test is input size and aspect ratio limitations. What I have read so far VSR works for video with size up to and including 1080p. But maybe real max is 1440p on 4k screen. I also saw somewhere that it might now work for vertical video (height>width).

A good way to test with any size you want is to use ffdshow. Install ffdshow. You only need its raw video processing filter. Add that as a preferred external filter in MPC-HC. This filter has a resizing option where you can enter any desired size. So easy to test for example something slightly larger than 1080p, or to create a vertical video, with any input file you want.

m_srcRect can be used for input size in MPCVR.

[emoose]

Not having much luck with ffdshow using rev4533 it seems, added the "ffdshow raw video filter" as preferred external filter and enabled resizing in the ffdshow config, tested at 640x480 but video looks the same, and MPC-VR stats still shows the original video resolution there...

Tried testing a 4K video with desktop resolution at 5120x2880 via NV DSR, with superres disabled hwinfo showed it at ~80w, enabling it bumped it up to ~96w, toggling it in NVCP seemed to increase/drop the usage as I clicked it so guessing something is being activated there, but really couldn't see any difference in the output (image diff tools show them as identical too...)

Probably is worth adding some kind of check to save people from wasting power on it.

[clsid2]

ffdshow should show up here when active:
MPC-HC menu > Play > Filters
It won't get used with DXVA2 Native. It also has a tray icon when it is active.

You can use Ctrl+E to quickly reload a video in MPC-HC. That may help if dynamic size changes do not work.

I doubt it is doing something useful with such minor power increase. Certainly for 4k input.

[emoose]

Maybe it's NV12, noticed it wasn't listed in the filters media types for ffdshow, if I change LAV to use YUY2 that does let it take effect & shows in the Play > Filters menu, but with NV12 it doesn't show there any more, and we know that YUY2 doesn't actually work with VSR :/

Tried adding guid for it to the mediatypes but that didn't seem to help neither, I noticed if I start playback with YUY2 and then change filters to use NV12 it does seem to keep the resize & MPC-VR says it's NV12 too, but kinda doubtful things are all being applied properly with that.

[clsid2]

Try with YV12 input and let ffdshow output NV12

[emoose]

Aha nice looks like it's working now, ty! Now to see what sizes it works with.

E: 720x1280, 1080x1920, 1440x2560 have noticeable quality difference with SuperRes toggled (using No aspect ratio correction in ffdshow, with Double Size video frame in MPC), so vertical resolutions are fine.

No difference in image with 3840x2160 or 2160x3840, ditto with 3200x1800.

2562x1440 -> 5124x2880 does still have noticeable sharpness difference though, can't be capped at 2560x1440 then - maybe they capped to ultra-wide 1440p or something, will try some more soon.

E2: oh hmm, 3840x2160 does seem to have a sharpness increase actually:
On:

Off:

It's subtle but can see a difference on the headset on left side there, not much else though.
Really her face looks pretty much identical to me, even down to the teeth, wonder if it might only be applying to certain area of the image?
Or I guess if they never trained the AI for 2160p maybe it just isn't able to do much with it.

[clsid2]

Barely any difference. What is the extra power draw?

[kolunio82]

My Mpc hc player disabled HDR in windows cp all the time when I start playing video. I cant have HDR enabled in windows cp?

[emoose]

Barely any difference. What is the extra power draw?

About 10w, went from ~87w to ~97w when toggling it.

Might be even worse with the older cards though, saw some people saying it can reach 200w-300w for them but mines never gone near that high (highest I've seen on mine is maybe 150w or so)

My Mpc hc player disabled HDR in windows cp all the time when I start playing video. I cant have HDR enabled in windows cp?

Maybe try changing the settings in MPC-VR properties page (the window shown at #5 (comment)), there's some HDR related things there but not sure how much they might help.

[clsid2]

It is the renderer that disables HDR, not the player. The option in the renderer controls it.

[kolunio82]

It is the renderer that disables HDR, not the player. The option in the renderer controls it.

Ok but it is necessary? In MadVR i can enable hdr in windows cp and play any video.

[clsid2]

No, you adjust renderer options to not disable it. But it can potentially give you incorrect colors.

[kolunio82]

So in lav:
nv12 and p010 - unchecked others
ditherng - ordered or random?
in mpc:
chroma - bilinear
upscaling - catmul
downscaling - hamming
Correct?

[clsid2]

NV12 checked. If you want to process 10-bit video you need to uncheck P010 and the other 10/16-bit formats in LAV.

Leave the settings under "Shader video processor" as default. Those scaling algorithms are NOT used because scaling is done by DXVA2/D3D11 video processor!

[Pikkuhippi]

@kolunio82
If you mean that your Windows HDR turns off when you playback videos with the renderer, change the setting:
"Auto display HDR On/Off -> not used" this will tell it to not disable it like clsid2 mentioned.

[emoose]

Seems like 531.26 hotfix might not be applying VSR to P010 at all anymore, so ram issue doesn't happen, can anyone else who had the ram issue before confirm? (you'd need to use the older RTX MPC-VR that didn't check for P010 to test: https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.0)
For me playing P010 gave no increased power/gpu usage, and no memory issue, but NV12 still worked fine.

So for P010 you'd still need to do the same as what we suggest here, using LAV to convert P010 to NV12 instead. Still hopeful they'll add proper support for P010 in future (seeing as VSR did seem to apply fine to it before besides the memory leak...)

Hotfix links can be found here: https://www.reddit.com/r/nvidia/comments/11l92kg/geforce_hotfix_driver_version_53126/

[kolunio82]

Seems like 531.26 hotfix might not be applying VSR to P010 at all anymore, so ram issue doesn't happen, can anyone else who had the ram issue before confirm? (you'd need to use the older RTX MPC-VR that didn't check for P010 to test: https://github.com/emoose/VideoRenderer/releases/tag/rtx-1.0) For me playing P010 gave no increased power/gpu usage, and no memory issue, but NV12 still worked fine.

So for P010 you'd still need to do the same as what we suggest here, using LAV to convert P010 to NV12 instead. Still hopeful they'll add proper support for P010 in future (seeing as VSR did seem to apply fine to it before besides the memory leak...)

Hotfix links can be found here: https://www.reddit.com/r/nvidia/comments/11l92kg/geforce_hotfix_driver_version_53126/

So how can I configure lav exactly?

[emoose]

So how can I configure lav exactly?

It's mentioned in bottom part of #1 (comment)

[kolunio82]

When i enable hdr in wibdows, not mpc hc, settinga vsr don't work. I need to have hdr disabled in windows all the time? :(

[emoose]

@kolunio82 try the rtx-1.2 build at #5 (comment), seemed to help others with the same issue.

[kolunio82]

Thank you very much. Its working. Vsr dont support Apple prores? I have few videos in this format and Vsr its not available.

[clsid2]

ProsRes is usually 10-bit which means you have to use the P010 workaround.

[kolunio82]

ProsRes is usually 10-bit which means you have to use the P010 workaround.

So what shoud I do exactly? Im sorry for noob question. Im not advanced user.

[marcmy]

Hi, are these fixes for P010 and HDR still required? I just got the latest nvidia driver update (545.84) which enabled support for 20 series. Just checking if there were any changes or hotfixes made since this was last released. Thanks.

[hani3887gg]

can i use vsr on a 4k video?