Windows start lagging/stuttering after a few minutes unless I regularly switch workspaces.

[starcraft66]

Platform

NixOS 20.09beta

GPU, drivers, and screen setup

NVIDIA GeForce GTX 1080 Ti
NVIDIA Driver Version: 450.66
Dual Monitors, both 3840x2160@60Hz

glxinfo -B
name of display: :0
display: :0  screen: 0
direct rendering: Yes
Memory info (GL_NVX_gpu_memory_info):
    Dedicated video memory: 11264 MB
    Total available memory: 11264 MB
    Currently available dedicated video memory: 9490 MB
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: GeForce GTX 1080 Ti/PCIe/SSE2
OpenGL core profile version string: 4.6.0 NVIDIA 450.66
OpenGL core profile shading language version string: 4.60 NVIDIA
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.6.0 NVIDIA 450.66
OpenGL shading language version string: 4.60 NVIDIA
OpenGL context flags: (none)
OpenGL profile mask: (none)

OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 450.66
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20

Environment

I'm running i3-gaps. i3 version 4.18.2 (2020-07-26) © 2009 Michael Stapelberg and contributors

picom version

v8

Configuration:

I don't actually have a picom config file, I just start it with picom --backend xrender.

I can't use the glx backend because it causes a weird issue with my terminal emulator where the cursor is lagging behind everything I type. There used to be an issue for this I commented on recently but it seems to have been deleted because I searched up and down for it and I can't find it anymore.

Steps of reproduction

1. Start i3 and picom, go do some work without switching workspaces like a long browsing session or playing a game.
2. Wait a while, not exactly sure how long but ~10 mins.
3. There will be frequent stuttering/frame skipping.

Expected behavior

I expect there to be no stuttering and that scrolling/gaming always be smooth.

Current Behavior

After about 10 minutes of working without switching workspaces, a strange stuttering phenomenon starts to happen and the only way to make it go away is to switch workspaces back and forth but the problem will always keep persisting as time goes on.

I made a YouTube video describing the problem here: https://www.youtube.com/watch?v=SJrVwb6lbV4
I recorded it on my phone in case I couldn't properly capture the problem using a screen recorder.

Other details

https://www.youtube.com/watch?v=SJrVwb6lbV4

There are timestamps in the video description.