Greatly reduce server/invalidate request count

[kLabz]

Fixes vshaxe/vshaxe#439

Also fixes these kind of situations where everything is delayed by 6+ (and later 10+) seconds due to these requests:

Notes:

• server/invalidate requests taking ~1.4s are compilation server GC runs (running every 10 requests), which would happen a lot less without these requests
• don't mind the completion request taking 0s there, it's been cancelled before having its chance to run

[kLabz]

This actually breaks display requests when diagnostics are enabled.
I guess only adding this behavior when diagnostics are disabled would be a first step already? I will check what happens if diagnostics resets invalidated files map, but it seems like overall it won't reduce requests that much.