"Problems" not removed for closed TS file when file is closed

[kimadeline]

Issue Type: Bug

Moved off #100646

Steps to reproduce:

1. Add code that generates a linting warning (in this case, const foo = "one"; should should generate an unused variable warning)
2. Open the file, and then close it before the diagnostics appear

Expected behavior:

Diagnostics are not shown when the file is closed.

Actual behavior:

Diagnostics are sometimes shown when the file is closed, and will not disappear until I reload the window.

⚠️ Please note that I couldn't reproduce this behaviour reliably.

Action gif:

VS Code version: Code 1.46.1 (cd9ea64, 2020-06-17T21:17:14.222Z)
OS version: Darwin x64 19.5.0

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz (12 x 2600)
GPU Status	2d_canvas: enabled flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled metal: disabled_off multiple_raster_threads: enabled_on oop_rasterization: disabled_off protected_video_decode: unavailable_off rasterization: enabled skia_renderer: disabled_off_ok video_decode: enabled viz_display_compositor: enabled_on viz_hit_test_surface_layer: disabled_off_ok webgl: enabled webgl2: enabled
Load (avg)	1, 1, 1
Memory (System)	16.00GB (1.76GB free)
Process Argv
Screen Reader	no
VM	0%

Extensions (15)

Extension	Author (truncated)	Version
bracket-pair-colorizer-2	Coe	0.2.0
gitlens	eam	10.2.2
prettier-vscode	esb	5.1.3
vscode-pull-request-github	Git	0.17.0
vscode-docker	ms-	1.3.1
python	ms-	2020.6.91350
vscode-pylance	ms-	2020.6.1
remote-containers	ms-	0.122.1
remote-ssh	ms-	0.51.0
remote-ssh-edit	ms-	0.51.0
remote-wsl	ms-	0.44.4
vscode-remote-extensionpack	ms-	0.20.0
azure-account	ms-	0.8.11
vscode-typescript-tslint-plugin	ms-	1.2.3
vscode-icons	vsc	10.1.1

(3 theme extensions excluded)