High memory consumption when loading ARM JSON template.

[ptsapralis]

Type: Performance Issue

Memory consumption on the Apple Silicon Mac skyrockets at 3.5+ GB once I open a single small-size ARM template into VS Code.

Extension version: 0.15.11
VS Code version: Code 1.89.1 (Universal) (dc96b837cf6bb4af9cd736aa3af08cf8279f7685, 2024-05-07T05:14:24.611Z)
OS version: Darwin arm64 23.5.0
Modes:

System Info

Item	Value
CPUs	Apple M1 Pro (10 x 24)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	4, 3, 3
Memory (System)	16.00GB (0.05GB free)
Process Argv	--crash-reporter-id 16489c40-bc09-45e3-bcc5-5b284c33690b
Screen Reader	no
VM	0%

Process Info

CPU %	Mem MB	   PID	Process
    8	   115	 32101	code main
    3	    49	 32105	   gpu-process
    0	    16	 32106	   utility-network-service
    2	    98	 32110	window [1] (Dockerfile — Ubuntu Jammy Server (22.04) Lab Docker Image)
    0	   180	 32111	window [2] (Azure-DEVOPS-Pipelines-Self-Hosted-Agents-Azure-Gallery-Image-VM-template-V2.json — IntegrationScaleSet-Images)
    1	    98	 32112	window [3] (main-sbapi.tf — em-supplierbookingapi-infra)
    1	    98	 32114	window [4] (generate-ssr-load-test-links.sh — ssr-service-load-test)
    1	    98	 32115	window [5] (Slack Channel exp-tech-inc-too-many-500s-on-bag-fs KQL Queries (20240410).loganalytics)
    0	    33	 32120	extensionHost [5]
    0	    16	 32154	     electron-nodejs (languageserver.js )
    2	    66	 32121	shared-process
    0	     0	 37381	     /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=
    0	    16	 32122	fileWatcher [5]
    0	    33	 32123	extensionHost [4]
    0	    16	 32149	     electron-nodejs (server.js )
    0	    16	 32187	     electron-nodejs (languageserver.js )
    0	    33	 32124	extensionHost [1]
    0	    16	 32144	     electron-nodejs (server.js )
    0	    16	 32145	     electron-nodejs (server.js )
    0	    16	 32167	     electron-nodejs (languageserver.js )
    0	    16	 32330	     /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=32124
    0	    66	 32125	extensionHost [2]
    0	    16	 32152	     /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=32125
    0	    16	 32190	     electron-nodejs (languageserver.js )
    1	    66	 32220	     /Users/ptsapralis/Library/Application Support/Code/User/globalStorage/ms-dotnettools.vscode-dotnet-runtime/.dotnet/7.0.20~arm64/dotnet /Users/ptsapralis/.vscode/extensions/msazurermtools.azurerm-vscode-tools-0.15.11/languageServer/Microsoft.ArmLanguageServer.dll --logLevel Warning
    0	    49	 32126	extensionHost [3]
    0	     0	 32141	     /Users/ptsapralis/.vscode/extensions/hashicorp.terraform-2.30.1-darwin-arm64/bin/terraform-ls serve
    0	    16	 32188	     electron-nodejs (languageserver.js )
    0	    33	 32129	fileWatcher [2]
    0	    16	 32131	fileWatcher [3]
    0	    16	 32132	fileWatcher [1]
    0	    16	 32133	fileWatcher [4]
    0	    33	 32142	ptyHost
    0	     0	 32151	     /bin/bash --init-file /Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/contrib/terminal/browser/media/shellIntegration-bash.sh
    2	    82	 37376	window [6] (Issue Reporter)

Workspace Info

|  Window (generate-ssr-load-test-links.sh — ssr-service-load-test)
|  Window (main-sbapi.tf — em-supplierbookingapi-infra)
|  Window (Azure-DEVOPS-Pipelines-Self-Hosted-Agents-Azure-Gallery-Image-VM-template-V2.json — IntegrationScaleSet-Images)
|  Window (Dockerfile — Ubuntu Jammy Server (22.04) Lab Docker Image)
|    Folder (ssr-service-load-test): 211 files
|      File types: js(36) svg(30) ttf(9) woff(9) woff2(9) log(6) txt(4) css(3)
|                  ico(3) webmanifest(3)
|      Conf files:
|    Folder (em-supplierbookingapi-infra): 13 files
|      File types: tfvars(4) tf(3) hcl(2) gitignore(1) md(1)
|      Conf files:
|    Folder (IntegrationScaleSet-Images): 10 files
|      File types: sh(4) json(2) gitignore(1) md(1)
|      Conf files:
|    Folder (Ubuntu Jammy Server (22.04) Lab Docker Image): 3 files
|      File types: md(1)
|      Conf files: dockerfile(1);

A/B Experiments

vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscorecescf:30445987
vscod805cf:30301675
binariesv615:30325510
vsaa593cf:30376535
py29gd2263:31024239
vscaat:30438848
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
pythongtdpath:30769146
welcomedialog:30910333
pythonidxpt:30866567
pythonnoceb:30805159
asynctok:30898717
pythontestfixt:30902429
pythonregdiag2:30936856
pythonmypyd1:30879173
pythoncet0:30885854
h48ei257:31000450
pythontbext0:30879054
accentitlementsc:30995553
dsvsc016:30899300
dsvsc017:30899301
dsvsc018:30899302
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
chatpanelc:31048052
dsvsc021:30996838
945dj816:31013170
pythoncenvpt:31062603
a69g1124:31058053
dvdeprecationcf:31061161
pythonprt:31056678
dwnewjupytercf:31046870
26j00206:31048877

[ptsapralis]

Impact of this issue on memory consumption shown in attached screenshots (before & after launching Visual Studio Code with a single ARM template):

Even after closing VS Code, the "dotnet" stays alive not releasing the memory back to free pool.

[demenone]

Same here. I just put killalll dotnet on the weekends.

[btull89]

I'm running into the same issue on Windows 11. The status bar at the bottom of VS Code says "Loading ARM schemas" when the slowdown occurs.

[theramzay]

Same for me. MacBook Pro 2021 M1 Pro. macOS 14.5

[markwragg]

I am also experiencing this issue. It makes the extension unusable.

[StephenWeatherford]

Is the extension usable after the schema loading finishes (and Intellisense starts being responsive)? The list of schemas available has grown substantially since this code was originally designed/written, so now it takes quite a while to do the initial loading of schemas, plus significant memory usage. Improving this would not be trivial.

[btull89]

Is the extension usable after the schema loading finishes (and Intellisense starts being responsive)? The list of schemas available has grown substantially since this code was originally designed/written, so now it takes quite a while to do the initial loading of schemas, plus significant memory usage. Improving this would not be trivial.

This somehow doesn't seem to be as big of an issue for me lately. I tried to reproduce it but couldn't. I also tried loading multiple azure deploy files and they all loaded within 3 minutes...

[ptsapralis]

Hi. I wouldn’t call it “usable” on mid-range systems. The screenshots below show how it behaves:   When VS Code is launched for the 1st time, the “dotnet” component takes up to almost 5GB of memory. After I close VS Code, that memory is not freed up - “dotnet” keeps running & consuming memory. When I launch the software for a 2nd time, a new “dotnet” component starts running consuming an extra 5GB of memory. Even after I close it, the 2 “dotnet” instances keep running & using that memory. It’s OK when running on my high-end Mac (M1 Max / 32GB) - but it becomes pretty much useless on a mid-range system. I recall the situation was much worse when I raised this ticket a few months ago: a separate “dotnet” instance (with around 3-4GB of memory allocation) was launched for each ARM template I would load into the editor. It seems this is no longer the case. However, I think that 5GB memory consumption for schemas/definitions/etc. is too high - there must be some more efficient way of handling this. Best, — Panos Tsapralis Principal Engineer Buyagift & Red Letter Days Mobile: 07507477883 Email: ***@***.*** 10 Back Hill, London, EC1R 5EN   Buyagift & Red Letter Days are trading names of Experience More Ltd.Registered in England and Wales. Company No: 03883868 Registered Address:10 Back Hill, London, EC1R 5EN This message and any attachments are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person.

…

On 11 Sep 2024, at 18:25, Brandon Tull ***@***.***> wrote: Is the extension usable after the schema loading finishes (and Intellisense starts being responsive)? The list of schemas available has grown substantially since this code was originally designed/written, so now it takes quite a while to do the initial loading of schemas, plus significant memory usage. Improving this would not be trivial. This somehow doesn't seem to be as big of an issue for me lately. I tried to reproduce it but couldn't. I also tried loading multiple azure deploy files and they all loaded within 3 minutes... — Reply to this email directly, view it on GitHub <#1709 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADLRXZII6EPRTMF22NFY4RTZWB4I5AVCNFSM6AAAAABIRV5HU2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBUGI3TCMBSGU>. You are receiving this because you authored the thread.

[theramzay]

Hi. I wouldn’t call it “usable” on mid-range systems. The screenshots below show how it behaves:   When VS Code is launched for the 1st time, the “dotnet” component takes up to almost 5GB of memory. After I close VS Code, that memory is not freed up - “dotnet” keeps running & consuming memory. When I launch the software for a 2nd time, a new “dotnet” component starts running consuming an extra 5GB of memory. Even after I close it, the 2 “dotnet” instances keep running & using that memory. It’s OK when running on my high-end Mac (M1 Max / 32GB) - but it becomes pretty much useless on a mid-range system. I recall the situation was much worse when I raised this ticket a few months ago: a separate “dotnet” instance (with around 3-4GB of memory allocation) was launched for each ARM template I would load into the editor. It seems this is no longer the case. However, I think that 5GB memory consumption for schemas/definitions/etc. is too high - there must be some more efficient way of handling this. Best, — Panos Tsapralis Principal Engineer Buyagift & Red Letter Days Mobile: 07507477883 Email: @.*** 10 Back Hill, London, EC1R 5EN   Buyagift & Red Letter Days are trading names of Experience More Ltd.Registered in England and Wales. Company No: 03883868 Registered Address:10 Back Hill, London, EC1R 5EN This message and any attachments are confidential and may be privileged or otherwise protected from disclosure. If you are not the intended recipient, you must not copy this message or attachment or disclose the contents to any other person.
…
On 11 Sep 2024, at 18:25, Brandon Tull @.***> wrote: Is the extension usable after the schema loading finishes (and Intellisense starts being responsive)? The list of schemas available has grown substantially since this code was originally designed/written, so now it takes quite a while to do the initial loading of schemas, plus significant memory usage. Improving this would not be trivial. This somehow doesn't seem to be as big of an issue for me lately. I tried to reproduce it but couldn't. I also tried loading multiple azure deploy files and they all loaded within 3 minutes... — Reply to this email directly, view it on GitHub <#1709 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADLRXZII6EPRTMF22NFY4RTZWB4I5AVCNFSM6AAAAABIRV5HU2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBUGI3TCMBSGU. You are receiving this because you authored the thread.

Same for me. And for my 16gb system 5gb is pretty much(Jetbrains Rider with 63 projects in solution eats 5gb for example) and you easily can found yourself with 30gb of swap, just because you forget explicitly go to activity monitor and kill this «dotnet» processes that this extension spawns every time you open ARM template for quick edit/view.

[meisterheister]

Same issue here, I waited several minutes, still loading. System (Windows 11, 8GB RAM) is unusable and never saw it completing.
In the end I disabled the extension, very sad.

[StephenWeatherford]

I'm sorry, I know this is frustrating. The number of schemas keeps growing, and fixing this would require a substantial rewrite. We just don't have the resources for that anymore.

[ptsapralis]

I'm sorry, I know this is frustrating. The number of schemas keeps growing, and fixing this would require a substantial rewrite. We just don't have the resources for that anymore.

Hi. Good thing a member of the Microsoft team has FINALLY come back to this chat with a meaningful reply - thank you very much for this.

While I am well aware of such circumstances like lack of resources / etc. (I've had my own good share of that over the last 40+ years), please, allow me to gently push back by saying that (I believe that) the "VS Code + ARM templates" combination is a key component of the wider Microsoft Azure+DEVOPS experience, right? So, I'd guess I'm not the only one to find it strange that there are not enough resources available to fix this issue (which is literally rendering VS Code useless for any serious work on ARM deployments). There must be some way to fix this ASAP - if nothing else, the team can at least just push back on releasing new features for VS Code to make time for rewriting the ARM handling.

[StephenWeatherford]

Pushing back is always acceptable.

cc @puicchan