node+npm needed on build images for Windows, and Mariner-2

[radical]

dotnet/aspire CI is running on https://helix.dot.net/#heading-windows-vs2022preview-amd64-open and https://helix.dot.net/#heading-1es-mariner-2 and those don't appear to have node+npm installed on them by default. We need these for running some new tests.

Linux at least has node installed though, but no npm. And Windows is missing both.

cc @RussKie @joperezr

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Added node 20 to windows and linux queues

[radical]

@dougbu any ETA on this one? We have tests blocked on dotnet/aspire due to this.
cc @eerhardt

[dougbu]

Sorry. I haven't had a chance to look. I'll keep a tab open here and get back to you b/4 Monday

[dougbu]

oh, I mixed this up w/ #3122. this issue is part of a "spike" of operations issues we're working through. it's no longer really on our backlog, more a matter of finding an assignee and / or deciding what we do when

could you speak to reason behind the urgency @radical❓ for example, why are automated tests on additional platforms needed❓

/cc @ilyas1974 and @garath

[radical]

or example, why are automated tests on additional platforms needed❓

Using node+npm projects with dotnet/aspire is an explicit feature of the product, in parallel to all the others, and utilizes explicit aspire APIs for it. So, running node+npm dependent tests is not a different platform for us, it is just testing one part of the product.

Does that answer your question?

[dougbu]

or example, why are automated tests on additional platforms needed❓

Using node+npm projects with dotnet/aspire is an explicit feature of the product, in parallel to all the others, and utilizes explicit aspire APIs for it. So, running node+npm dependent tests is not a different platform for us, it is just testing one part of the product.

Does that answer your question?

not exactly. if you can test node+npm on Linux, what's the urgency behind also testing those scenarios on Windows❓

[radical]

cc @eerhardt

[eerhardt]

if you can test node+npm on Linux, what's the urgency behind also testing those scenarios on Windows❓

Because they are different, and we can have different bugs based on the OS. One example is here:

https://github.com/dotnet/aspire/pull/5142/files#diff-7a45254774650a016a455a5455a121b2ea58a5d9f5ee718dadd0a25127559967

npm isn't npm.exe on Windows, but it is npm.cmd. So when looking for the command the existing code wasn't finding it correctly.

[radical]

Any update on this? Also Linux image has node 12 installed which reached EOL two years ago.
This is actively blocking testing on dotnet/aspire. dotnet/aspire#2225 (comment)
cc @davidfowl

[riarenas]

I will start looking at this today.

A few questions.

https://helix.dot.net/#heading-1es-mariner-2

Is an image we get from 1ES, and not something we should modify, so we need to decide how we want to fix this scenario:

• Can you use docker instead of needing NPM on the host?
• Alternatively, could you try moving to the build.ubuntu.* images from https://helix.dot.net/#1ESHostedPoolImagesWestUS-rg-Internal-Linux where we do install our artifacts?

If neither of these options help, we will need to create a new image, which we can try to base off the existing mariner image, but it will take some experimentation to get that working end to end.

It seems we at least think that we install a portable version of Node 16 in the windows image under C:\arcade-tools We install node 10 and 16 in this way. Can you check if this installation would help with your needs?

[radical]

Can you use docker instead of needing NPM on the host?

This needs to be explored, and would be non-trivial to implement. Especially, because aspire depends on talking to docker, and creating/building/running images.

Alternatively, could you try moving to the build.ubuntu.* images from helix.dot.net#1ESHostedPoolImagesWestUS-rg-Internal-Linux where we do install our artifacts?

How different is this from the existing images?

Also, I want to confirm that I shared the correct information. These are the helix queues that we use specifically, where we want this to be available:

            HelixTargetQueues: Windows.11.Amd64.Client.Open
            HelixTargetQueues: Windows.11.Amd64.Client

            HelixTargetQueues: Ubuntu.2204.Amd64.Open
            HelixTargetQueues: Ubuntu.2204.Amd64

[riarenas]

How different is this from the existing images?

The software installed in the images is located in the Helix website. We have full control over the build.ubuntu images, while anything that starts with 1es-* is provided as is by 1ES and we don't control, or even know fully what is installed in them.

Also, I want to confirm that I shared the correct information. These are the helix queues that we use specifically, where we want this to be available:

That is certainly different from the issue description. I was looking at the build images, not the test images. Do you need this for build, or for tests in helix? (or both?)

[radical]

Also, I want to confirm that I shared the correct information. These are the helix queues that we use specifically, where we want this to be available:

That is certainly different from the issue description. I was looking at the build images, not the test images. Do you need this for build, or for tests in helix? (or both?)

Oh my apologies. Yes, the initial ask was on build images because most of our tests were running there. But now we have moved most to helix. So, we need this on the helix images too.

[riarenas]

Thanks for the clarification! I can't find us installing node in the windows helix queues, and looks like we only install really old node versions in the ubuntu queue.

So to clarify, the places where node + npm are needed:

• Build images:
  • windows.vs2022preview.amd64[.open]
  • build.ubuntu.2204.amd64[.open]
• Helix test queues:
  • windows.11.amd64.client[.open]
  • ubuntu.2204.amd64[.open]

I'm making the following assumptions:

• The latest LTS (node 20) would work
• You are able to use the build.ubuntu.* ubuntu build images provided by dnceng, as we shouldn't install additional software directly over the 1es-mariner-2 image.

Let me know if either of these assumptions don't hold!

[radical]

Thanks for the clarification! I can't find us installing node in the windows helix queues, and looks like we only install really old node versions in the ubuntu queue.

So to clarify, the places where node + npm are needed:

• Build images:

  • windows.vs2022preview.amd64[.open]
  • build.ubuntu.2204.amd64[.open]

• Helix test queues:

  • windows.11.amd64.client[.open]
  • ubuntu.2204.amd64[.open]

Correct.

I'm making the following assumptions:

• The latest LTS (node 20) would work

Yes.

• You are able to use the build.ubuntu.* ubuntu build images provided by dnceng, as we shouldn't install additional software directly over the 1es-mariner-2 image.

Yes, I will try to move us to that.

Thank you!