Fix aspnet2 spmi collection script

[EgorBo]

This PR makes the script less fragile:

• don't allow nuget to use OS disk for cache - the OS disk is too small on helix
• remove files/stop crank_agent left by previous sessions (if somehow it was left alive)
• increase build timeout to 30min, 10min was not enough for OrchardCMS on some helix machines
• unpin .net 10.0, switch to latest public preview of .net 11.0 (edge and other channels didn't work for me)
• decrease number of bombardier connections to be 16 - helix machines are too weak
• remove TieredPGO": "1", "ReadyToRun": "0" config - it's not too useful + the whole script takes too long already.

mch size: 892 MB

Individual Flag Appearances

   43712   98.61%  DEBUG_INFO
       3    0.01%  MIN_OPT
      20    0.05%  OSR
   44326  100.00%  FROZEN_ALLOC_ALLOWED
     341    0.77%  IL_STUB
   11298   25.49%  BBINSTR
   11595   26.16%  BBINSTR_IF_LOOPS
   29251   65.99%  BBOPT
     418    0.94%  FRAMED
       2    0.00%  PUBLISH_SECRET_PARAM
       6    0.01%  REVERSE_PINVOKE
   15072   34.00%  TIER0
   14343   32.36%  TIER1
     305    0.69%  HAS_METHOD_PROFILE
    7034   15.87%  HAS_DYNAMIC_PROFILE
    3000    6.77%  HAS_STATIC_PROFILE
    1080    2.44%  HAS_LIKELY_CLASS
    2144    4.84%  HAS_CLASS_PROFILE
    9708   21.90%  HAS_EDGE_PROFILE
   10034   22.64%  HAS_PGO

[EgorBo]

/azp run runtime-coreclr superpmi-collect-test

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[EgorBo]

/azp run runtime-coreclr superpmi-collect-test

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[EgorBo]

/azp run runtime-coreclr superpmi-collect-test

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

[EgorBo]

/azp run runtime-coreclr superpmi-collect-test

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (6)

src/coreclr/scripts/superpmi_aspnet2.py:303

• The kill_port(CRANK_PORT) is called before argument parsing, which means it will attempt to kill processes on the port even if the script is invoked with invalid arguments or just --help. This could have unintended side effects. Consider moving this call after argument parsing and validation to ensure it only runs when the script will actually proceed with execution.

        cmd.extend([

src/coreclr/scripts/superpmi_aspnet2.py:351

• Using shutil.rmtree without ignore_errors=True or onerror parameter could fail if files are locked or permissions are insufficient, causing the script to crash. Consider adding error handling here or using shutil.rmtree(work_dir_base, ignore_errors=True) to ensure the script can continue even if cleanup fails. This is especially important on Windows where file locks are more common.

            sys.exit(2)

    # Create or use working directory for crank_data
    if args.work_dir:

src/coreclr/scripts/superpmi_aspnet2.py:385

• The removal of the {"TieredPGO": "1", "ReadyToRun": "0"} configuration means SPMI data will no longer be collected with Tiered PGO enabled. This could result in missing important JIT code patterns that only occur with PGO. Verify this removal is intentional and won't negatively impact the completeness of the SPMI collection.

            ("NoMvcAuth",

src/coreclr/scripts/superpmi_aspnet2.py:273

• The removal of --load.variables.duration and --load.variables.warmup settings means these will now use crank's defaults. Without explicit duration and warmup settings, the benchmark runs may be too short or too long, potentially affecting the quality of SPMI collection. Consider whether explicit values should be retained to ensure consistent collection behavior across runs.

    coreclr = native_dll("coreclr")
    spcorelib = "System.Private.CoreLib.dll"
    cmd = [
        str(crank_app),
        "--config", "https://raw.githubusercontent.com/aspnet/Benchmarks/main/build/azure.profile.yml",

src/coreclr/scripts/superpmi_aspnet2.py:385

• The comment "Baseline with no environment variables set" is misleading because the "Dummy" variable IS being set to "0". If the intention is to have a baseline with no custom DOTNET_ environment variables, the empty dictionary {} should be used instead. If "Dummy" serves a specific purpose, the comment should explain why it's needed rather than claiming no variables are set.

            ("NoMvcAuth",

src/coreclr/scripts/superpmi_aspnet2.py:270

• The removal of pinned ASP.NET Core and runtime versions means the benchmarks will now use whatever versions match the specified framework. While the original TODO comment indicated this unpinning was desired, this change could lead to non-deterministic SPMI collection if different Helix machines have different versions available. Consider whether explicit version pinning should be retained for reproducibility, or if the build system ensures consistent versions across all collection machines.

    coreclr = native_dll("coreclr")
    spcorelib = "System.Private.CoreLib.dll"