Skip to content

Share a single Windows VM config script to add data science tooling via Nexus#4981

Merged
marrobi merged 26 commits into
mainfrom
copilot/update-powershell-script-for-tooling
Jul 24, 2026
Merged

Share a single Windows VM config script to add data science tooling via Nexus#4981
marrobi merged 26 commits into
mainfrom
copilot/update-powershell-script-for-tooling

Conversation

Copilot AI commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What is being addressed

The DSVMs were removed (#4890), leaving the remaining Guacamole Windows VMs with only a Nexus proxy config script (vm_config.ps1) that installed no tooling. This PR adds data science tooling, Azure CLI, and Storage Explorer via Nexus (the VMs have no internet access) and consolidates the per-VM PowerShell configuration into a single base script shared across all three Windows VMs — the airlock review VMs keep their specialised download logic layered on top.

This supersedes #4889 (which duplicated the Nexus pip/conda config into each review VM).

How is this addressed

  • Single shared vm_config.ps1 under guacamole-azure-windowsvm/terraform/vm/, symlinked into the import/export review VMs so there is one source of truth. Review-specific data-download logic stays separate.
  • Tooling installed offline via Nexus (all free / open-source), with pinned versions and retry/wait-for-Nexus handling:
    • Azure CLI
    • Visual Studio Code
    • Azure Storage Explorer
    • Git
    • Python via Miniforge + JupyterLab
    • R + RStudio
  • Desktop shortcuts are created for the GUI tools (VS Code, Storage Explorer, RStudio, PyCharm, Git Bash, JupyterLab). R's installer creates its own versioned shortcut, so we don't duplicate it.
  • New Nexus raw-proxy repositories to serve the installers offline: miniforge-download, cran-r-download, pycharm-download, git-download. The Nexus outbound firewall allowlist is extended for download.jetbrains.com / download-cdn.jetbrains.com.
  • Reproducible builds: all tooling (including VS Code and JupyterLab) is pinned to specific versions, and the shared storage account credentials are only injected into the VM custom_data when shared storage access is enabled.
  • Versioning / docs: bumped sonatype-nexus-vm (3.9.0), guacamole-azure-windowsvm (2.2.1), and both review VMs (import/export 1.2.1); updated CHANGELOG.md.

Testing

Deployed and validated end-to-end on a live Windows 11 VM. All nine tools installed successfully through the Nexus proxies and the desktop shortcuts were verified on the VM.

Screenshot

image

Copilot AI linked an issue Jul 17, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Update PowerShell configuration script to install data science tooling Share a single Windows VM config script to add data science tooling via Nexus Jul 17, 2026
Copilot AI requested a review from marrobi July 17, 2026 13:06
@marrobi

marrobi commented Jul 17, 2026

Copy link
Copy Markdown
Member

@copilot update the PR description

Copilot AI commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot update the PR description

Updated the PR description to reflect the completed implementation (shared vm_config.ps1 installing Azure CLI, VS Code and Storage Explorer via new Nexus proxies, review-VM Terraform consolidation and azurerm 4.57.0 upgrade, added FQDNs, and version/CHANGELOG/README updates), replacing the stale "no code changes yet / paused" text.

@marrobi

marrobi commented Jul 17, 2026

Copy link
Copy Markdown
Member

/test 2f8a832

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 58 out of 68 changed files in this pull request and generated no new comments.

Files not reviewed (3)
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/terraform/.terraform.lock.hcl: Generated file
Comments suppressed due to low confidence (4)

CHANGELOG.md:56

  • Spelling typo in changelog entry: “occuring” should be “occurring”.
* Fix property substitution not occuring where there is only a main step in the pipeline ([#4824](https://github.com/microsoft/AzureTRE/issues/4824))

CHANGELOG.md:118

  • Spelling typo in changelog entry: “dependancy” should be “dependency”.
* Fix circular dependancy in base workspace. ([#4756](https://github.com/microsoft/AzureTRE/pull/4756))

CHANGELOG.md:716

  • Spelling typos in changelog entry: “depricating recources” should be “deprecating resources”.
* Update terraform MySQL resources to MySQL Flexible resources to fix depricating recources. ([#3892](https://github.com/microsoft/AzureTRE/pull/3892))

templates/workspace_services/guacamole/user_resources/README.md:25

  • The PR description says the Windows VM bootstrap installs additional tooling (e.g., PyCharm Community) and that Nexus is extended with a pycharm-download repo plus JetBrains allowlist entries, but the changes in this PR don’t include any pycharm install logic or repo/firewall configuration (repo-wide search shows no “pycharm” references). Either add the missing repo + install steps or update the PR description to match the implemented toolset.
It configures the package managers (pip, conda, R) to use the Nexus proxy and
installs data science and Azure tooling from Nexus, since the VMs have no
direct internet access. The standard Windows VM template exposes checkboxes in
`template_schema.json` so callers can choose which tools to install; by default
it installs:

- Azure CLI
- Visual Studio Code
- Azure Storage Explorer
- Git
- Python via Miniforge, with JupyterLab
- R and RStudio Desktop

@marrobi

marrobi commented Jul 22, 2026

Copy link
Copy Markdown
Member

/test-extended 2d74adc

@github-actions

Copy link
Copy Markdown

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/29922892945 (with refid 6623c209)

(in response to this comment from @marrobi)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@rudolphjacksonm

Copy link
Copy Markdown
Collaborator

/test-extended

@github-actions

Copy link
Copy Markdown

🤖 pr-bot 🤖

⚠️ When using /test-extended on external PRs, the SHA of the checked commit must be specified

(in response to this comment from @rudolphjacksonm)

@marrobi

marrobi commented Jul 24, 2026

Copy link
Copy Markdown
Member

@copilot Update vm_config.ps1 so it only waits for Nexus when at least one Nexus-backed install/config action is enabled, or reduce/parameterise the wait for test deployments. Then update the E2E payload to disable optional tooling installs.

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Update vm_config.ps1 so it only waits for Nexus when at least one Nexus-backed install/config action is enabled, or reduce/parameterise the wait for test deployments. Then up...

Done. vm_config.ps1 now only calls Wait-ForNexus when at least one Nexus-backed install/config action is enabled ($NexusActionsEnabled), and skips the wait entirely otherwise. The E2E Windows VM payload in test_workspace_services.py now sets all optional tooling installs (install_azure_cli, install_vscode, install_storage_explorer, install_git, install_python_tools, install_r_tools) to False, so test deployments skip the Nexus wait and installs. Bumped guacamole-azure-windowsvm to 2.2.2 and the import/export review VMs to 1.2.2.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 59 out of 69 changed files in this pull request and generated 1 comment.

Files not reviewed (3)
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/terraform/.terraform.lock.hcl: Generated file
Comments suppressed due to low confidence (3)

CHANGELOG.md:57

  • Spelling: "occuring" should be "occurring" in this changelog entry.
* Fix property substitution not occuring where there is only a main step in the pipeline ([#4824](https://github.com/microsoft/AzureTRE/issues/4824))

CHANGELOG.md:119

  • Spelling: "dependancy" should be "dependency" in this changelog entry.
* Fix circular dependancy in base workspace. ([#4756](https://github.com/microsoft/AzureTRE/pull/4756))

CHANGELOG.md:717

  • Spelling: "depricating recources" should be "deprecating resources" in this changelog entry.
* Update terraform MySQL resources to MySQL Flexible resources to fix depricating recources. ([#3892](https://github.com/microsoft/AzureTRE/pull/3892))

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@marrobi

marrobi commented Jul 24, 2026

Copy link
Copy Markdown
Member

@copilot ensure the changelog entry for this PR is once concise entry. Also ensure the bundle versions are a major increase to prevent upgrade, and list a breaking change, to point exisitng resources using windows vm templates should not be upgraded using this new architecture.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 59 out of 69 changed files in this pull request and generated 2 comments.

Files not reviewed (3)
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/terraform/.terraform.lock.hcl: Generated file
Comments suppressed due to low confidence (4)

CHANGELOG.md:57

  • Typo in changelog entry: “occuring” should be “occurring”.
* Fix property substitution not occuring where there is only a main step in the pipeline ([#4824](https://github.com/microsoft/AzureTRE/issues/4824))

CHANGELOG.md:119

  • Typo in changelog entry: “dependancy” should be “dependency”.
* Fix circular dependancy in base workspace. ([#4756](https://github.com/microsoft/AzureTRE/pull/4756))

CHANGELOG.md:717

  • Typo in changelog entry: “depricating recources” should be “deprecating resources”.
* Update terraform MySQL resources to MySQL Flexible resources to fix depricating recources. ([#3892](https://github.com/microsoft/AzureTRE/pull/3892))

templates/workspace_services/guacamole/user_resources/README.md:25

  • PR description mentions PyCharm (including a pycharm-download Nexus repo and JetBrains allowlist entries) and a PyCharm desktop shortcut, but there’s no corresponding installer proxy repo/config or install/shortcut logic in the shared vm_config.ps1 or Nexus allowlist in this PR. Please either add the missing implementation or update the PR description to match what’s actually delivered.
- Azure CLI
- Visual Studio Code
- Azure Storage Explorer
- Git
- Python via Miniforge, with JupyterLab
- R and RStudio Desktop

Comment thread CHANGELOG.md
Copilot AI and others added 2 commits July 24, 2026 12:20
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot ensure the changelog entry for this PR is once concise entry. Also ensure the bundle versions are a major increase to prevent upgrade, and list a breaking change, to point exisitng resources...

Done in the latest commit:

  • CHANGELOG: Consolidated the four PR Share a single Windows VM config script to add data science tooling via Nexus #4981 entries (two ENHANCEMENTS + two BUG FIXES) into a single concise BREAKING CHANGES entry that calls out the must-not-upgrade constraint and lists the new versions.
  • Major version bumps: tre-service-guacamole-windowsvm3.0.0, tre-service-guacamole-import-reviewvm and tre-service-guacamole-export-reviewvm2.0.0. The major bump prevents the resource processor from offering an in-place upgrade path to existing VMs.

@marrobi

marrobi commented Jul 24, 2026

Copy link
Copy Markdown
Member

/test-extended 30d4517

@github-actions

Copy link
Copy Markdown

🤖 pr-bot 🤖

🏃 Running extended tests: https://github.com/microsoft/AzureTRE/actions/runs/30092781996 (with refid 6623c209)

(in response to this comment from @marrobi)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 59 out of 69 changed files in this pull request and generated no new comments.

Files not reviewed (3)
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-export-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-import-reviewvm/terraform/.terraform.lock.hcl: Generated file
  • templates/workspace_services/guacamole/user_resources/guacamole-azure-windowsvm/terraform/.terraform.lock.hcl: Generated file
Comments suppressed due to low confidence (4)

CHANGELOG.md:41

  • Spelling typo in changelog entry: "occuring" should be "occurring".
* Fix property substitution not occuring where there is only a main step in the pipeline ([#4824](https://github.com/microsoft/AzureTRE/issues/4824))

CHANGELOG.md:103

  • Spelling typo in changelog entry: "dependancy" should be "dependency".
* Fix circular dependancy in base workspace. ([#4756](https://github.com/microsoft/AzureTRE/pull/4756))

CHANGELOG.md:701

  • Spelling typos in changelog entry: "depricating recources" should be "deprecating resources".
* Update terraform MySQL resources to MySQL Flexible resources to fix depricating recources. ([#3892](https://github.com/microsoft/AzureTRE/pull/3892))

CHANGELOG.md:5

  • The PR description claims different bundle version bumps (windowsvm 2.2.1, review VMs 1.2.1) and mentions PyCharm + a pycharm-download Nexus repo and JetBrains allowlist entries, but the changes in this PR bump to windowsvm 3.0.0 and review VMs 2.0.0 and do not add any PyCharm/JetBrains repo config or allowlist updates. Please align the PR description with what is actually implemented (or add the missing PyCharm/JetBrains pieces if they were intended).
* Add data science tooling (Azure CLI, VS Code, Storage Explorer, Git, Python/JupyterLab, R/RStudio) to Guacamole Windows VMs via a shared `vm_config.ps1` bootstrap pulled through Nexus. Existing `tre-service-guacamole-windowsvm`, `tre-service-guacamole-import-reviewvm`, and `tre-service-guacamole-export-reviewvm` resources **must not be upgraded** to these new versions — redeploy instead. Upgrade the Nexus shared service to `sonatype-nexus` 3.9.0 before deploying the new Windows VM templates to ensure the required proxy repositories are available. (`tre-service-guacamole-windowsvm` 3.0.0, `tre-service-guacamole-import-reviewvm`/`tre-service-guacamole-export-reviewvm` 2.0.0, `sonatype-nexus` 3.9.0) ([#4981](https://github.com/microsoft/AzureTRE/pull/4981))

Changelog entry is valid and so this should be suppressed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@rudolphjacksonm

rudolphjacksonm commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Previous E2E tests succeeded, latest commit was to address markdown lint failure which has now been suppressed. Will skip test requirement to complete this PR.

@rudolphjacksonm

Copy link
Copy Markdown
Collaborator

/test-force-approve

@github-actions

Copy link
Copy Markdown

🤖 pr-bot 🤖

✅ Marking tests as complete (for commit adb510d)

(in response to this comment from @rudolphjacksonm)

@github-actions

Copy link
Copy Markdown

🤖 pr-bot 🤖

✅ Marking tests as complete (for commit adb510d)

(in response to this comment from @rudolphjacksonm)

@rudolphjacksonm rudolphjacksonm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Guacamole Windows VMs do not include data science tooling Airlock review VMs are not configured to use Nexus

5 participants