Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spack Windows support #27021

Merged
merged 23 commits into from
Mar 17, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
9176a1b
Adding basic Windows features (#21259)
Feb 1, 2021
f784400
Workarounds for install errors on Windows (#21890)
BetsyMcPhail Oct 21, 2021
1fa6272
Create rename utility function
BetsyMcPhail Jan 26, 2021
f8f9511
Windows: Symlink support
BetsyMcPhail Oct 22, 2021
1a39e73
Allow 'spack external find' to find executables on the system path (#…
BetsyMcPhail Aug 6, 2021
03a7cc6
Features/windows install (#22204)
Oct 22, 2021
63888df
use pytest stdout/err capture (#22584)
loulawrence Apr 1, 2021
3942672
MSVC Compiler Find and vcvars env integration (#22856)
BetsyMcPhail Oct 22, 2021
4acfd10
Windows: Create installer and environment
loulawrence Jan 8, 2021
efe986a
Packaging of netlib-lapack for windows (#24993)
May 17, 2021
8a9a547
Windows: VS and CMake support
BetsyMcPhail Jun 10, 2021
472e461
Use Python's zipfile, if available (#24556)
BetsyMcPhail Jun 28, 2021
0f3290d
Relocate spack_cmd and scripts from installer to root bin (#24651)
johnwparent Jul 7, 2021
b36428a
Add Github Actions for Windows (#24504)
johnwparent Jan 25, 2022
c8ffc12
Spack on Windows package ports
johnwparent Oct 22, 2021
0b94b8d
Add Windows to platform and target changes
johnwparent Oct 22, 2021
2a454bc
Update tests support for Windows
BetsyMcPhail Oct 21, 2021
88d7119
Expand external find for Windows (#27588)
johnwparent Mar 15, 2022
725ccf6
Path handling (#28402)
johnwparent Jan 21, 2022
606936a
"spack commands --update-completion"
johnwparent Jan 24, 2022
8212765
Failed _write now track that the DB is inconsistent
scheibelp Feb 4, 2022
5e27e36
Windows Support: Testing Suite integration
johnwparent Mar 16, 2022
872d29e
PWSH Support
johnwparent Mar 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/execute_installer.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
$ proc = Start-Process ${{ env.spack_installer }}\spack.exe "/install /quiet" -Passthru
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many of these files are missing the license information at the top, not sure if we need to add this to the license checker or not

$handle = $proc.Handle # cache proc.Handle
$proc.WaitForExit();

if ($proc.ExitCode -ne 0) {
Write-Warning "$_ exited with status code $($proc.ExitCode)"
}
16 changes: 16 additions & 0 deletions .github/workflows/setup_git.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# (c) 2021 Lawrence Livermore National Laboratory

Set-Location spack

git config --global user.email "spack@example.com"
git config --global user.name "Test User"

if ($(git branch --show-current) -ne "develop")
{
git branch develop origin/develop
}

if ($(git branch --show-current) -ne "features/windows-support")
{
git branch features/windows-support origin/features/windows-support
}
4 changes: 4 additions & 0 deletions .github/workflows/system_shortcut_check.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
param ($systemFolder, $shortcut)

$start = [System.Environment]::GetFolderPath("$systemFolder")
Invoke-Item "$start\Programs\Spack\$shortcut"
184 changes: 184 additions & 0 deletions .github/workflows/windows_python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,184 @@
name: windows tests
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alalazo I wonder if we could combine a lot of these platform-specific workflows into a single workflow with a matrix over different platforms. There shouldn't be a ton of differences between platforms other than installing dependencies, right?

Copy link
Contributor Author

@johnwparent johnwparent Jan 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to take a pass at this in this PR if time permits, or at a later date. When implementing I initially created a prototype to that effect that I can publish if need be.


on:
push:
branches:
- develop
- releases/**
pull_request:
branches:
- develop
- releases/**
defaults:
run:
shell:
powershell Invoke-Expression -Command ".\share\spack\qa\windows_test_setup.ps1"; {0}
jobs:
validate:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python Packages
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade vermin
- name: vermin (Spack's Core)
run: vermin --backport argparse --backport typing -t='2.7-' -t='3.5-' -v spack/lib/spack/spack/ spack/lib/spack/llnl/ spack/bin/
- name: vermin (Repositories)
run: vermin --backport argparse --backport typing -t='2.7-' -t='3.5-' -v spack/var/spack/repos
# Run style checks on the files that have been changed
style:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six setuptools flake8 isort>=4.3.5 mypy>=0.800 black pywin32 types-python-dateutil
- name: Create local develop
run: |
.\spack\.github\workflows\setup_git.ps1
- name: Run style tests
run: |
spack style
- name: Verify license headers
run: |
python spack\bin\spack license verify
unittest:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six pywin32 setuptools codecov coverage
- name: Create local develop
run: |
.\spack\.github\workflows\setup_git.ps1
- name: Unit Test
run: |
echo F|xcopy .\spack\share\spack\qa\configuration\windows_config.yaml $env:USERPROFILE\.spack\windows\config.yaml
spack unit-test --verbose --ignore=lib/spack/spack/test/cmd
unittest-cmd:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six pywin32 setuptools codecov coverage
- name: Create local develop
run: |
.\spack\.github\workflows\setup_git.ps1
- name: Command Unit Test
run: |
echo F|xcopy .\spack\share\spack\qa\configuration\windows_config.yaml $env:USERPROFILE\.spack\windows\config.yaml
spack unit-test lib/spack/spack/test/cmd --verbose
buildtest:
runs-on: windows-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six pywin32 setuptools codecov coverage
- name: Build Test
run: |
spack compiler find
echo F|xcopy .\spack\share\spack\qa\configuration\windows_config.yaml $env:USERPROFILE\.spack\windows\config.yaml
spack external find cmake
spack external find ninja
spack install abseil-cpp
generate-installer-test:
runs-on: windows-latest
steps:
- name: Disable Windows Symlinks
run: |
git config --global core.symlinks false
shell:
powershell
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six pywin32 setuptools codecov coverage
- name: Add Light and Candle to Path
run: |
$env:WIX >> $GITHUB_PATH
- name: Run Installer
run: |
.\spack\share\spack\qa\setup_spack.ps1
spack make-installer -s spack -g SILENT pkg
echo "installer_root=$((pwd).Path)" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf8 -Append
env:
ProgressPreference: SilentlyContinue
- uses: actions/upload-artifact@v2
with:
name: Windows Spack Installer Bundle
path: ${{ env.installer_root }}\pkg\Spack.exe
- uses: actions/upload-artifact@v2
with:
name: Windows Spack Installer
path: ${{ env.installer_root}}\pkg\Spack.msi
execute-installer:
needs: generate-installer-test
runs-on: windows-latest
defaults:
run:
shell: pwsh
steps:
- uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install Python packages
run: |
python -m pip install --upgrade pip six pywin32 setuptools codecov coverage
- name: Setup installer directory
run: |
mkdir -p spack_installer
echo "spack_installer=$((pwd).Path)\spack_installer" | Out-File -FilePath $Env:GITHUB_ENV -Encoding utf8 -Append
- uses: actions/download-artifact@v2
with:
name: Windows Spack Installer Bundle
path: ${{ env.spack_installer }}
- name: Execute Bundled Installer
run: |
$proc = Start-Process ${{ env.spack_installer }}\spack.exe "/install /quiet" -Passthru
$handle = $proc.Handle # cache proc.Handle
$proc.WaitForExit();
$LASTEXITCODE
env:
ProgressPreference: SilentlyContinue
- uses: actions/download-artifact@v2
with:
name: Windows Spack Installer
path: ${{ env.spack_installer }}
- name: Execute MSI
run: |
$proc = Start-Process ${{ env.spack_installer }}\spack.msi "/quiet" -Passthru
$handle = $proc.Handle # cache proc.Handle
$proc.WaitForExit();
$LASTEXITCODE
20 changes: 20 additions & 0 deletions bin/haspywin.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import subprocess
import sys


def getpywin():
try:
import win32con # noqa
except ImportError:
subprocess.check_call(
[sys.executable, "-m", "pip", "-q", "install", "--upgrade", "pip"])
subprocess.check_call(
[sys.executable, "-m", "pip", "-q", "install", "pywin32"])


if __name__ == '__main__':
getpywin()
Loading