Skip to content

110353 add windows testing to python agent #726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 34 commits into from
Closed

110353 add windows testing to python agent #726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
99e68d9
Adding scripts for testing multiple versions of python on windows
Feb 19, 2020
0fbb2a6
merging master and resolving conflicts
Feb 19, 2020
a69ef7e
Commenting out the other tests
Feb 19, 2020
aa4b9a2
Moving anonymous parralel steps out of parallel step
Feb 19, 2020
af93be4
Testing the difference between \'s and /'s in a powershell label on w…
Feb 19, 2020
c88e611
changing power shell files to bats, and accounting for existing insta…
Feb 19, 2020
bd0af6e
Adding system32 to the environmental path
Feb 20, 2020
45760a8
Adding install chcolatey back to build step
Feb 20, 2020
fa8d3ab
Missed a slash in folder path
Feb 20, 2020
0020034
Fixing 7zip syntax, adding all steps to a central file temporarily
Feb 20, 2020
0c57059
Adding missing parameters, and executing pytest via python
Feb 20, 2020
3e452ae
Adding env refresh between batch calls, include logic for installing …
Feb 20, 2020
6c7c36c
Adding gherkin feature download
Feb 25, 2020
7f2bb57
Adding async module for pytest
Feb 25, 2020
7f86139
Removing extra argument and adding argument for python2
Feb 25, 2020
8525ac9
Adding -vv to pytest to triage skipped tests
Feb 25, 2020
608537b
renaming python.exe to pythonXX.exe to ensure we execute correct vers…
Feb 25, 2020
2731a38
Forcing python to re-install via choco
ElWPenn Feb 25, 2020
37c6b9e
Replacing install module bat with using requirements file
ElWPenn Feb 25, 2020
1d2030c
Removing hyphen from pip install command
ElWPenn Feb 25, 2020
754571f
Removing switch that was breaking move, removing python3 uninstall, a…
ElWPenn Feb 25, 2020
3144df8
Removing version call
ElWPenn Feb 25, 2020
fc07eb6
Adding pytest batch to handle python version arguments
ElWPenn Feb 25, 2020
9b51a72
adding arguments to call for pytest.bat
ElWPenn Feb 25, 2020
1c05ab6
account for python2 requirements for install modules, adjusting scrip…
ElWPenn Feb 25, 2020
07027a1
fixing switch in pip install
ElWPenn Feb 26, 2020
c8d4f48
Removing call from executing pytest so failures are caught when tests…
ElWPenn Feb 26, 2020
ffae864
Modifying batch to not mask failures
ElWPenn Feb 26, 2020
86c5ddb
Adding a dir to see c:\ for python 3.7 to triage why it isn't working
ElWPenn Feb 26, 2020
2522d9d
Getting contents of python directory for debugging
ElWPenn Feb 26, 2020
8608515
Changing python3.7.2 command to python since installing it doesn't fu…
ElWPenn Feb 26, 2020
15aa5e7
removing debugging data, adding logic to skip renaming python.exe if …
ElWPenn Feb 26, 2020
7fdb816
Removing closing SETLOCAL DisableDelayedExpansion statement to preven…
ElWPenn Feb 26, 2020
f306165
Changing python 3.7.2 to use py when executing tests
ElWPenn Feb 26, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
123 changes: 121 additions & 2 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,13 +111,132 @@ pipeline {
name: "Python",
steps: this
)
def mapPatallelTasks = pythonTasksGen.generateParallelTests()
parallel(mapPatallelTasks)
//def mapPatallelTasks = pythonTasksGen.generateParallelTests()
//parallel(mapPatallelTasks)
}
}
}
}
}
stage('Parallel Windows Tests') {
parallel {
stage('Windows Py 2.7.16') {
agent { label 'windows-2019-immutable' }
options { skipDefaultCheckout() }
environment {
PATH = "C:\\Windows\\System32;${env.PATH}";
}
steps {
withGithubNotify(context: 'Test', tab: 'tests') {
deleteDir()
unstash "source"
dir("${BASE_DIR}") {
bat label: 'Execute Tests', script: ".\\tests\\scripts\\configure_windows_instance.bat 2.7.16 python27"
/*
bat label: 'Install Chocolatey', script: ".\\tests\\scripts\\install_chocolatey.bat"
bat label: 'Uninstall Python3', script: "choco uninstall python3"
bat label: 'Install Modules', script: ".\\tests\\scripts\\download_json_schema.bat"
bat label: 'Install Modules', script: ".\\tests\\scripts\\install_modules.bat python"
bat label: 'Execute Tests', script: "pytest"
*/
}
}
}
}
stage('Windows Py 3.5.4') {
agent { label 'windows-2019-immutable' }
options { skipDefaultCheckout() }
environment {
PATH = "C:\\Windows\\System32;${env.PATH}";
}
steps {
withGithubNotify(context: 'Test', tab: 'tests') {
deleteDir()
unstash "source"
dir("${BASE_DIR}") {
bat label: 'Execute Tests', script: ".\\tests\\scripts\\configure_windows_instance.bat 3.5.4 python35"
/*
bat label: 'Install Chocolatey', script: ".\\tests\\scripts\\install_chocolatey.bat"
bat label: 'Install Python3.5', script: "./tests/scripts/install_python.bat 3.5.4"
bat label: 'Install Modules', script: "./tests/scripts/download_json_schema.bat"
bat label: 'Install Modules', script: "./tests/scripts/install_modules.bat py"
bat label: 'Execute Tests', script: "pytest"
*/
}
}
}
}
stage('Windows Py 3.6.7') {
agent { label 'windows-2019-immutable' }
options { skipDefaultCheckout() }
environment {
PATH = "C:\\Windows\\System32;${env.PATH}";
}
steps {
withGithubNotify(context: 'Test', tab: 'tests') {
deleteDir()
unstash "source"
dir("${BASE_DIR}") {
bat label: 'Execute Tests', script: ".\\tests\\scripts\\configure_windows_instance.bat 3.6.7 python36"
/*
bat label: 'Install Chocolatey', script: ".\\tests\\scripts\\install_chocolatey.bat"
bat label: 'Install Python3.6', script: "./tests/scripts/install_python.bat 3.6.7"
bat label: 'Install Modules', script: "./tests/scripts/download_json_schema.bat"
bat label: 'Install Modules', script: "./tests/scripts/install_modules.bat py"
bat label: 'Execute Tests', script: "pytest"
*/
}
}
}
}
stage('Windows Py 3.7.2') {
agent { label 'windows-2019-immutable' }
options { skipDefaultCheckout() }
environment {
PATH = "C:\\Windows\\System32;${env.PATH}";
}
steps {
withGithubNotify(context: 'Test', tab: 'tests') {
deleteDir()
unstash "source"
dir("${BASE_DIR}") {
bat label: 'Execute Tests', script: ".\\tests\\scripts\\configure_windows_instance.bat 3.7.2 py"
/*
bat label: 'Install Chocolatey', script: ".\\tests\\scripts\\install_chocolatey.bat"
bat label: 'Install Python3.7', script: "./tests/scripts/install_python.bat 3.7.2"
bat label: 'Install Modules', script: "./tests/scripts/download_json_schema.bat"
bat label: 'Install Modules', script: "./tests/scripts/install_modules.bat py"
bat label: 'Execute Tests', script: "pytest"
*/
}
}
}
}
stage('Windows Py 3.8.1') {
agent { label 'windows-2019-immutable' }
options { skipDefaultCheckout() }
environment {
PATH = "C:\\Windows\\System32;${env.PATH}";
}
steps {
withGithubNotify(context: 'Test', tab: 'tests') {
deleteDir()
unstash "source"
dir("${BASE_DIR}") {
bat label: 'Execute Tests', script: ".\\tests\\scripts\\configure_windows_instance.bat 3.8.1 python38"
/*
bat label: 'Install Chocolatey', script: ".\\tests\\scripts\\install_chocolatey.bat"
bat label: 'Install Python3.7', script: "./tests/scripts/install_python.bat 3.8.1"
bat label: 'Install Modules', script: "./tests/scripts/download_json_schema.bat"
bat label: 'Install Modules', script: "./tests/scripts/install_modules.bat py"
bat label: 'Execute Tests', script: "pytest"
*/
}
}
}
}
}
}
stage('Building packages') {
options { skipDefaultCheckout() }
when {
Expand Down
8 changes: 8 additions & 0 deletions tests/scripts/configure_windows_instance.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.\tests\scripts\install_chocolatey.bat^
&& refreshenv^
&& .\tests\scripts\install_python.bat %1 %2^
&& refreshenv^
&& %2 -m pip install -r .\tests\requirements\requirements-base.txt^
&& .\tests\scripts\download_json_schema.bat^
&& .\tests\scripts\download_gherkin_features.bat^
&& .\tests\scripts\execute_pytest_windows.bat %1 %2
6 changes: 6 additions & 0 deletions tests/scripts/download_gherkin_features.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
mkdir .\tests\tempFeatures
Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it be possible to add some docs/headers about how this particular script works, what arguments are required and what requirements are needed?

curl https://codeload.github.com/elastic/apm/zip/master -o .\tests\tempFeatures\features.zip
Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is codeload.github.com the official and supported URL?

Copy link
Contributor

@beniwohli beniwohli Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the URL they use when you click on the download button on the website. We use it for a long time already and never had problems (other than the occasional network glitch)

Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not familiar with that particular entrypoint, then I searched and found that's not officially supported but some internal entrypoint which it could potentially be changed. The reference I found is below one:

We could potentially use curl -L https://github.com/elastic/apm/archive/master.zip to fetch the artifact, similar to what we do in the apm-integration-testing, what do you think?

Copy link
Contributor

@beniwohli beniwohli Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, although I don't anticipate them changing those URLs anytime soon, they are mentioned in the docs, even if only as en example response: https://developer.github.com/v3/repos/contents/#get-archive-link

Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear what they do support or not! Anyway, no strong opinion, up to @ElWPenn how to proceed then. I'm glad to know a bit more about this! Thanks Beni for the feedback

7z x .\tests\tempFeatures\features.zip -o.\tests\tempFeatures *.feature -y -r
mkdir .\tests\bdd\features
xcopy .\tests\tempFeatures\apm-master\tests\agents\gherkin-specs\* .\tests\bdd\features /Y /S
del .\tests\tempFeatures\ /F /Q /S
6 changes: 6 additions & 0 deletions tests/scripts/download_json_schema.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
mkdir .\tests\tempSchema
curl https://codeload.github.com/elastic/apm-server/zip/master -o .\tests\tempSchema\master.zip
7z x .\tests\tempSchema\master.zip -o.\tests\tempSchema *.json -y -r
mkdir .\tests\.schemacache
xcopy .\tests\tempSchema\apm-server-master\docs\spec\* .\tests\.schemacache /Y /S
del .\tests\tempSchema\ /F /Q /S
7 changes: 7 additions & 0 deletions tests/scripts/execute_pytest_windows.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SETLOCAL EnableDelayedExpansion
set pythonVersion=%1
set pythonExecutable=%2
set majorVersion=!pythonVersion:~0,1!
set minorVersion=!pythonVersion:~2,1!
if %majorVersion% EQU 2 (set pyArgs=--ignore-glob="*/py3_*.py" --ignore-glob="*/asyncio/*") else ( if %minorVersion% EQU 5 (set pyArgs=--ignore-glob="*/asyncio/*"))
%pythonExecutable% -m pytest --cov --cov-report xml:coverage.xml %pyArgs%
1 change: 1 addition & 0 deletions tests/scripts/install_chocolatey.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this required? IIUC, the choco binary is available for the Windows CI Workers

7 changes: 7 additions & 0 deletions tests/scripts/install_modules.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
SETLOCAL EnableDelayedExpansion
set pythonVersion=%1
set pythonExecutable=%2
set majorVersion=!pythonVersion:~0,1!
echo !majorVersion!
if %majorVersion% EQU 2 (!pythonExecutable! -m pip install -r .\tests\requirements\requirements-base.txt) else (!pythonExecutable! -m pip install -r .\tests\requirements\requirements-base.txt)
SETLOCAL DisableDelayedExpansion
9 changes: 9 additions & 0 deletions tests/scripts/install_python.bat
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
SETLOCAL EnableDelayedExpansion
Copy link
Member

@v1v v1v Feb 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it be possible to add some docs/headers about how this particular script works, what arguments are required and what requirements are needed?

set pythonVersion=%1
set pythonExecutable=%2
set majorVersion=!pythonVersion:~0,1!
set minorVersion=!pythonVersion:~2,1!
if %majorVersion% EQU 3 (choco install python3 --version=%1 -y --force --allow-downgrade) else (choco install python2 -y --force)
if %majorVersion% EQU 3 ( if %minorVersion% NEQ 7 (move c:\%pythonExecutable%\python.exe c:\%pythonExecutable%\%pythonExecutable%.exe)) else (move c:\%pythonExecutable%\python.exe c:\%pythonExecutable%\%pythonExecutable%.exe)
SETLOCAL DisableDelayedExpansion