Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pr 280 support windows #282

Merged
merged 27 commits into from
Jul 6, 2015
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
94e975f
Add Appveyor support
mdboom Jun 25, 2015
d9450cf
Fix up color printing on Windows
pv Jun 27, 2015
624009d
Allow multi-argument range spec for Git
pv Jun 27, 2015
913851b
Fix up util.check_output + util.which on Windows
pv Jun 27, 2015
65ac571
Fix up executable discovery for virtualenv & conda on Windows
pv Jun 27, 2015
efbde6f
Fix up memory benchmark test on Windows
pv Jun 27, 2015
7e5f51e
Fix issues in test_environment on windows
pv Jun 27, 2015
9344e17
Use colorama instead of psutil for tests, because psutil is not pure-…
pv Jun 27, 2015
7fa11ff
Work around the 260 char PATH_MAX on Windows
pv Jun 27, 2015
1b96efc
Fix up test failure in test_workflow on 32-bit Python 2.x
pv Jun 27, 2015
2f85a69
Fix right-aligning of benchmark results; Windows cmd terminal wraps o…
pv Jun 27, 2015
1362881
Fix multiprocessing test failures, when run via python setup.py test …
pv Jun 27, 2015
5fa41ed
Ensure setup.py test passes the given extra arguments to pytest
pv Jun 27, 2015
6d48594
Run pytest in appveyor with --tb=native to avoid pytest bug
pv Jun 27, 2015
1d84c83
Try harder to remove directories on Windows
pv Jun 27, 2015
c74a558
Fix division by zero in asv continuous
pv Jun 27, 2015
413fe9d
Disable colorama colors on Windows -- too many failure modes in practice
pv Jun 27, 2015
b685890
Mark a failing test on Windows/Appveyor as xfail
pv Jun 28, 2015
f353c7b
Skip multiprocessing using tests when run from py.test on Windows
pv Jun 28, 2015
cc6acca
Convert xrange to while loop, because xrange on 32-bit Python 2.x can…
pv Jun 28, 2015
35d8c99
Tell jquery to parse incoming data as json, for json files (asv previ…
pv Jun 28, 2015
9eda5f5
Fix up serving long path names on Windows for asv preview
pv Jun 28, 2015
aae7a03
Refactor Conda/Virtualenv by moving common helper methods to Environm…
pv Jun 28, 2015
0d2a125
Enable running all test_environment tests if conda is available
pv Jun 28, 2015
0f0ef02
Mark hglib requiring test as skipif rather than xfail
pv Jun 28, 2015
53457ec
Add test checking check_presence doesn't return obvious false negatives
pv Jun 28, 2015
d880953
Add comment explaining not using xrange + make Environment.setup priv…
pv Jul 6, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .continuous-integration/appveyor/install-miniconda.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Sample script to install anaconda under windows
# Authors: Stuart Mumford
# Borrwed from: Olivier Grisel and Kyle Kastner
# License: BSD 3 clause

$MINICONDA_URL = "http://repo.continuum.io/miniconda/"

function DownloadMiniconda ($version, $platform_suffix) {
$webclient = New-Object System.Net.WebClient
$filename = "Miniconda-" + $version + "-Windows-" + $platform_suffix + ".exe"

$url = $MINICONDA_URL + $filename

$basedir = $pwd.Path + "\"
$filepath = $basedir + $filename
if (Test-Path $filename) {
Write-Host "Reusing" $filepath
return $filepath
}

# Download and retry up to 3 times in case of network transient errors.
Write-Host "Downloading" $filename "from" $url
$retry_attempts = 2
for($i=0; $i -lt $retry_attempts; $i++){
try {
$webclient.DownloadFile($url, $filepath)
break
}
Catch [Exception]{
Start-Sleep 1
}
}
if (Test-Path $filepath) {
Write-Host "File saved at" $filepath
} else {
# Retry once to get the error message if any at the last try
$webclient.DownloadFile($url, $filepath)
}
return $filepath
}

function InstallMiniconda ($python_version, $architecture, $python_home) {
Write-Host "Installing miniconda" $python_version "for" $architecture "bit architecture to" $python_home
if (Test-Path $python_home) {
Write-Host $python_home "already exists, skipping."
return $false
}
if ($architecture -eq "x86") {
$platform_suffix = "x86"
} else {
$platform_suffix = "x86_64"
}
$filepath = DownloadMiniconda $python_version $platform_suffix
Write-Host "Installing" $filepath "to" $python_home
$args = "/InstallationType=AllUsers /S /AddToPath=1 /RegisterPython=1 /D=" + $python_home
Write-Host $filepath $args
Start-Process -FilePath $filepath -ArgumentList $args -Wait -Passthru
#Start-Sleep -s 15
if (Test-Path C:\conda) {
Write-Host "Miniconda $python_version ($architecture) installation complete"
} else {
Write-Host "Failed to install Python in $python_home"
Exit 1
}
}

function main () {
InstallMiniconda $env:MINICONDA_VERSION $env:PLATFORM $env:PYTHON
}

main
47 changes: 47 additions & 0 deletions .continuous-integration/appveyor/windows_sdk.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
:: To build extensions for 64 bit Python 3, we need to configure environment
:: variables to use the MSVC 2010 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 4 (SDK v7.1)
::
:: To build extensions for 64 bit Python 2, we need to configure environment
:: variables to use the MSVC 2008 C++ compilers from GRMSDKX_EN_DVD.iso of:
:: MS Windows SDK for Windows 7 and .NET Framework 3.5 (SDK v7.0)
::
:: 32 bit builds do not require specific environment configurations.
::
:: Note: this script needs to be run with the /E:ON and /V:ON flags for the
:: cmd interpreter, at least for (SDK v7.0)
::
:: More details at:
:: https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
:: http://stackoverflow.com/a/13751649/163740
::
:: Author: Olivier Grisel
:: License: CC0 1.0 Universal: http://creativecommons.org/publicdomain/zero/1.0/
@ECHO OFF

SET COMMAND_TO_RUN=%*
SET WIN_SDK_ROOT=C:\Program Files\Microsoft SDKs\Windows

SET MAJOR_PYTHON_VERSION="%PYTHON_VERSION:~0,1%"
IF %MAJOR_PYTHON_VERSION% == "2" (
SET WINDOWS_SDK_VERSION="v7.0"
) ELSE IF %MAJOR_PYTHON_VERSION% == "3" (
SET WINDOWS_SDK_VERSION="v7.1"
) ELSE (
ECHO Unsupported Python version: "%MAJOR_PYTHON_VERSION%"
EXIT 1
)

IF "%PYTHON_ARCH%"=="64" (
ECHO Configuring Windows SDK %WINDOWS_SDK_VERSION% for Python %MAJOR_PYTHON_VERSION% on a 64 bit architecture
SET DISTUTILS_USE_SDK=1
SET MSSdk=1
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Setup\WindowsSdkVer.exe" -q -version:%WINDOWS_SDK_VERSION%
"%WIN_SDK_ROOT%\%WINDOWS_SDK_VERSION%\Bin\SetEnv.cmd" /x64 /release
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
) ELSE (
ECHO Using default MSVC build environment for 32 bit architecture
ECHO Executing: %COMMAND_TO_RUN%
call %COMMAND_TO_RUN% || EXIT 1
)
47 changes: 47 additions & 0 deletions appveyor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# AppVeyor.com is a Continuous Integration service to build and run tests under
# Windows

environment:

global:
PYTHON: "C:\\conda"
MINICONDA_VERSION: "latest"
CMD_IN_ENV: "cmd /E:ON /V:ON /C .\\.continuous-integration\\appveyor\\windows_sdk.cmd"

matrix:

- PYTHON_VERSION: "2.6"
platform: x64
PYTHON_ARCH: "64"
- PYTHON_VERSION: "2.7"
platform: x64
PYTHON_ARCH: "64"
- PYTHON_VERSION: "2.7"
platform: x86
PYTHON_ARCH: "32"
- PYTHON_VERSION: "3.4"
platform: x64
PYTHON_ARCH: "64"

install:
# Install miniconda using a powershell script.
- "powershell .continuous-integration/appveyor/install-miniconda.ps1"
- "SET PATH=%PYTHON%;%PYTHON%\\Scripts;%PATH%"

# Install the build and runtime dependencies of the project.
- "conda update --yes conda"
# Create a conda environment
- "conda create -q --yes -n test python=%PYTHON_VERSION%"
- "activate test"

# Check that we have the expected version of Python
- "python --version"

# Install specified version of dependencies
- "conda install -q --yes six pytest"

# Not a .NET project
build: false

test_script:
- "%CMD_IN_ENV% python setup.py test -a --tb=native"
8 changes: 7 additions & 1 deletion asv/benchmarks.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@
from . import util


WIN = (os.name == "nt")


# Can't use benchmark.__file__, because that points to the compiled
# file, so it can't be run by another version of Python.
BENCHMARK_RUN_SCRIPT = os.path.join(
Expand Down Expand Up @@ -75,7 +78,10 @@ def run_benchmark(benchmark, root, env, show_stderr=False, quick=False,
log.info(initial_message)

def log_result(msg):
padding = " "*(util.get_terminal_width() - len(initial_message) - 14 - 1 - len(msg))
padding_length = util.get_terminal_width() - len(initial_message) - 14 - 1 - len(msg)
if WIN:
padding_length -= 1
padding = " "*padding_length
log.add(" {0}{1}".format(padding, msg))

with log.indent():
Expand Down
9 changes: 8 additions & 1 deletion asv/commands/continuous.py
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,14 @@ def run(cls, conf, branch=None, base=None, factor=2.0, show_stderr=False, bench=
table = []
slowed_down = False
for name, benchmark in six.iteritems(all_benchmarks):
change = after[name] / before[name]
if before[name] == 0:
if after[name] == 0:
change = 1.0
else:
change = float('inf')
else:
change = after[name] / before[name]

if change > factor or change < 1.0 / factor:
table.append(
(change, before[name], after[name], name, benchmark))
Expand Down
2 changes: 1 addition & 1 deletion asv/commands/publish.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ def run(cls, conf):
log.set_nitems(5)

if os.path.exists(conf.html_dir):
shutil.rmtree(conf.html_dir)
util.long_path_rmtree(conf.html_dir)

benchmarks = Benchmarks.load(conf)

Expand Down
7 changes: 5 additions & 2 deletions asv/console.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
from __future__ import (absolute_import, division, print_function,
unicode_literals)

import os
import warnings
import codecs
import contextlib
import locale
Expand All @@ -18,6 +20,8 @@
import six
from six.moves import xrange, input

WIN = (os.name == "nt")


def isatty(file):
"""
Expand Down Expand Up @@ -168,11 +172,10 @@ def color_print(*args, **kwargs):
"""

file = kwargs.get('file', sys.stdout)

end = kwargs.get('end', '')

write = file.write
if isatty(file):
if isatty(file) and not WIN:
for i in xrange(0, len(args), 2):
msg = args[i]
if i + 1 == len(args):
Expand Down
4 changes: 2 additions & 2 deletions asv/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,7 @@ def create(self):

if not self.check_presence():
if os.path.exists(self._path):
shutil.rmtree(self._path)
util.long_path_rmtree(self._path)

if not os.path.exists(self._env_dir):
try:
Expand All @@ -279,7 +279,7 @@ def create(self):
except:
log.error("Failure creating environment for {0}".format(self.name))
if os.path.exists(self._path):
shutil.rmtree(self._path)
util.long_path_rmtree(self._path)
raise

self.save_info_file(self._path)
Expand Down
33 changes: 26 additions & 7 deletions asv/plugins/conda.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@
from .. import util


WIN = (os.name == "nt")


class Conda(environment.Environment):
"""
Manage an environment using conda.
Expand Down Expand Up @@ -75,8 +78,9 @@ def get_environments(cls, conf, python):
def check_presence(self):
if not super(Conda, self).check_presence():
return False
for fn in ['pip', 'python']:
if not os.path.isfile(os.path.join(self._path, 'bin', fn)):
for executable in ['pip', 'python']:
exe = self._find_executable(executable)
if not os.path.isfile(exe):
return False
try:
self._run_executable('python', ['-c', 'pass'])
Expand All @@ -102,9 +106,9 @@ def _setup(self):
'pip'])

log.info("Installing requirements for {0}".format(self.name))
self._install_requirements()
self._install_requirements(conda)

def _install_requirements(self):
def _install_requirements(self, conda):
self.install('wheel')

if self._requirements:
Expand All @@ -117,11 +121,26 @@ def _install_requirements(self):
args.append("{0}={1}".format(key, val))
else:
args.append(key)
self._run_executable('conda', args)

util.check_output([conda] + args)

def _find_executable(self, executable):
"""Find an executable in the environment"""
if WIN:
executable += ".exe"

exe = os.path.join(self._path, 'Scripts', executable)
if os.path.isfile(exe):
return exe
exe = os.path.join(self._path, executable)
if os.path.isfile(exe):
return exe
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe move this to util or a base class, since it's repeated in virtualenv.py?


return os.path.join(self._path, 'bin', executable)

def _run_executable(self, executable, args, **kwargs):
return util.check_output([
os.path.join(self._path, 'bin', executable)] + args, **kwargs)
exe = self._find_executable(executable)
return util.check_output([exe] + args, **kwargs)

def install(self, package):
log.info("Installing into {0}".format(self.name))
Expand Down
2 changes: 1 addition & 1 deletion asv/plugins/git.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def get_date(self, hash):
def get_hashes_from_range(self, range_spec):
args = ['log', '--quiet', '--first-parent', '--format=format:%H']
if range_spec != "":
args += [range_spec]
args += range_spec.split()
output = self._run_git(args, valid_return_codes=(0, 1), dots=False)
return output.strip().split()

Expand Down
Loading