Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Travis configuration #406

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
45 changes: 19 additions & 26 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,47 +1,40 @@
language: python
python: 2.7
sudo: false
cache:
pip: true
directories:
- $HOME/buildout-cache
- $HOME/.pylint.d
- eggs

global:
- secure: "nTXMNU2aYbCtvKD74PndF8Xm2h6IvsxPjr6vj45AxjXmaB7Wf5oGi/4b6ObV\nexBtUQkFnc+M0ThGyUznOCPi3YxcbTzsyhQYhHUtmtw/6QRyYgo4E0GLDItU\n37Ff6wpxl3NMMJTvpi6SOVUvIJUl3+cs+4bkMkS48RSFGg2rGmo=" # Coveralls.io token
env:
matrix:
- PLONE_VERSION=4.3
- PLONE_VERSION=5.0
- PLONE_VERSION=5.1
matrix:
fast_finish: true
include:
# pyXX-plone_sunburst,pyXX-plone_barceloneta
- python: "2.7"
env: TOXENV=py27-plone-4.3
- python: "2.7"
env: TOXENV=py27-plone-5.0
- python: "2.7"
env: TOXENV=py27-plone-5.1
- python: "2.7"
env: TOXENV=isort,flake8,docs,

before_install:
- mkdir -p $HOME/buildout-cache/{downloads,eggs,extends}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is what I call a "a test setup from old times"; caching has been around Travis since 2016.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, tell me how to do it better?

if you use buildout and not the standard tools like pip which is known for travis you have to tell them which directory to cache:

https://github.com/plone/plone.api/blob/master/.travis.yml#L3-L6

So the maked line is necessary to be idempotent and deterministic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it isn't; if you check what I did you'll learn how to fix it. we have been using it that way for years and we already had that discussion in bobtemplates.plone as @jensens mentioned.

- mkdir $HOME/.buildout
- echo "[buildout]" > $HOME/.buildout/default.cfg
- echo "download-cache = $HOME/buildout-cache/downloads" >> $HOME/.buildout/default.cfg
- echo "eggs-directory = $HOME/buildout-cache/eggs" >> $HOME/.buildout/default.cfg
- echo "extends-cache = $HOME/buildout-cache/extends" >> $HOME/.buildout/default.cfg
- echo "abi-tag-eggs = true" >> $HOME/.buildout/default.cfg
- cp plone-$PLONE_VERSION.x.cfg buildout.cfg

install:
- travis_retry pip install -U setuptools==33.1.1 tox coveralls coverage

before_script:
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- python bootstrap.py --setuptools-version=$(curl -s https://raw.githubusercontent.com/plone/buildout.coredev/$PLONE_VERSION/requirements.txt | grep setuptools)
- bin/buildout annotate
- bin/buildout

script:
- travis_retry tox
- bin/code-analysis
- bin/test
- bin/pylint --py3k --disable=no-absolute-import src/plone/api || true

after_success:
- coverage combine
- coveralls
- pip install coverage
- coverage run bin/test
- pip install coveralls
- coveralls

notifications:
irc:
Expand Down
30 changes: 11 additions & 19 deletions base.cfg
Original file line number Diff line number Diff line change
@@ -1,26 +1,18 @@
[buildout]
package-name = plone.api
package-extras = [test]
test-eggs =
plone.api

parts +=
omelette
code-analysis
pylint

# network speedup
socket-timeout = 5
allow-hosts =
*.python.org
docutils.sourceforge.net
prdownloads.sourceforge.net
effbot.org
dist.plone.org
[code-analysis]
recipe = plone.recipe.codeanalysis
directory = src/plone/api
pre-commit-hook = True
flake8 = True
return-status-codes = True

[versions]
plone.api =
setuptools = 33.1.1
zc.buildout = 2.8.0

[omelette]
recipe = collective.recipe.omelette
eggs = ${test:eggs}
[pylint]
recipe = zc.recipe.egg
eggs = pylint < 2.0
210 changes: 210 additions & 0 deletions bootstrap.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
##############################################################################
#
# Copyright (c) 2006 Zope Foundation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
"""Bootstrap a buildout-based project

Simply run this script in a directory containing a buildout.cfg.
The script accepts buildout command-line options, so you can
use the -c option to specify an alternate configuration file.
"""

import os
import shutil
import sys
import tempfile

from optparse import OptionParser

__version__ = '2015-07-01'
# See zc.buildout's changelog if this version is up to date.

tmpeggs = tempfile.mkdtemp(prefix='bootstrap-')

usage = '''\
[DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]

Bootstraps a buildout-based project.

Simply run this script in a directory containing a buildout.cfg, using the
Python that you want bin/buildout to use.

Note that by using --find-links to point to local resources, you can keep
this script from going over the network.
'''

parser = OptionParser(usage=usage)
parser.add_option("--version",
action="store_true", default=False,
help=("Return bootstrap.py version."))
parser.add_option("-t", "--accept-buildout-test-releases",
dest='accept_buildout_test_releases',
action="store_true", default=False,
help=("Normally, if you do not specify a --version, the "
"bootstrap script and buildout gets the newest "
"*final* versions of zc.buildout and its recipes and "
"extensions for you. If you use this flag, "
"bootstrap and buildout will get the newest releases "
"even if they are alphas or betas."))
parser.add_option("-c", "--config-file",
help=("Specify the path to the buildout configuration "
"file to be used."))
parser.add_option("-f", "--find-links",
help=("Specify a URL to search for buildout releases"))
parser.add_option("--allow-site-packages",
action="store_true", default=False,
help=("Let bootstrap.py use existing site packages"))
parser.add_option("--buildout-version",
help="Use a specific zc.buildout version")
parser.add_option("--setuptools-version",
help="Use a specific setuptools version")
parser.add_option("--setuptools-to-dir",
help=("Allow for re-use of existing directory of "
"setuptools versions"))

options, args = parser.parse_args()
if options.version:
print("bootstrap.py version %s" % __version__)
sys.exit(0)


######################################################################
# load/install setuptools

try:
from urllib.request import urlopen
except ImportError:
from urllib2 import urlopen

ez = {}
if os.path.exists('ez_setup.py'):
exec(open('ez_setup.py').read(), ez)
else:
exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)

if not options.allow_site_packages:
# ez_setup imports site, which adds site packages
# this will remove them from the path to ensure that incompatible versions
# of setuptools are not in the path
import site
# inside a virtualenv, there is no 'getsitepackages'.
# We can't remove these reliably
if hasattr(site, 'getsitepackages'):
for sitepackage_path in site.getsitepackages():
# Strip all site-packages directories from sys.path that
# are not sys.prefix; this is because on Windows
# sys.prefix is a site-package directory.
if sitepackage_path != sys.prefix:
sys.path[:] = [x for x in sys.path
if sitepackage_path not in x]

setup_args = dict(to_dir=tmpeggs, download_delay=0)

if options.setuptools_version is not None:
setup_args['version'] = options.setuptools_version
if options.setuptools_to_dir is not None:
setup_args['to_dir'] = options.setuptools_to_dir

ez['use_setuptools'](**setup_args)
import setuptools
import pkg_resources

# This does not (always?) update the default working set. We will
# do it.
for path in sys.path:
if path not in pkg_resources.working_set.entries:
pkg_resources.working_set.add_entry(path)

######################################################################
# Install buildout

ws = pkg_resources.working_set

setuptools_path = ws.find(
pkg_resources.Requirement.parse('setuptools')).location

# Fix sys.path here as easy_install.pth added before PYTHONPATH
cmd = [sys.executable, '-c',
'import sys; sys.path[0:0] = [%r]; ' % setuptools_path +
'from setuptools.command.easy_install import main; main()',
'-mZqNxd', tmpeggs]

find_links = os.environ.get(
'bootstrap-testing-find-links',
options.find_links or
('http://downloads.buildout.org/'
if options.accept_buildout_test_releases else None)
)
if find_links:
cmd.extend(['-f', find_links])

requirement = 'zc.buildout'
version = options.buildout_version
if version is None and not options.accept_buildout_test_releases:
# Figure out the most recent final version of zc.buildout.
import setuptools.package_index
_final_parts = '*final-', '*final'

def _final_version(parsed_version):
try:
return not parsed_version.is_prerelease
except AttributeError:
# Older setuptools
for part in parsed_version:
if (part[:1] == '*') and (part not in _final_parts):
return False
return True

index = setuptools.package_index.PackageIndex(
search_path=[setuptools_path])
if find_links:
index.add_find_links((find_links,))
req = pkg_resources.Requirement.parse(requirement)
if index.obtain(req) is not None:
best = []
bestv = None
for dist in index[req.project_name]:
distv = dist.parsed_version
if _final_version(distv):
if bestv is None or distv > bestv:
best = [dist]
bestv = distv
elif distv == bestv:
best.append(dist)
if best:
best.sort()
version = best[-1].version
if version:
requirement = '=='.join((requirement, version))
cmd.append(requirement)

import subprocess
if subprocess.call(cmd) != 0:
raise Exception(
"Failed to execute command:\n%s" % repr(cmd)[1:-1])

######################################################################
# Import and run buildout

ws.add_entry(tmpeggs)
ws.require(requirement)
import zc.buildout.buildout

if not [a for a in args if '=' not in a]:
args.append('bootstrap')

# if -c was provided, we push it back into args for buildout' main function
if options.config_file is not None:
args[0:0] = ['-c', options.config_file]

zc.buildout.buildout.main(args)
shutil.rmtree(tmpeggs)
4 changes: 4 additions & 0 deletions plone-4.3.x.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,7 @@
extends =
https://raw.github.com/collective/buildout.plonetest/master/test-4.3.x.cfg
base.cfg

[versions]
# FIXME
flake8 = 3.3.0
6 changes: 5 additions & 1 deletion plone-5.0.x.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
extends =
https://raw.github.com/collective/buildout.plonetest/master/test-5.0.x.cfg
base.cfg


[versions]
# FIXME: outdated https://dist.plone.org/release/5.0-latest/versions.cfg
setuptools = 33.1.1
zc.buildout = 2.9.5
11 changes: 1 addition & 10 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,7 @@ not_skip=__init__.py

[coverage:run]
branch = True
source =
src

omit =
*.rst
*/doctests/*
*/tests/*
include = src/plone/api/*

[coverage:report]
precision = 2
Expand All @@ -33,9 +27,6 @@ omit =
*/doctests/*
*/tests/*

[coverage:html]
directory = reports/coverage

# When Python 2-3 compatible:
# [bdist_wheel]
# universal = 1
Expand Down