Merge pull request #13 from lizardsystem/reinout-version-upgrades

Version upgrades (work in progress)

.travis.yml

@@ -1,17 +1,17 @@
 language: python
+sudo: false
 python:
-  - 2.6
   - 2.7
-before_install:
-  - sudo apt-get install python-psycopg2
-notifications:
-  irc:
-    channels: "irc.freenode.org#nens"
-    on_success: change
-  email: false
+cache:
+  directories:
+   - eggs
+   - $HOME/.local
 install:
   - deactivate
+  - pip install --user psycopg2
   - python bootstrap.py
   - bin/buildout
 script:
   - bin/test
+after_success:
+  - bin/coveralls

CHANGES.rst

@@ -7,6 +7,25 @@ Changelog for nensskel
 
 - Update packages mitigating the 'ValueError: too many values to unpack'.
 
+- Updated versions, using new lizard5 KGS. You can just as well remove the
+  'extends' line for your own projects, but at least for testing nensskel, I'm
+  keeping it in for now.
+  [reinout]
+
+- Fixed djangorecipe setup. Deprecated options have been removed (fixes
+  #12).
+  [reinout]
+
+- Using ``wsgi.py`` file and the new djangorecipe ``scripts-with-settings`
+  option: this fixes the gunicorn (and celery) setup for modern
+  django/gunicorn combinations.
+  [reinout]
+
+- Improved tests so that more buildout failures show up (like the necessary
+  djangorecipe changes from the previous point). The test setup now also
+  includes coverallls.io integration for coverage reports.
+  [reinout]
+
 
 1.35 (2015-01-21)
 -----------------

README.rst

@@ -1,17 +1,21 @@
 Django code skeletons
 =====================
 
+.. image:: https://travis-ci.org/lizardsystem/nensskel.svg?branch=master
+    :target: https://travis-ci.org/lizardsystem/nensskel
+
+
+.. image:: https://coveralls.io/repos/lizardsystem/nensskel/badge.svg?branch=reinout-version-upgrades&service=github
+  :target: https://coveralls.io/github/lizardsystem/nensskel?branch=reinout-version-upgrades
+
+
+
 ``nensskel`` provides so-called "paster templates" so you can easily create new
 Django applications, websites and python libraries.  They're not fully
 generic, as they originated within one company (see below).
 
-The nensskel script creates a directory structure for you with (some 
-partially filled) basic files (like a ``README.rst`` and a ``setup.py``). 
-Optionally you can use it to automatically create a trunk/tags/branches 
-structure in svn.
-
-.. image:: https://secure.travis-ci.org/lizardsystem/nensskel.png?branch=master
-   :target: http://travis-ci.org/#!/lizardsystem/nensskel
+The nensskel script creates a directory structure for you with (some
+partially filled) basic files (like a ``README.rst`` and a ``setup.py``).
 
 Call the nensskel script to get usage information.
 
@@ -20,7 +24,7 @@ Installation is straightforward with easy_install::
     $ easy_install nensskel
 
 or with pip::
-    
+
     $ pip install nensskel
 
 (Probably you need to run it with sudo on osx/linux).
@@ -33,12 +37,13 @@ Don't forget to update from time to time::
 Available templates
 -------------------
 
-* **basic_package:**    A basic setuptools-enabled package
-* **nens_djangoapp:**   A buildout for nens Django applications
+Three nens-specific templates are included:
+
 * **nens_library:**     A buildout for nens libraries
+
+* **nens_djangoapp:**   A buildout for nens Django applications
+
 * **nens_lizardsite:**  A buildout for nens lizard Django sites
-* **nens_leansite:**  A buildout for nens Django sites (doesn't inherit from other templates)
-* **paste_deploy:**     A web application deployed through paste.deploy
 
 
 Company-centricity
@@ -49,13 +54,15 @@ We set it up the way we like it and we change it the way we like it so it might
 not fit your case 100%.
 
 But it does provide a good example to give you a head start with a completely
-setup Django application/site. For example, the Django sites come with a 
+setup Django application/site. For example, the Django sites come with a
 full-blown nginx config and Django-staticfiles is included for easy css/js setup.
 
 If the available templates don't serve your need, you can do two things:
-1. Clone the repository and edit the template as you like; possibly submit a pull
-request
-2. Make your own template
+
+- Clone the repository and edit the template as you like; possibly submit a
+  pull request
+
+- Make your own template
 
 
 Drop us a line if you need help.
@@ -67,7 +74,7 @@ Example for your own skeleton
 When you want to create your own skeleton, nensskel can be a nice, small
 example.  Just download the .tar.gz and unpack it.
 
-Do the regular ``python bootstrap.py && bin/buildout`` and you're good to go. 
+Do the regular ``python bootstrap.py && bin/buildout`` and you're good to go.
 The nensskel and paster scripts are created in the *bin/* directory.
 
 There is one python script in ``nensskel/*.py`` per template.  The actual

bootstrap.py

@@ -25,7 +25,10 @@
 
 from optparse import OptionParser
 
-tmpeggs = tempfile.mkdtemp()
+__version__ = '2015-07-01'
+# See zc.buildout's changelog if this version is up to date.
+
+tmpeggs = tempfile.mkdtemp(prefix='bootstrap-')
 
 usage = '''\
 [DESIRED PYTHON FOR BUILDOUT] bootstrap.py [options]
@@ -40,8 +43,9 @@
 '''
 
 parser = OptionParser(usage=usage)
-parser.add_option("-v", "--version", help="use a specific zc.buildout version")
-
+parser.add_option("--version",
+                  action="store_true", default=False,
+                  help=("Return bootstrap.py version."))
 parser.add_option("-t", "--accept-buildout-test-releases",
                   dest='accept_buildout_test_releases',
                   action="store_true", default=False,
@@ -59,25 +63,33 @@
 parser.add_option("--allow-site-packages",
                   action="store_true", default=False,
                   help=("Let bootstrap.py use existing site packages"))
+parser.add_option("--buildout-version",
+                  help="Use a specific zc.buildout version")
 parser.add_option("--setuptools-version",
-                  help="use a specific setuptools version")
-
+                  help="Use a specific setuptools version")
+parser.add_option("--setuptools-to-dir",
+                  help=("Allow for re-use of existing directory of "
+                        "setuptools versions"))
 
 options, args = parser.parse_args()
+if options.version:
+    print("bootstrap.py version %s" % __version__)
+    sys.exit(0)
+
 
 ######################################################################
 # load/install setuptools
 
 try:
-    if options.allow_site_packages:
-        import setuptools
-        import pkg_resources
     from urllib.request import urlopen
 except ImportError:
     from urllib2 import urlopen
 
 ez = {}
-exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
+if os.path.exists('ez_setup.py'):
+    exec(open('ez_setup.py').read(), ez)
+else:
+    exec(urlopen('https://bootstrap.pypa.io/ez_setup.py').read(), ez)
 
 if not options.allow_site_packages:
     # ez_setup imports site, which adds site packages
@@ -88,12 +100,19 @@
     # We can't remove these reliably
     if hasattr(site, 'getsitepackages'):
         for sitepackage_path in site.getsitepackages():
-            sys.path[:] = [x for x in sys.path if sitepackage_path not in x]
+            # Strip all site-packages directories from sys.path that
+            # are not sys.prefix; this is because on Windows
+            # sys.prefix is a site-package directory.
+            if sitepackage_path != sys.prefix:
+                sys.path[:] = [x for x in sys.path
+                               if sitepackage_path not in x]
 
 setup_args = dict(to_dir=tmpeggs, download_delay=0)
 
 if options.setuptools_version is not None:
     setup_args['version'] = options.setuptools_version
+if options.setuptools_to_dir is not None:
+    setup_args['to_dir'] = options.setuptools_to_dir
 
 ez['use_setuptools'](**setup_args)
 import setuptools
@@ -110,7 +129,12 @@
 
 ws = pkg_resources.working_set
 
+setuptools_path = ws.find(
+    pkg_resources.Requirement.parse('setuptools')).location
+
+# Fix sys.path here as easy_install.pth added before PYTHONPATH
 cmd = [sys.executable, '-c',
+       'import sys; sys.path[0:0] = [%r]; ' % setuptools_path +
        'from setuptools.command.easy_install import main; main()',
        '-mZqNxd', tmpeggs]
 
@@ -123,11 +147,8 @@
 if find_links:
     cmd.extend(['-f', find_links])
 
-setuptools_path = ws.find(
-    pkg_resources.Requirement.parse('setuptools')).location
-
 requirement = 'zc.buildout'
-version = options.version
+version = options.buildout_version
 if version is None and not options.accept_buildout_test_releases:
     # Figure out the most recent final version of zc.buildout.
     import setuptools.package_index
@@ -167,7 +188,7 @@ def _final_version(parsed_version):
 cmd.append(requirement)
 
 import subprocess
-if subprocess.call(cmd, env=dict(os.environ, PYTHONPATH=setuptools_path)) != 0:
+if subprocess.call(cmd) != 0:
     raise Exception(
         "Failed to execute command:\n%s" % repr(cmd)[1:-1])
 

buildout.cfg

@@ -1,5 +1,6 @@
 [buildout]
-index = http://packages.lizardsystem.nl
+index = https://packages.lizard.net
+extends = https://packages.lizard.net/kgs/latest.cfg
 show-picked-versions = true
 socket-timeout = 1
 find-links =
@@ -17,26 +18,17 @@ eggs =
 [versions]
 # Specific pins
 nensskel =
+setuptools = 18.3.1
 
 # Reported by buildout
-cheetah = 2.4.4
-collective.recipe.omelette = 0.15
-coverage = 3.6
-distribute = 0.6.34
-markdown = 2.2.1
-mock = 1.0.1
-nose = 1.3.0
-paste = 1.7.5.1
-pastedeploy = 1.5.0
-pastescript = 1.7.5
+Cheetah = 2.4.4
+Markdown = 2.6.2
+Paste = 2.0.2
+PasteDeploy = 1.5.2
+PasteScript = 2.0.2
+coveralls = 1.0
+docopt = 0.6.2
 pbp.recipe.noserunner = 0.2.6
-pep8 = 1.4.2
-pkginfo = 0.9.1
-pyflakes = 0.6.1
-setuptools = 15.1
-zc.buildout = 2.3.1
-zc.recipe.egg = 2.0.1
-zest.releaser = 3.43
 
 
 [omelette]
@@ -54,11 +46,12 @@ eggs =
     pep8
     pyflakes
     zest.releaser
+    coveralls
 
 
 [test]
 recipe = pbp.recipe.noserunner
-eggs = ${buildout:eggs}
+eggs =
     nensskel
     nensskel[test]
 defaults =

nensskel/smoketest.rst

@@ -113,12 +113,12 @@ The python library:
     Creating directory .../aaa/bin'.
     ...
     Generated script .../aaa/bin/buildout...
-    >>> output = system('bin/buildout -t 1') # '-t 1' is the socket timeout...
-    >>> if 'Traceback' in output:
-    ...     print output
-    ... else:
-    ...     print "succeeded"
-    succeeded
+    >>> print 'start', system('bin/buildout') #doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+    start...
+    Installing sphinx.
+    ...
+
+(We test for sphinx as that's the last part being installed).
 
 The lizard app:
 
@@ -128,12 +128,10 @@ The lizard app:
     Creating directory .../li-zard/bin'.
     ...
     Generated script .../li-zard/bin/buildout...
-    >>> output = system('bin/buildout -t 1') # '-t 1' is the socket timeout...
-    >>> if 'Traceback' in output:
-    ...     print output
-    ... else:
-    ...     print "succeeded"
-    succeeded
+    >>> print 'start', system('bin/buildout') #doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+    start...
+    Installing sphinx.
+    ...
 
 The lizard site:
 
@@ -144,9 +142,7 @@ The lizard site:
     Creating directory .../nieuwegein/bin'.
     ...
     Generated script .../nieuwegein/bin/buildout...
-    >>> output = system('bin/buildout -t 1') # '-t 1' is the socket timeout...
-    >>> if 'Traceback' in output:
-    ...     print output
-    ... else:
-    ...     print "succeeded"
-    succeeded
+    >>> print 'start', system('bin/buildout') #doctest: +ELLIPSIS, +NORMALIZE_WHITESPACE
+    start...
+    Installing sphinx.
+    ...

nensskel/templates/djangoapp/buildout.cfg_tmpl

@@ -24,13 +24,8 @@ eggs =
 
 
 [versions]
-# Specific pins. We inherit lots of pins from the KGS, so if you add a
-# development egg, you really need to un-pin it here.
+# Specific pins.
 ${project} =
-gp.recipe.node =
-raven =
-zc.buildout =
-zc.recipe.egg =
 
 # Reported by buildout.
 
@@ -81,13 +76,13 @@ paths =
 # See http://jacobian.org/writing/django-apps-with-buildout/
 recipe = djangorecipe
 project = ${package}
-projectegg = ${package}
 settings = testsettings
 test = ${package}
 eggs =
     ${project}
     ${project}[test]
 
+
 [omelette]
 # Creates a nice parts/omelette with our used eggs
 recipe = collective.recipe.omelette