Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Rewrite packaging system, rip out Mercurial #63

Merged
merged 19 commits into from

2 participants

@kini
Owner

I've considerably simplified the packaging system. The new SPKG is
uncompressed, and has a Mercurial repository tracking spkg-install et
al. like all other SPKGs do. The src/ directory consists of a bunch of
source tarballs of dependencies, an sdist tarball of sagenb itself,
and (optionally) a copy of the git repository of sagenb suitable for
beginning development from.

References to Mercurial, dependencies on Mercurial tools, etc. are also
removed.

kini added some commits
@kini kini Fix stripping 62099fb
@kini kini Clean up setup.py bfdffa0
@kini kini Switch MANIFEST.in stuff to git 1237b7b
@kini kini Add some scripts to automate packaging of deps
depdist.sh will be used to create a dist/ directory which should be
moved to the sagenb spkg directory and renamed to src/ .

fetch_deps.py is a helper script for depdist.sh which uses distribute
internals to fetch package tarballs from PyPI.
38327ca
@kini kini Put fetch_deps in util/ 74a53e4
@kini kini Moved SPKG.txt to the SPKG's own repo 83c7fab
@kini kini Delete old packaging system and some old files 9ebcc69
@kini kini Rename depdist.sh to dist.sh; sanitize git repo f37537b
@kini kini Remove outdated Makefile 97a817f
@kini kini Fix packaging of git repo
`setup.py sdist` doesn't copy empty directories, as I found out after
a lot of head scratching. This breaks git repos, so we can't trust sdist
to copy our git repo into the source tarball of sagenb. So we'll store
the git repo separately as sagenb.git , and move it into place when
running spkg-install.
0538713
@kini kini Make sure the staging area is clean too 8f14524
@kini kini Add sagemath/sagenb github remote to the new repo 600bd94
@kini kini Remove hg-git dependencies f0ecab2
@kini kini Merge branch 'master' into packaging
Conflicts:
	Makefile
efe5ac9
@kini kini referenced this pull request
Closed

Jmol enhancements #61

@kini
Owner

Here's an example of a sagenb SPKG created by dist.sh after these commits: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

You will need to take a look at it in order to review this branch because it contains the new SPKG repo I made for sagenb.

@kini kini Fix github remote tracking
Apparently you can't set a branch to track a remote tracking branch that
you haven't fetched yet.
9143deb
@kini
Owner

SPKG updated according to the latest commit: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg (same link)

@jhpalmieri

Would it make any sense to omit the git repository and instead document how to install it for people who want to work on sagenb development?

@kini
Owner

In my opinion, yes, absolutely. Sage currently ships its repositories inside both source and binary tarballs, which is unorthodox to say the least. The stated goal is to make the barrier of entry to development extremely low in order to gather mathematicians who are not experienced software engineers and open-source development collaborators, but who nonetheless have been hacking on Sage for their own purposes/research, and entice them to give us their code by making it extremely easy to do so. But I don't think this goal makes much sense in the context of sagenb, which is a web app and doesn't really seek contributions from inexperienced programmers who are experts in some other field.

kini added some commits
@kini kini Insert old advice from SPKG.txt
This old advice for release management of sagenb was in the SPKG.txt and
was generally out of date. Since the SPKG will now only contain a bunch
of tarballs, a lot of those instructions don't make much sense there, so
I've moved them to the sagenb README for now.
99c95c9
@kini kini Fix dumb oversight 5c5802a
@kini kini Make packaging the git repo optional
New behavior: `./dist.sh` prepares tarballs of sagenb and its
dependencies in dist/ , and `./dist.sh -g` additionally places an
optimized copy of the git repository in dist/
3afe890
@kini
Owner

OK, so packaging the git repo is now optional. Pass -g to dist.sh if you want a git repo.

New SPKG is up at the same URL: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

@jhpalmieri

When I run dist.sh, I see the following:

Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!

Then everything seems to get downloaded successfully. I also see

warning: sdist: standard file not found: should have one of README, README.txt

I have no idea what this means.

Otherwise, everything looks good, and the new SPKG.txt file has good directions for how to use dist.sh. Should SPKG.txt list the non-included dependencies also, like Python, to record where it should go in SAGE_ROOT/spkg/standard/deps?

@jhpalmieri

Oh, it might be a good idea to provide instructions (in SPKG.txt? at the top of dist.sh?) for how to use the git repository in the directory sagenb.git, if you run dist.sh -g. (I think you should unpack sagenb-0.9.1.tar.gz and then move sagenb.git to sagenb-0.9.1/.git. Is that right?)

@kini
Owner
@jhpalmieri

Those don't appear on a stable connection, such as on sage.math.

I just tested, and I also see them on sage.math (although with "Errno 111" instead of "Errno 61"). Also, they are completely reproducible: I see four of these "Download error" messages every time I run the script. Maybe you could add a print statement print "Fetching {0} ...".format(pkg) at the start of the loop, so that the later message "Fetched ..." will reassure the user that the desired package was retrieved successfully.

@kini
Owner
@kini
Owner
@jhpalmieri

No, the instructions are correct - spkg-install will install the git repo correctly if sagenb.git is present in the src directory.

Oh, right. Sorry.

@kini
Owner

OK, I've made the changes you suggested - the ones to the dist scripts in the above commit, and the ones to SPKG.txt in the new SPKG I've just uploaded to the same URL as before: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

@jhpalmieri

I think this looks good now. I've reviewed the changes and tried out dist.sh. Everything seems to work. Does that suffice for a positive review?

@kini
Owner

Yup! Thanks for helping me out with this!

@kini kini merged commit b0ce4e7 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 12, 2012
  1. @kini

    Fix stripping

    kini authored
  2. @kini

    Clean up setup.py

    kini authored
Commits on Jun 13, 2012
  1. @kini

    Switch MANIFEST.in stuff to git

    kini authored
  2. @kini

    Add some scripts to automate packaging of deps

    kini authored
    depdist.sh will be used to create a dist/ directory which should be
    moved to the sagenb spkg directory and renamed to src/ .
    
    fetch_deps.py is a helper script for depdist.sh which uses distribute
    internals to fetch package tarballs from PyPI.
  3. @kini

    Put fetch_deps in util/

    kini authored
  4. @kini
  5. @kini
  6. @kini
  7. @kini

    Remove outdated Makefile

    kini authored
  8. @kini

    Fix packaging of git repo

    kini authored
    `setup.py sdist` doesn't copy empty directories, as I found out after
    a lot of head scratching. This breaks git repos, so we can't trust sdist
    to copy our git repo into the source tarball of sagenb. So we'll store
    the git repo separately as sagenb.git , and move it into place when
    running spkg-install.
  9. @kini
  10. @kini
  11. @kini

    Remove hg-git dependencies

    kini authored
Commits on Jun 14, 2012
  1. @kini

    Merge branch 'master' into packaging

    kini authored
    Conflicts:
    	Makefile
  2. @kini

    Fix github remote tracking

    kini authored
    Apparently you can't set a branch to track a remote tracking branch that
    you haven't fetched yet.
Commits on Jun 15, 2012
  1. @kini

    Insert old advice from SPKG.txt

    kini authored
    This old advice for release management of sagenb was in the SPKG.txt and
    was generally out of date. Since the SPKG will now only contain a bunch
    of tarballs, a lot of those instructions don't make much sense there, so
    I've moved them to the sagenb README for now.
Commits on Jun 16, 2012
  1. @kini

    Fix dumb oversight

    kini authored
  2. @kini

    Make packaging the git repo optional

    kini authored
    New behavior: `./dist.sh` prepares tarballs of sagenb and its
    dependencies in dist/ , and `./dist.sh -g` additionally places an
    optimized copy of the git repository in dist/
Commits on Jun 20, 2012
  1. @kini

    Clearer status messages

    kini authored
This page is out of date. Refresh to see the latest.
View
74 .hgignore
@@ -1,74 +0,0 @@
-# Boring file regexps:
-\.scssc$
-\.sassc$
-\.git$
-\.gitignore$
-\.hi$
-\.mtl$
-\.obj$
-\.db$
-\.o$
-\.o\.cmd$
-\.ko$
-\.ko\.cmd$
-\.mod\.c$
-(^|/)\.tmp_versions($|/)
-(^|/)CVS($|/)
-(^|/)RCS($|/)
-~$
-#(^|/)\.[^/]
-(^|/)_darcs($|/)
-\.tmp$
-\.hg$
-\.rej$
-\.sobj$
-\.bak$
-\.BAK$
-\.dylib$
-\.fig$
-\.eps$
-\.cpp$
-\.orig$
-\.patch$
-(^|/)vssver\.scc$
-\.sw.$
-(^|/)MT($|/)
-(^|/)\{arch\}($|/)
-(^|/).arch-ids($|/)
-(^|/),
-\.class$
-\.prof$
-(^|/)\.DS_Store$
-(^|/)BitKeeper($|/)
-(^|/)ChangeSet($|/)
-(^|/)\.svn($|/)
-\.py[co]$
-\#
-\.cvsignore$
-(^|/)Thumbs\.db$
-^\.gitk-tmp
-# sage files to ignore:
-(^|/).doctest*
-^dist($|/)
-^notes($|/)
-(^|/).old($|/)
-(^|/).doctest($|/)
-(^|/)deleted($|/)
-^build($|/)
-^mirror($|/)
-^new($|/)
-^commit($|/)
-^export($|/)
-MANIFEST
-PKG-INFO
-pull
-push
-release_notes.txt
-sage_notebook*
-\.sagenb
-^sass/src/.sass-cache
-sagenb/data/jmol
-^sagenb.egg-info
-setup.cfg
-sagenb/data/jsmath/fonts
-sagenb/data/MathJax
View
16 MANIFEST.in
@@ -1,12 +1,8 @@
-recursive-include sagenb *
-recursive-include sass *
-recursive-include util *
+graft sagenb
+graft sass
+graft util
prune sass/src/.sass-cache
-recursive-include flask_version *
-include *.txt Makefile MANIFEST.in .hgignore .hgtags sdist spkg-dist SPKG.txt go .gitignore README.rst
-recursive-include .hg *
-prune .hg/git
-prune .hg/patches
-exclude .hg/git-*
-exclude .hg/hgrc
+graft flask_version
+include *.txt MANIFEST.in .gitignore dist.sh README.rst
+prune .git
global-exclude *.pyc *.pyo *.orig *.rej *~ \#* *\# *.sassc *.scssc *.DS_Store
View
37 Makefile
@@ -1,37 +0,0 @@
-# -*- makefile -*-
-# Makefile for the Sage Notebook project (http://nb.sagemath.org/).
-# GNU make help: http://www.gnu.org/software/make/manual
-
-SAGE = sage
-HG = $(SAGE) -hg
-PYTHON = $(SAGE) -python
-SETUP = $(PYTHON) setup.py
-
-install:
- $(SETUP) install
-
-develop:
- $(SETUP) develop
-
-update:
- $(HG) pull http://boxen.math.washington.edu:8100
- $(HG) update
-
-doc: doc-mathjax
-doc-pngmath:
- $(SAGE) -docbuild reference html $(OPTS)
-doc-mathjax:
- $(SAGE) -docbuild reference html -j $(OPTS)
-doc-jsmath: doc-mathjax # for backwards compatibility
-
-test:
- $(SAGE) -t -sagenb $(OPTS)
-ptest:
- $(SAGE) -tp \
- `$(PYTHON) -c "import multiprocessing as m; print m.cpu_count()"` \
- -sagenb $(OPTS)
-
-spkg:
- ./spkg-dist
-
-.PHONY: clean distclean
View
73 README.rst
@@ -53,3 +53,76 @@ Development
See the Sage Developer's guide, part of the Sage documentation, for
instructions.
+
+Miscellaneous Release Instructions
+----------------------------------
+
+The following advice for release managers of sagenb is taken from the
+old SPKG.txt file that was sitting around. Most of it is probably
+outdated, but here it is anyway. It is modified slightly to cause it to
+make sense in some cases.
+
+ To cut a new release of sagenb, make sure that:
+
+ * All changes are committed.
+
+ * ``.gitignore`` and ``MANIFEST.in`` are current.
+
+ * The notebook runs.
+
+ * The doctests pass: ``sage -t --sagenb``
+
+ * The notebook will be possible to install from the new SPKG without
+ an internet connection.
+
+ * Any dependencies that must be downloaded can be added in
+ ``util/fetch_deps.py`` and inserted in ``setup.py``.
+ Dependencies of dependencies need not be put in ``setup.py``,
+ but need to be put in ``util/fetch_deps.py`` (until we can make
+ it smarter).
+
+ * The Selenium tests pass (optional, for now).
+
+ * The localization file ``sagenb.pot`` is up-to-date.
+
+ * Run ``pybabel extract -F /path/to/babel.cfg /path/to/project -o
+ /path/to/sagenb.po`` (get pybabel with ``easy_install
+ pybabel``).
+
+ * Copy the headers from the existing ``sagenb.pot``.
+
+ * Replace ``sagenb.pot`` with ``sagenb.po``.
+
+ * Then, update the version in ``setup.py`` and commit this change.
+
+ * Run ``dist.sh``, optionally with a ``-g`` argument to package
+ the git repo too.
+
+ * Copy the newly generated ``dist/`` directory from the sagenb
+ repo to the SPKG's root directory and rename it ``src/``
+ , replacing the ``src/`` directory that is currently there
+
+ * Pack up the SPKG with ``sage --pkg --no-compress`` (because
+ everything in ``src/`` is already compressed)
+
+ * Install and test the new spkg: ``sage -f sagenb-*.spkg``
+
+ * Don't forget to push all changes in the sagenb repo to github.
+
+ Stylesheets (CSS): see ``sass/readme.txt``.
+
+ To add a locale to an existing install:
+
+ * Create a new locale, or download one from
+ http://wiki.sagemath.org/i18n . To create a new locale:
+
+ * Edit and save a copy of ``sagenb.pot`` using your favorite text
+ editor or POEdit (http://poedit.net)
+
+ * (Recommended) Post the new locale to
+ http://wiki.sagemath.org/i18n
+
+ * Compile your copy via ``msgfmt sagenb.pot -o sagenb.mo``
+
+ * Copy ``sagenb.mo`` to ``sagenb/locale/xx_YY/LC_MESSAGES/``, where
+ xx_YY is a locale code (en_US, pt_BR, en_UK, etc.)
View
98 SPKG.txt
@@ -1,98 +0,0 @@
-= Sage Notebook =
-
-== Description ==
-
-The Sage Notebook is a web-based graphical user interface for
-mathematical software.
-
-== License ==
-
-GPLv2+
-
-== SPKG Maintainers ==
-
- * William Stein
- * Tim Dumol
- * Mitesh Patel
-
-== Upstream Contact ==
-
- * William Stein
- * Homepage: http://nb.sagemath.org/
-
-== Dependencies ==
-
- * Python >= 2.6.4
- * jinja2 >= 2.5.5
- * Sphinx >= 1.0.4 (pretty docstrings)
-
- Included dependencies (for specific version numbers, see the
- spkg-dist file):
-
- * twisted
- * pytz
- * Babel
- * Werkzeug
- * speaklater
- * python-openid
- * Flask
- * Flask-Silk
- * Flask-AutoIndex
- * Flask-Babel
- * Flask-OpenID
- * zope.testbrowser >= 3.8.1 (maybe not anymore?)
-
-== Special Update / Build Instructions ==
-
- * To release a new spkg, check that
-
- * All changes are committed.
- * .hgignore and MANIFEST.in are current.
- * The patch queue is clear.
- * The notebook runs.
- * The doctests pass: sage -t -sagenb
- * Test for ability to install without internet connection.
- Any dependencies that must be downloaded can be added in ./spkg-dist and inserted
- in setup.py. Dependencies of dependencies need not be put in setup.py, but
- need to be put in ./spkg-dist.
- * The Selenium tests pass (optional, for now).
- * The localization file ``sagenb.pot`` is up-to-date .
- * Run ``pybabel extract -F /path/to/babel.cfg /path/to/project
- -o /path/to/sagenb.po`` (get pybabel with ``easy_install
- pybabel``).
- * Copy the headers from the existing ``sagenb.pot``.
- * Replace ``sagenb.pot`` with ``sagenb.po``.
-
- Then,
-
- * Update the version in setup.py and commit this change.
- * Run spkg-dist.
- * Install and test the new spkg: sage -f dist/sagenb-*.spkg
-
- Contact a maintainer about pushing updates to the public Mercurial
- repository.
-
- * Stylesheets (CSS): see sass/readme.txt.
-
- * To add a locale to an existing install:
-
- * Create a new locale, or download one from
- `http://wiki.sagemath.org/i18n`_:
-
- * To create a new locale:
-
- * Edit and save a copy of ``sagenb.pot``
- using your favorite text editor or POEdit (http://poedit.net)
-
- * (Recommended) Post the new locale to
- `http://wiki.sagemath.org/i18n`_
-
- * Compile your copy via ``msgfmt sagenb.pot -o sagenb.mo``
-
- * Copy ``sagenb.mo`` to ``sagenb/locale/xx_YY/LC_MESSAGES/``,
- where xx_YY is a locale code (en_US, pt_BR, en_UK, etc.)
-
-
-== Changelog ==
-
- * For an accurate log of changes, run "hg log" in src/sagenb.
View
58 dist.sh
@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+
+# This script should be run when creating a new SPKG for shipping sagenb
+# with Sage. For more complete instructions on how to generate a new
+# SPKG, read the SPKG.txt file in the current SPKG.
+#
+# Usage: dist.sh [-g]
+#
+# -g Also package the git repository
+
+die () {
+ echo >&2 "$@"
+ exit 1
+}
+
+while getopts ":g" opt; do
+ case $opt in
+ g) INSTALL_REPO=1 ;;
+ *) die "Invalid option!"
+ esac
+done
+
+cd ${0%/*}
+
+if [ -n "$INSTALL_REPO" ]; then
+ git diff --quiet && git diff --cached --quiet ||
+ die "Uncommitted changes in sagenb - please commit, stash, or discard"
+fi
+
+rm -rf dist
+mkdir -p dist
+
+echo "Fetching source tarballs of (sub)dependencies of sagenb to dist/"
+python util/fetch_deps.py dist || die "Couldn't fetch all (sub)dependencies"
+
+echo "Creating source tarball of sagenb itself in dist/"
+python setup.py sdist > sdist.log || die "Couldn't make sagenb source tarball"
+
+if [ -n "$INSTALL_REPO" ]; then
+ echo "Sanitizing sagenb git repo (with backup)"
+ mv .git .git-backup
+ git init
+ git fetch .git-backup
+ git remote add sagemath https://github.com/sagemath/sagenb
+ git branch -f master FETCH_HEAD
+ git remote update sagemath
+ git branch --set-upstream master sagemath/master
+ git reset --mixed
+ git gc --aggressive --prune=0
+
+ echo "Moving sanitized sagenb git repo to dist/"
+ mv .git dist/sagenb.git
+
+ echo "Restoring backup of git repo"
+ mv .git-backup .git
+fi
+
+echo "Done!"
View
7 go
@@ -1,7 +0,0 @@
-#!/usr/bin/env pynb
-import os
-os.system('python setup.py install')
-
-import sagenb.notebook.notebook_object as n
-n.notebook('sage_notebook', server_pool=[])
-
View
36 sdist
@@ -1,36 +0,0 @@
-#!/usr/bin/env bash
-
-# Check whether "sage" can be run (i.e. whether it's in the PATH)
-if ! sage -v >/dev/null; then
- echo >&2 "Error: $0 requires 'sage' to be in your PATH"
- echo >&2 "Maybe first call 'SAGE_ROOT/sage -sh'"
- exit 1
-fi
-
-echo "Trying to commit changes (if any)..."
-sage -hg commit
-status=$?
-if [ $status -eq 1 ]; then
- # 'hg commit' returns 1 if there are no changes to commit...
- echo "No changes to commit (not an error)"
-elif [ $status -ne 0 ]; then
- echo >&2 "Error: $0: Couldn't commit changes" \
- "('hg commit' failed with exit code $status)"
- exit 1
-else
- # $status == 0
- echo "Successfully committed the changes"
-fi
-
-echo "Running 'python setup.py sdist'..."
-sage -python setup.py sdist
-if [ $? -ne 0 ]; then
- echo >&2 "Error: $0: 'python setup.py sdist' failed"
- exit 1
-fi
-
-
-echo "**********************************************************"
-echo "* If this is an official SageNB release, don't forget to *"
-echo "* push the changes to the public Mercurial repository. *"
-echo "**********************************************************"
View
95 setup.py
@@ -2,64 +2,71 @@
# The setup.py for the Sage Notebook
##########################################################
-import os, sys, time
+import os
from setuptools import setup
-
-
import distutils.log
-if os.environ.get("SAGE_SETUPTOOLS_DEBUG","no")=="yes":
- distutils.log.set_threshold(distutils.log.DEBUG)
-def all_files(dir, lstrip):
+def lremove(string, prefix):
+ while string.startswith(prefix):
+ string = string[len(prefix):]
+ return string
+
+def all_files(dir, prefix):
"""
- Return list of all filenames in the given directory, with lstrip
+ Return list of all filenames in the given directory, with prefix
stripped from the left of the filenames.
"""
+
X = []
for F in os.listdir(dir):
ab = dir+'/'+F
if os.path.isfile(ab):
- X.append((ab).lstrip(lstrip))
+ X.append(lremove(ab, prefix))
elif os.path.isdir(ab):
- X.extend(all_files(ab, lstrip))
+ X.extend(all_files(ab, prefix))
return X
-code = setup(name = 'sagenb',
- version = '0.9.1', # the spkg-dist script assumes single quotes here
- description = 'The Sage Notebook',
- license = 'GNU Public License (GPL) v2+',
- author = 'William Stein et al.',
- author_email= 'http://groups.google.com/group/sage-notebook',
- url = 'http://code.google.com/p/sagenb',
- install_requires = [ 'twisted>=11.0.0'
- , 'flask'
- , 'flask-openid'
- , 'flask-autoindex'
- , 'babel'
- , 'flask-babel'
- , 'hg-git'
- , 'pyOpenSSL<=0.12'
- , 'webassets'
+install_requires = [ 'twisted>=11.0.0'
+ , 'flask'
+ , 'flask-openid'
+ , 'flask-autoindex'
+ , 'babel'
+ , 'flask-babel'
+ , 'pyOpenSSL<=0.12'
+ , 'webassets'
+ ]
+
+if __name__ == '__main__':
+ if os.environ.get("SAGE_SETUPTOOLS_DEBUG","no")=="yes":
+ distutils.log.set_threshold(distutils.log.DEBUG)
+
+ code = setup(name = 'sagenb',
+ version = '0.9.1',
+ description = 'The Sage Notebook',
+ license = 'GNU General Public License (GPL) v2+',
+ author = 'William Stein et al.',
+ author_email= 'sage-notebook@googlegroups.com',
+ url = 'http://github.com/sagemath/sagenb',
+ install_requires = install_requires,
+ test_suite = 'sagenb.testing.run_tests.all_tests',
+ packages = [ 'sagenb'
+ , 'sagenb.interfaces'
+ , 'sagenb.misc'
+ , 'sagenb.notebook'
+ , 'sagenb.notebook.compress'
+ , 'sagenb.simple'
+ , 'sagenb.storage'
+ , 'sagenb.testing'
+ , 'sagenb.testing.tests'
+ , 'sagenb.testing.selenium'
+ ],
+ scripts = [ 'sagenb/data/sage3d/sage3d',
],
- test_suite = 'sagenb.testing.run_tests.all_tests',
- packages = [ 'sagenb'
- , 'sagenb.interfaces'
- , 'sagenb.misc'
- , 'sagenb.notebook'
- , 'sagenb.notebook.compress'
- , 'sagenb.simple'
- , 'sagenb.storage'
- , 'sagenb.testing'
- , 'sagenb.testing.tests'
- , 'sagenb.testing.selenium'
- ],
- scripts = [ 'sagenb/data/sage3d/sage3d',
- ],
- package_data = {'sagenb':
- all_files('sagenb/data', 'sagenb/') +
- all_files('sagenb/translations', 'sagenb/')
- },
- )
+ package_data = {'sagenb':
+ all_files('sagenb/data', 'sagenb/') +
+ all_files('sagenb/translations', 'sagenb/')
+ },
+ )
View
257 spkg-dist
@@ -1,257 +0,0 @@
-#!/usr/bin/env python
-# Create a new Sage Notebook spkg.
-#
-# This spkg-dist doesn't need any Python modules of Sage, so it should
-# be okay to run it from a system-wide Python (provided it is sufficiently
-# recent). This script *does* need a sage executable to be in the PATH
-# because we need to run "sage -pkg" below and also the "sdist" script
-# runs "sage -python" and "sage -hg". -- Jeroen Demeyer
-
-
-import os
-import sys
-import stat
-import shutil
-import tarfile
-import subprocess
-from tempfile import mkdtemp
-
-from pkg_resources import Requirement
-from setuptools.package_index import PackageIndex
-
-# Get the version from setup.py.
-try:
- version_line = [f for f in open('setup.py').readlines() if 'version' in f][0]
-except IndexError:
- sys.stderr.write("Error: Found no version string in 'setup.py'\n")
- sys.exit(1)
-
-i = version_line.find("'")
-j = version_line.rfind("'")
-if i==-1 or j==-1 or j-i<3:
- sys.stderr.write(
- "Error: Illegal version string in 'setup.py':\n %s\n" % version_line)
- sys.stderr.write("Perhaps no single quotes used?\n")
- sys.exit(1)
-version = version_line[i + 1:j]
-print "New SageNB version: %s" % version
-
-# Create the source distribution.
-print "Creating the new source tarball..."
-sys.stdout.flush()
-proc = subprocess.Popen([os.path.join(os.path.curdir, 'sdist')], shell=True)
-if proc.wait():
- sys.stderr.write(
- "Error: './sdist' failed. See message(s) above for the specific error.\n")
- sys.exit(1)
-
-# Create the spkg.
-base = 'sagenb-%s' % version
-path = os.path.join('dist', base)
-if os.path.exists(path):
- shutil.rmtree(path)
-os.makedirs(path)
-
-file = 'sagenb-%s.tar.gz' % version # created by 'sdist' above
-print "Extracting %s..." % file
-sys.stdout.flush()
-t = tarfile.open(os.path.join('dist', file))
-t.extractall(path)
-print "Finished extraction."
-sys.stdout.flush()
-
-os.chdir(path) # cd dist/sagenb-x.y.z
-os.mkdir('src')
-print "Moving new source tree..."
-sys.stdout.flush()
-shutil.move(base, os.path.join('src', 'sagenb')) # mv sagenb-x.y.z src/sagenb
-
-print "Copying 'SPKG.txt'..."
-sys.stdout.flush()
-shutil.copy(os.path.join(os.path.pardir, os.path.pardir, 'SPKG.txt'),
- os.path.curdir) # cp ../../SPKG.txt . (which is dist/sagenb-x.y.z)
-
-print "Creating 'spkg-install'..."
-sys.stdout.flush()
-spkg_install = os.path.abspath(os.path.join(os.path.curdir, 'spkg-install'))
-spkg_install_fd = open(spkg_install, 'w')
-
-
-def fetch_packages():
- """
- Downloads the required dependencies and returns a string
- which installs the dependencies using easy_install
- """
- print "Fetching the required packages"
- pkg_index = PackageIndex()
-
- tmp_dir = mkdtemp()
-
- # in order of dependencies
- required_packages = ( 'twisted>=11.0.0'
- , 'pytz>=2011n'
- , 'Babel>=0.9.6'
- , 'Werkzeug>=0.8.2'
- , 'speaklater>=1.2'
- , 'python-openid>=2.2.5'
- , 'Flask>=0.8'
- , 'Flask-Silk>=0.1.1'
- , 'Flask-AutoIndex>=0.4.0'
- , 'Flask-Babel>=0.8'
- , 'Flask-OpenID>=1.0.1'
- , 'dulwich>=0.8.0'
- , 'hg-git>=0.3.1'
- , 'pyOpenSSL<=0.12'
- , 'webassets>=0.7.1'
- )
- pkg_locations = []
-
- for pkg in required_packages:
- print "Fetching %s" % pkg
- dist = pkg_index.fetch_distribution(Requirement.parse(pkg), tmp_dir,
- True, True)
- pkg_locations.append(os.path.abspath(dist.location))
-
- install_str = ''
- for location in pkg_locations:
- shutil.copy(location, os.path.join('src', os.path.basename(location)))
-
- return [os.path.basename(location) for location in pkg_locations]
-
-install_dependencies=""
-for pkg_location in fetch_packages():
- # --allow-hosts=None prevents downloading
- install_dependencies += r"""
-easy_install --allow-hosts=None %(pkg)s
-if [ $? -ne 0 ]; then
- echo >&2 "Error installing %(pkg)s."
- exit 1
-fi
-
-""" % {'pkg': pkg_location}
-
-
-os.chdir(os.path.pardir) # cd .. (now in dist/)
-
-# Write the whole file from a single raw Python string:
-# (The #! has to be on the first line of the script.)
-spkg_install_fd.write(
-r"""#!/usr/bin/env bash
-
-# spkg-install for SageNB, generated by SageNB's spkg-dist
-
-if [ -z "$SAGE_LOCAL" ]; then
- echo >&2 "SAGE_LOCAL undefined - exiting..."
- exit 1
-fi
-
-cd src
-[ -z "$CPATH" ] || CPATH="$CPATH":
-[ -z "$LIBRARY_PATH" ] || LIBRARY_PATH="$LIBRARY_PATH":
-export CPATH="$CPATH""$SAGE_LOCAL"/include
-export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib
-
-%(install_dependencies)s
-
-cd sagenb
-python setup.py install
-if [ $? -ne 0 ]; then
- echo >&2 "Error running 'setup.py install'."
- exit 1
-fi
-
-mkdir -p "$SAGE_ROOT/devel" # Create if it doesn't already exist
-if [ $? -ne 0 ]; then
- echo >&2 "Error creating '$SAGE_ROOT/devel'."
- exit 1
-fi
-
-echo "Copying SageNB package to '$SAGE_ROOT/devel/sagenb-main'..."
-
-if [ -d "$SAGE_ROOT/devel/sagenb-main" ]; then
- echo "Moving old SageNB package to '$SAGE_ROOT/devel/sagenb-main-old'..."
- rm -rf "$SAGE_ROOT/devel/sagenb-main-old"
- mv "$SAGE_ROOT/devel/sagenb-main" "$SAGE_ROOT/devel/sagenb-main-old"
- if [ $? -ne 0 ]; then
- echo >&2 "Error moving the old 'sagenb-main' branch."
- exit 1
- fi
-fi
-
-rm -f "$SAGE_ROOT/devel/sagenb" # Delete just the link itself (if it exists)
-
-cd .. # Back to sagenb-x.y.z/src/
-cp -pr sagenb "$SAGE_ROOT/devel/sagenb-main" # Creates new sagenb-main dir
-if [ $? -ne 0 ]; then
- echo >&2 "Error copying the new SageNB package."
- exit 1
-fi
-
-# Create an appropriate hgrc file for the target
-cat > "$SAGE_ROOT/devel/sagenb-main/.hg/hgrc" <<"HEREDOC"
-[diff]
-git = true
-
-[extensions]
-hggit=
-
-[git]
-intree = True
-HEREDOC
-
-cd "$SAGE_ROOT/devel"
-ln -s sagenb-main sagenb # Create new symbolic link (deleted above)
-if [ $? -ne 0 ]; then
- echo >&2 "Error creating symbolic link to '$SAGE_ROOT/devel/sagenb-main'."
- exit 1
-fi
-
-# We use relative paths for relocatability.
-cd "$SAGE_ROOT/devel/sagenb"
-python setup.py develop --egg-path ../../../../devel/sagenb
-if [ $? -ne 0 ]; then
- echo >&2 "Error running 'setup.py develop'."
- exit 1
-fi
-
-cd "$SAGE_ROOT/local/lib/python/site-packages"
-# Use >/dev/null instead of grep -q (which doesn't work on Solaris)
-if ! grep sagenb easy-install.pth >/dev/null; then
- # Ugly work-around, we haven't found the real cause yet (see #10176):
- echo "No sagenb path found in 'easy-install.pth'"'!'
- echo "Adding relative sagenb path to 'easy-install.pth'..."
- sed -e '$ i \../../../../devel/sagenb' easy-install.pth > easy-install.pth.$$
- if [ $? -ne 0 ]; then
- echo >&2 "Error adding relative sagenb path to 'easy-install.pth'."
- exit 1
- fi
-else
- echo "Making sagenb path in 'easy-install.pth' relative..."
- sed 's/^.*sagenb.*$/..\/..\/..\/..\/devel\/sagenb/' easy-install.pth > easy-install.pth.$$
- if [ $? -ne 0 ]; then
- echo >&2 "Error patching 'easy-install.pth' to have relative path to SageNB."
- exit 1
- fi
-fi
-
-# Print paths for debugging
-echo "Old path: '`grep sagenb easy-install.pth`'"
-echo "New path: '`grep sagenb easy-install.pth.$$`'"
-
-# The following fails only on wrong file permissions etc.:
-mv -f easy-install.pth.$$ easy-install.pth
-if [ $? -ne 0 ]; then
- echo >&2 "Error overwriting original 'easy-install.pth'."
- exit 1
-fi
-""" % {'install_dependencies': install_dependencies})
-spkg_install_fd.close()
-os.chmod(spkg_install, stat.S_IRWXU | stat.S_IRGRP | stat.S_IXGRP |
- stat.S_IROTH | stat.S_IXOTH) # -rwxr-xr-x
-
-# We are still in dist/, now package sagenb-x.y.z/ :
-print "Running 'sage -pkg %s'..."%base
-sys.stdout.flush()
-if subprocess.call(['sage', '-pkg', base]):
- sys.stderr.write('Error: "sage -pkg %s" failed.\nSee message(s) above for the specific error.\n'%base)
- sys.exit(1)
View
51 util/fetch_deps.py
@@ -0,0 +1,51 @@
+#!/usr/bin/env python
+"""
+Downloads certain (sub)dependencies of sagenb into the given directory
+and returns a list of their names. This script is run from dist.sh, and
+need not be run independently.
+"""
+# These are linearly ordered such that no package depends on something
+# lower than it in the list (or not in the list).
+#
+# TODO: Automatically generate this (using distribute internals ?)
+required_packages = [ 'twisted>=11.0.0'
+ , 'pytz>=2011n'
+ , 'Babel>=0.9.6'
+ , 'Werkzeug>=0.8.2'
+ , 'speaklater>=1.2'
+ , 'python-openid>=2.2.5'
+ , 'Flask>=0.8'
+ , 'Flask-Silk>=0.1.1'
+ , 'Flask-AutoIndex>=0.4.0'
+ , 'Flask-Babel>=0.8'
+ , 'Flask-OpenID>=1.0.1'
+ , 'pyOpenSSL<=0.12'
+ , 'webassets>=0.7.1'
+ ]
+
+import os, shutil, sys
+from pkg_resources import Requirement
+from setuptools.package_index import PackageIndex
+
+def die(message):
+ sys.stderr.write(message)
+ sys.exit(1)
+
+if __name__ == '__main__':
+ if len(sys.argv) < 2:
+ print "Please supply a destination directory for the fetched packages!"
+ sys.exit(1)
+ dest_dir = os.path.abspath(sys.argv[1])
+
+ print "Fetching packages:"
+ pkg_index = PackageIndex()
+ with open(os.path.join(dest_dir, 'install_order'), 'w') as fd:
+ for pkg in required_packages:
+ print "(--- Processing requirement '{0}'".format(pkg)
+ dist = pkg_index.fetch_distribution(Requirement.parse(pkg),
+ dest_dir, True, True)
+ if dist is None:
+ die("Couldn't find package satisfying '{0}'!".format(pkg))
+ print " ---) Fetched {0} {1}".format(dist.project_name,
+ dist.version)
+ fd.write(os.path.basename(dist.location) + '\n')
Something went wrong with that request. Please try again.