Skip to content
This repository

Rewrite packaging system, rip out Mercurial #63

Merged
merged 19 commits into from almost 2 years ago

2 participants

Keshav Kini John H. Palmieri
Keshav Kini
Owner
kini commented June 13, 2012

I've considerably simplified the packaging system. The new SPKG is
uncompressed, and has a Mercurial repository tracking spkg-install et
al. like all other SPKGs do. The src/ directory consists of a bunch of
source tarballs of dependencies, an sdist tarball of sagenb itself,
and (optionally) a copy of the git repository of sagenb suitable for
beginning development from.

References to Mercurial, dependencies on Mercurial tools, etc. are also
removed.

added some commits June 12, 2012
Keshav Kini Fix stripping 62099fb
Keshav Kini Clean up setup.py bfdffa0
Keshav Kini Switch MANIFEST.in stuff to git 1237b7b
Keshav Kini Add some scripts to automate packaging of deps
depdist.sh will be used to create a dist/ directory which should be
moved to the sagenb spkg directory and renamed to src/ .

fetch_deps.py is a helper script for depdist.sh which uses distribute
internals to fetch package tarballs from PyPI.
38327ca
Keshav Kini Put fetch_deps in util/ 74a53e4
Keshav Kini Moved SPKG.txt to the SPKG's own repo 83c7fab
Keshav Kini Delete old packaging system and some old files 9ebcc69
Keshav Kini Rename depdist.sh to dist.sh; sanitize git repo f37537b
Keshav Kini Remove outdated Makefile 97a817f
Keshav Kini Fix packaging of git repo
`setup.py sdist` doesn't copy empty directories, as I found out after
a lot of head scratching. This breaks git repos, so we can't trust sdist
to copy our git repo into the source tarball of sagenb. So we'll store
the git repo separately as sagenb.git , and move it into place when
running spkg-install.
0538713
Keshav Kini Make sure the staging area is clean too 8f14524
Keshav Kini Add sagemath/sagenb github remote to the new repo 600bd94
Keshav Kini Remove hg-git dependencies f0ecab2
Keshav Kini Merge branch 'master' into packaging
Conflicts:
	Makefile
efe5ac9
Keshav Kini kini referenced this pull request June 13, 2012
Closed

Jmol enhancements #61

Keshav Kini
Owner
kini commented June 13, 2012

Here's an example of a sagenb SPKG created by dist.sh after these commits: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

You will need to take a look at it in order to review this branch because it contains the new SPKG repo I made for sagenb.

Keshav Kini Fix github remote tracking
Apparently you can't set a branch to track a remote tracking branch that
you haven't fetched yet.
9143deb
Keshav Kini
Owner
kini commented June 13, 2012

SPKG updated according to the latest commit: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg (same link)

John H. Palmieri

Would it make any sense to omit the git repository and instead document how to install it for people who want to work on sagenb development?

Keshav Kini
Owner
kini commented June 15, 2012

In my opinion, yes, absolutely. Sage currently ships its repositories inside both source and binary tarballs, which is unorthodox to say the least. The stated goal is to make the barrier of entry to development extremely low in order to gather mathematicians who are not experienced software engineers and open-source development collaborators, but who nonetheless have been hacking on Sage for their own purposes/research, and entice them to give us their code by making it extremely easy to do so. But I don't think this goal makes much sense in the context of sagenb, which is a web app and doesn't really seek contributions from inexperienced programmers who are experts in some other field.

added some commits June 15, 2012
Keshav Kini Insert old advice from SPKG.txt
This old advice for release management of sagenb was in the SPKG.txt and
was generally out of date. Since the SPKG will now only contain a bunch
of tarballs, a lot of those instructions don't make much sense there, so
I've moved them to the sagenb README for now.
99c95c9
Keshav Kini Fix dumb oversight 5c5802a
Keshav Kini Make packaging the git repo optional
New behavior: `./dist.sh` prepares tarballs of sagenb and its
dependencies in dist/ , and `./dist.sh -g` additionally places an
optimized copy of the git repository in dist/
3afe890
Keshav Kini
Owner
kini commented June 15, 2012

OK, so packaging the git repo is now optional. Pass -g to dist.sh if you want a git repo.

New SPKG is up at the same URL: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

John H. Palmieri

When I run dist.sh, I see the following:

Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!

Then everything seems to get downloaded successfully. I also see

warning: sdist: standard file not found: should have one of README, README.txt

I have no idea what this means.

Otherwise, everything looks good, and the new SPKG.txt file has good directions for how to use dist.sh. Should SPKG.txt list the non-included dependencies also, like Python, to record where it should go in SAGE_ROOT/spkg/standard/deps?

John H. Palmieri

Oh, it might be a good idea to provide instructions (in SPKG.txt? at the top of dist.sh?) for how to use the git repository in the directory sagenb.git, if you run dist.sh -g. (I think you should unpack sagenb-0.9.1.tar.gz and then move sagenb.git to sagenb-0.9.1/.git. Is that right?)

Keshav Kini
Owner
kini commented June 18, 2012
John H. Palmieri

Those don't appear on a stable connection, such as on sage.math.

I just tested, and I also see them on sage.math (although with "Errno 111" instead of "Errno 61"). Also, they are completely reproducible: I see four of these "Download error" messages every time I run the script. Maybe you could add a print statement print "Fetching {0} ...".format(pkg) at the start of the loop, so that the later message "Fetched ..." will reassure the user that the desired package was retrieved successfully.

Keshav Kini
Owner
kini commented June 18, 2012
Keshav Kini
Owner
kini commented June 18, 2012
John H. Palmieri

No, the instructions are correct - spkg-install will install the git repo correctly if sagenb.git is present in the src directory.

Oh, right. Sorry.

Keshav Kini
Owner
kini commented June 20, 2012

OK, I've made the changes you suggested - the ones to the dist scripts in the above commit, and the ones to SPKG.txt in the new SPKG I've just uploaded to the same URL as before: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

John H. Palmieri

I think this looks good now. I've reviewed the changes and tried out dist.sh. Everything seems to work. Does that suffice for a positive review?

Keshav Kini
Owner
kini commented June 20, 2012

Yup! Thanks for helping me out with this!

Keshav Kini kini merged commit b0ce4e7 into from June 20, 2012
Keshav Kini kini closed this June 20, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 19 unique commits by 1 author.

Jun 12, 2012
Keshav Kini Fix stripping 62099fb
Keshav Kini Clean up setup.py bfdffa0
Keshav Kini Switch MANIFEST.in stuff to git 1237b7b
Keshav Kini Add some scripts to automate packaging of deps
depdist.sh will be used to create a dist/ directory which should be
moved to the sagenb spkg directory and renamed to src/ .

fetch_deps.py is a helper script for depdist.sh which uses distribute
internals to fetch package tarballs from PyPI.
38327ca
Keshav Kini Put fetch_deps in util/ 74a53e4
Keshav Kini Moved SPKG.txt to the SPKG's own repo 83c7fab
Keshav Kini Delete old packaging system and some old files 9ebcc69
Keshav Kini Rename depdist.sh to dist.sh; sanitize git repo f37537b
Keshav Kini Remove outdated Makefile 97a817f
Jun 13, 2012
Keshav Kini Fix packaging of git repo
`setup.py sdist` doesn't copy empty directories, as I found out after
a lot of head scratching. This breaks git repos, so we can't trust sdist
to copy our git repo into the source tarball of sagenb. So we'll store
the git repo separately as sagenb.git , and move it into place when
running spkg-install.
0538713
Keshav Kini Make sure the staging area is clean too 8f14524
Keshav Kini Add sagemath/sagenb github remote to the new repo 600bd94
Keshav Kini Remove hg-git dependencies f0ecab2
Keshav Kini Merge branch 'master' into packaging
Conflicts:
	Makefile
efe5ac9
Keshav Kini Fix github remote tracking
Apparently you can't set a branch to track a remote tracking branch that
you haven't fetched yet.
9143deb
Jun 15, 2012
Keshav Kini Insert old advice from SPKG.txt
This old advice for release management of sagenb was in the SPKG.txt and
was generally out of date. Since the SPKG will now only contain a bunch
of tarballs, a lot of those instructions don't make much sense there, so
I've moved them to the sagenb README for now.
99c95c9
Keshav Kini Fix dumb oversight 5c5802a
Keshav Kini Make packaging the git repo optional
New behavior: `./dist.sh` prepares tarballs of sagenb and its
dependencies in dist/ , and `./dist.sh -g` additionally places an
optimized copy of the git repository in dist/
3afe890
Jun 19, 2012
Keshav Kini Clearer status messages f16c221
This page is out of date. Refresh to see the latest.
74  .hgignore
... ...
@@ -1,74 +0,0 @@
1  
-# Boring file regexps:
2  
-\.scssc$
3  
-\.sassc$
4  
-\.git$
5  
-\.gitignore$
6  
-\.hi$
7  
-\.mtl$
8  
-\.obj$
9  
-\.db$
10  
-\.o$
11  
-\.o\.cmd$
12  
-\.ko$
13  
-\.ko\.cmd$
14  
-\.mod\.c$
15  
-(^|/)\.tmp_versions($|/)
16  
-(^|/)CVS($|/)
17  
-(^|/)RCS($|/)
18  
-~$
19  
-#(^|/)\.[^/]
20  
-(^|/)_darcs($|/)
21  
-\.tmp$
22  
-\.hg$
23  
-\.rej$
24  
-\.sobj$
25  
-\.bak$
26  
-\.BAK$
27  
-\.dylib$
28  
-\.fig$
29  
-\.eps$
30  
-\.cpp$
31  
-\.orig$
32  
-\.patch$
33  
-(^|/)vssver\.scc$
34  
-\.sw.$
35  
-(^|/)MT($|/)
36  
-(^|/)\{arch\}($|/)
37  
-(^|/).arch-ids($|/)
38  
-(^|/),
39  
-\.class$
40  
-\.prof$
41  
-(^|/)\.DS_Store$
42  
-(^|/)BitKeeper($|/)
43  
-(^|/)ChangeSet($|/)
44  
-(^|/)\.svn($|/)
45  
-\.py[co]$
46  
-\#
47  
-\.cvsignore$
48  
-(^|/)Thumbs\.db$
49  
-^\.gitk-tmp
50  
-# sage files to ignore:
51  
-(^|/).doctest*
52  
-^dist($|/)
53  
-^notes($|/)
54  
-(^|/).old($|/)
55  
-(^|/).doctest($|/)
56  
-(^|/)deleted($|/)
57  
-^build($|/)
58  
-^mirror($|/)
59  
-^new($|/)
60  
-^commit($|/)
61  
-^export($|/)
62  
-MANIFEST
63  
-PKG-INFO
64  
-pull
65  
-push
66  
-release_notes.txt
67  
-sage_notebook*
68  
-\.sagenb
69  
-^sass/src/.sass-cache
70  
-sagenb/data/jmol
71  
-^sagenb.egg-info
72  
-setup.cfg
73  
-sagenb/data/jsmath/fonts
74  
-sagenb/data/MathJax
16  MANIFEST.in
... ...
@@ -1,12 +1,8 @@
1  
-recursive-include sagenb *
2  
-recursive-include sass *
3  
-recursive-include util *
  1
+graft sagenb
  2
+graft sass
  3
+graft util
4 4
 prune sass/src/.sass-cache
5  
-recursive-include flask_version *
6  
-include *.txt Makefile MANIFEST.in .hgignore .hgtags sdist spkg-dist SPKG.txt go .gitignore README.rst
7  
-recursive-include .hg *
8  
-prune .hg/git
9  
-prune .hg/patches
10  
-exclude .hg/git-*
11  
-exclude .hg/hgrc
  5
+graft flask_version
  6
+include *.txt MANIFEST.in .gitignore dist.sh README.rst
  7
+prune .git
12 8
 global-exclude *.pyc *.pyo *.orig *.rej *~ \#* *\# *.sassc *.scssc *.DS_Store
37  Makefile
... ...
@@ -1,37 +0,0 @@
1  
-# -*- makefile -*-
2  
-# Makefile for the Sage Notebook project (http://nb.sagemath.org/).
3  
-# GNU make help: http://www.gnu.org/software/make/manual
4  
-
5  
-SAGE = sage
6  
-HG = $(SAGE) -hg
7  
-PYTHON = $(SAGE) -python
8  
-SETUP = $(PYTHON) setup.py
9  
-
10  
-install:
11  
-	$(SETUP) install
12  
-
13  
-develop:
14  
-	$(SETUP) develop
15  
-
16  
-update:
17  
-	$(HG) pull http://boxen.math.washington.edu:8100
18  
-	$(HG) update
19  
-
20  
-doc:	doc-mathjax
21  
-doc-pngmath:
22  
-	$(SAGE) -docbuild reference html $(OPTS)
23  
-doc-mathjax:
24  
-	$(SAGE) -docbuild reference html -j $(OPTS)
25  
-doc-jsmath: doc-mathjax # for backwards compatibility
26  
-
27  
-test:
28  
-	$(SAGE) -t -sagenb $(OPTS)
29  
-ptest:
30  
-	$(SAGE) -tp \
31  
-	`$(PYTHON) -c "import multiprocessing as m; print m.cpu_count()"` \
32  
-	-sagenb $(OPTS)
33  
-
34  
-spkg:
35  
-	./spkg-dist
36  
-
37  
-.PHONY:	clean distclean
73  README.rst
Source Rendered
@@ -53,3 +53,76 @@ Development
53 53
 
54 54
 See the Sage Developer's guide, part of the Sage documentation, for
55 55
 instructions.
  56
+
  57
+Miscellaneous Release Instructions
  58
+----------------------------------
  59
+
  60
+The following advice for release managers of sagenb is taken from the
  61
+old SPKG.txt file that was sitting around. Most of it is probably
  62
+outdated, but here it is anyway. It is modified slightly to cause it to
  63
+make sense in some cases.
  64
+
  65
+    To cut a new release of sagenb, make sure that:
  66
+
  67
+    * All changes are committed.
  68
+
  69
+    * ``.gitignore`` and ``MANIFEST.in`` are current.
  70
+
  71
+    * The notebook runs.
  72
+
  73
+    * The doctests pass: ``sage -t --sagenb``
  74
+
  75
+    * The notebook will be possible to install from the new SPKG without
  76
+      an internet connection.
  77
+
  78
+      * Any dependencies that must be downloaded can be added in
  79
+        ``util/fetch_deps.py`` and inserted in ``setup.py``.
  80
+        Dependencies of dependencies need not be put in ``setup.py``,
  81
+        but need to be put in ``util/fetch_deps.py`` (until we can make
  82
+        it smarter).
  83
+
  84
+    * The Selenium tests pass (optional, for now).
  85
+
  86
+    * The localization file ``sagenb.pot`` is up-to-date.
  87
+
  88
+      * Run ``pybabel extract -F /path/to/babel.cfg /path/to/project -o
  89
+        /path/to/sagenb.po`` (get pybabel with ``easy_install
  90
+        pybabel``).
  91
+
  92
+      * Copy the headers from the existing ``sagenb.pot``.
  93
+
  94
+      * Replace ``sagenb.pot`` with ``sagenb.po``.
  95
+
  96
+      * Then, update the version in ``setup.py`` and commit this change.
  97
+
  98
+      * Run ``dist.sh``, optionally with a ``-g`` argument to package
  99
+        the git repo too.
  100
+
  101
+      * Copy the newly generated ``dist/`` directory from the sagenb
  102
+        repo to the SPKG's root directory and rename it ``src/``
  103
+        , replacing the ``src/`` directory that is currently there
  104
+
  105
+      * Pack up the SPKG with ``sage --pkg --no-compress`` (because
  106
+        everything in ``src/`` is already compressed)
  107
+
  108
+      * Install and test the new spkg: ``sage -f sagenb-*.spkg``
  109
+
  110
+      * Don't forget to push all changes in the sagenb repo to github.
  111
+
  112
+    Stylesheets (CSS): see ``sass/readme.txt``.
  113
+
  114
+    To add a locale to an existing install:
  115
+
  116
+    * Create a new locale, or download one from
  117
+      http://wiki.sagemath.org/i18n . To create a new locale:
  118
+
  119
+      * Edit and save a copy of ``sagenb.pot`` using your favorite text
  120
+        editor or POEdit (http://poedit.net)
  121
+
  122
+      * (Recommended) Post the new locale to
  123
+        http://wiki.sagemath.org/i18n
  124
+
  125
+    * Compile your copy via ``msgfmt sagenb.pot -o sagenb.mo``
  126
+
  127
+    * Copy ``sagenb.mo`` to ``sagenb/locale/xx_YY/LC_MESSAGES/``, where
  128
+      xx_YY is a locale code (en_US, pt_BR, en_UK, etc.)
98  SPKG.txt
... ...
@@ -1,98 +0,0 @@
1  
-= Sage Notebook =
2  
-
3  
-== Description ==
4  
-
5  
-The Sage Notebook is a web-based graphical user interface for
6  
-mathematical software.
7  
-
8  
-== License ==
9  
-
10  
-GPLv2+
11  
-
12  
-== SPKG Maintainers ==
13  
-
14  
- * William Stein
15  
- * Tim Dumol
16  
- * Mitesh Patel
17  
-
18  
-== Upstream Contact ==
19  
-
20  
- * William Stein
21  
- * Homepage: http://nb.sagemath.org/
22  
-
23  
-== Dependencies ==
24  
-
25  
- * Python >= 2.6.4
26  
- * jinja2 >= 2.5.5
27  
- * Sphinx >= 1.0.4 (pretty docstrings)
28  
-
29  
- Included dependencies (for specific version numbers, see the
30  
- spkg-dist file):
31  
-
32  
- * twisted
33  
- * pytz
34  
- * Babel
35  
- * Werkzeug
36  
- * speaklater
37  
- * python-openid
38  
- * Flask
39  
- * Flask-Silk
40  
- * Flask-AutoIndex
41  
- * Flask-Babel
42  
- * Flask-OpenID
43  
- * zope.testbrowser >= 3.8.1 (maybe not anymore?)
44  
-
45  
-== Special Update / Build Instructions ==
46  
-
47  
- * To release a new spkg, check that
48  
-
49  
-   * All changes are committed.
50  
-   * .hgignore and MANIFEST.in are current.
51  
-   * The patch queue is clear.
52  
-   * The notebook runs.
53  
-   * The doctests pass: sage -t -sagenb
54  
-   * Test for ability to install without internet connection.
55  
-     Any dependencies that must be downloaded can be added in ./spkg-dist and inserted
56  
-     in setup.py. Dependencies of dependencies need not be put in setup.py, but
57  
-     need to be put in ./spkg-dist.
58  
-   * The Selenium tests pass (optional, for now).
59  
-   * The localization file ``sagenb.pot`` is up-to-date .
60  
-      * Run ``pybabel extract -F /path/to/babel.cfg /path/to/project
61  
-        -o /path/to/sagenb.po`` (get pybabel with ``easy_install
62  
-        pybabel``).
63  
-      * Copy the headers from the existing ``sagenb.pot``.
64  
-      * Replace ``sagenb.pot`` with ``sagenb.po``.
65  
-
66  
-   Then,
67  
-
68  
-   * Update the version in setup.py and commit this change.
69  
-   * Run spkg-dist.
70  
-   * Install and test the new spkg: sage -f dist/sagenb-*.spkg
71  
-
72  
-   Contact a maintainer about pushing updates to the public Mercurial
73  
-   repository.
74  
-
75  
- * Stylesheets (CSS): see sass/readme.txt.
76  
-
77  
- * To add a locale to an existing install:
78  
- 
79  
-   * Create a new locale, or download one from
80  
-     `http://wiki.sagemath.org/i18n`_:
81  
-
82  
-      * To create a new locale:
83  
-  
84  
-        * Edit and save a copy of ``sagenb.pot``
85  
-          using your favorite text editor or POEdit (http://poedit.net)
86  
-
87  
-        * (Recommended) Post the new locale to
88  
-          `http://wiki.sagemath.org/i18n`_
89  
-
90  
-   * Compile your copy via ``msgfmt sagenb.pot -o sagenb.mo``
91  
-
92  
-   * Copy ``sagenb.mo`` to ``sagenb/locale/xx_YY/LC_MESSAGES/``,
93  
-     where xx_YY is a locale code (en_US, pt_BR, en_UK, etc.)
94  
-
95  
-
96  
-== Changelog ==
97  
-
98  
- * For an accurate log of changes, run "hg log" in src/sagenb.
58  dist.sh
... ...
@@ -0,0 +1,58 @@
  1
+#!/usr/bin/env bash
  2
+
  3
+# This script should be run when creating a new SPKG for shipping sagenb
  4
+# with Sage. For more complete instructions on how to generate a new
  5
+# SPKG, read the SPKG.txt file in the current SPKG.
  6
+#
  7
+# Usage: dist.sh [-g]
  8
+#
  9
+#   -g      Also package the git repository
  10
+
  11
+die () {
  12
+    echo >&2 "$@"
  13
+    exit 1
  14
+}
  15
+
  16
+while getopts ":g" opt; do
  17
+    case $opt in
  18
+        g) INSTALL_REPO=1 ;;
  19
+        *) die "Invalid option!"
  20
+    esac
  21
+done
  22
+
  23
+cd ${0%/*}
  24
+
  25
+if [ -n "$INSTALL_REPO" ]; then
  26
+    git diff --quiet && git diff --cached --quiet ||
  27
+        die "Uncommitted changes in sagenb - please commit, stash, or discard"
  28
+fi
  29
+
  30
+rm -rf dist
  31
+mkdir -p dist
  32
+
  33
+echo "Fetching source tarballs of (sub)dependencies of sagenb to dist/"
  34
+python util/fetch_deps.py dist || die "Couldn't fetch all (sub)dependencies"
  35
+
  36
+echo "Creating source tarball of sagenb itself in dist/"
  37
+python setup.py sdist > sdist.log || die "Couldn't make sagenb source tarball"
  38
+
  39
+if [ -n "$INSTALL_REPO" ]; then
  40
+    echo "Sanitizing sagenb git repo (with backup)"
  41
+    mv .git .git-backup
  42
+    git init
  43
+    git fetch .git-backup
  44
+    git remote add sagemath https://github.com/sagemath/sagenb
  45
+    git branch -f master FETCH_HEAD
  46
+    git remote update sagemath
  47
+    git branch --set-upstream master sagemath/master
  48
+    git reset --mixed
  49
+    git gc --aggressive --prune=0
  50
+
  51
+    echo "Moving sanitized sagenb git repo to dist/"
  52
+    mv .git dist/sagenb.git
  53
+
  54
+    echo "Restoring backup of git repo"
  55
+    mv .git-backup .git
  56
+fi
  57
+
  58
+echo "Done!"
7  go
... ...
@@ -1,7 +0,0 @@
1  
-#!/usr/bin/env pynb
2  
-import os
3  
-os.system('python setup.py install')
4  
-
5  
-import sagenb.notebook.notebook_object as n
6  
-n.notebook('sage_notebook', server_pool=[])
7  
-
36  sdist
... ...
@@ -1,36 +0,0 @@
1  
-#!/usr/bin/env bash
2  
-
3  
-# Check whether "sage" can be run (i.e. whether it's in the PATH)
4  
-if ! sage -v >/dev/null; then
5  
-    echo >&2 "Error: $0 requires 'sage' to be in your PATH"
6  
-    echo >&2 "Maybe first call 'SAGE_ROOT/sage -sh'"
7  
-    exit 1
8  
-fi
9  
-
10  
-echo "Trying to commit changes (if any)..."
11  
-sage -hg commit
12  
-status=$?
13  
-if [ $status -eq 1 ]; then
14  
-    # 'hg commit' returns 1 if there are no changes to commit...
15  
-    echo "No changes to commit (not an error)"
16  
-elif [ $status -ne 0 ]; then
17  
-    echo >&2 "Error: $0: Couldn't commit changes" \
18  
-             "('hg commit' failed with exit code $status)"
19  
-    exit 1
20  
-else
21  
-    # $status == 0
22  
-    echo "Successfully committed the changes"
23  
-fi
24  
-
25  
-echo "Running 'python setup.py sdist'..."
26  
-sage -python setup.py sdist
27  
-if [ $? -ne 0 ]; then
28  
-    echo >&2 "Error: $0: 'python setup.py sdist' failed"
29  
-    exit 1
30  
-fi
31  
-
32  
-
33  
-echo "**********************************************************"
34  
-echo "* If this is an official SageNB release, don't forget to *"
35  
-echo "* push the changes to the public Mercurial repository.   *"
36  
-echo "**********************************************************"
95  setup.py
@@ -2,64 +2,71 @@
2 2
 # The setup.py for the Sage Notebook
3 3
 ##########################################################
4 4
 
5  
-import os, sys, time
  5
+import os
6 6
 from setuptools import setup
7  
-
8  
-
9 7
 import distutils.log
10 8
 
11  
-if os.environ.get("SAGE_SETUPTOOLS_DEBUG","no")=="yes":
12  
-    distutils.log.set_threshold(distutils.log.DEBUG)
13 9
 
14 10
 
15  
-def all_files(dir, lstrip):
  11
+def lremove(string, prefix):
  12
+    while string.startswith(prefix):
  13
+        string = string[len(prefix):]
  14
+    return string
  15
+
  16
+def all_files(dir, prefix):
16 17
     """
17  
-    Return list of all filenames in the given directory, with lstrip
  18
+    Return list of all filenames in the given directory, with prefix
18 19
     stripped from the left of the filenames.
19 20
     """
  21
+
20 22
     X = []
21 23
     for F in os.listdir(dir):
22 24
         ab = dir+'/'+F
23 25
         if os.path.isfile(ab):
24  
-            X.append((ab).lstrip(lstrip))
  26
+            X.append(lremove(ab, prefix))
25 27
         elif os.path.isdir(ab):
26  
-            X.extend(all_files(ab, lstrip))
  28
+            X.extend(all_files(ab, prefix))
27 29
     return X
28 30
     
29 31
 
30  
-code = setup(name = 'sagenb',
31  
-      version     = '0.9.1',  # the spkg-dist script assumes single quotes here
32  
-      description = 'The Sage Notebook',
33  
-      license     = 'GNU Public License (GPL) v2+',
34  
-      author      = 'William Stein et al.',
35  
-      author_email= 'http://groups.google.com/group/sage-notebook',
36  
-      url         = 'http://code.google.com/p/sagenb',
37  
-      install_requires = [ 'twisted>=11.0.0'
38  
-                         , 'flask'
39  
-                         , 'flask-openid'
40  
-                         , 'flask-autoindex'
41  
-                         , 'babel'
42  
-                         , 'flask-babel'
43  
-                         , 'hg-git'
44  
-                         , 'pyOpenSSL<=0.12'
45  
-                         , 'webassets'
  32
+install_requires = [ 'twisted>=11.0.0'
  33
+                   , 'flask'
  34
+                   , 'flask-openid'
  35
+                   , 'flask-autoindex'
  36
+                   , 'babel'
  37
+                   , 'flask-babel'
  38
+                   , 'pyOpenSSL<=0.12'
  39
+                   , 'webassets'
  40
+                   ]
  41
+
  42
+if __name__ == '__main__':
  43
+    if os.environ.get("SAGE_SETUPTOOLS_DEBUG","no")=="yes":
  44
+        distutils.log.set_threshold(distutils.log.DEBUG)
  45
+
  46
+    code = setup(name = 'sagenb',
  47
+          version     = '0.9.1',
  48
+          description = 'The Sage Notebook',
  49
+          license     = 'GNU General Public License (GPL) v2+',
  50
+          author      = 'William Stein et al.',
  51
+          author_email= 'sage-notebook@googlegroups.com',
  52
+          url         = 'http://github.com/sagemath/sagenb',
  53
+          install_requires = install_requires,
  54
+          test_suite = 'sagenb.testing.run_tests.all_tests',
  55
+          packages    = [ 'sagenb'
  56
+                        , 'sagenb.interfaces'
  57
+                        , 'sagenb.misc'
  58
+                        , 'sagenb.notebook'
  59
+                        , 'sagenb.notebook.compress'
  60
+                        , 'sagenb.simple'
  61
+                        , 'sagenb.storage'
  62
+                        , 'sagenb.testing'
  63
+                        , 'sagenb.testing.tests'
  64
+                        , 'sagenb.testing.selenium'
  65
+                        ],
  66
+          scripts      = [ 'sagenb/data/sage3d/sage3d',
46 67
                          ],
47  
-      test_suite = 'sagenb.testing.run_tests.all_tests',
48  
-      packages    = [ 'sagenb'
49  
-                    , 'sagenb.interfaces'
50  
-                    , 'sagenb.misc'
51  
-                    , 'sagenb.notebook'
52  
-                    , 'sagenb.notebook.compress'
53  
-                    , 'sagenb.simple'
54  
-                    , 'sagenb.storage'
55  
-                    , 'sagenb.testing'
56  
-                    , 'sagenb.testing.tests'
57  
-                    , 'sagenb.testing.selenium'
58  
-                    ],
59  
-      scripts      = [ 'sagenb/data/sage3d/sage3d',
60  
-                     ],
61  
-      package_data = {'sagenb':
62  
-                          all_files('sagenb/data', 'sagenb/') +
63  
-                          all_files('sagenb/translations', 'sagenb/')
64  
-                     },
65  
-      )
  68
+          package_data = {'sagenb':
  69
+                              all_files('sagenb/data', 'sagenb/') +
  70
+                              all_files('sagenb/translations', 'sagenb/')
  71
+                         },
  72
+          )
257  spkg-dist
... ...
@@ -1,257 +0,0 @@
1  
-#!/usr/bin/env python
2  
-# Create a new Sage Notebook spkg.
3  
-#
4  
-# This spkg-dist doesn't need any Python modules of Sage, so it should
5  
-# be okay to run it from a system-wide Python (provided it is sufficiently
6  
-# recent).  This script *does* need a sage executable to be in the PATH
7  
-# because we need to run "sage -pkg" below and also the "sdist" script
8  
-# runs "sage -python" and "sage -hg". -- Jeroen Demeyer
9  
-
10  
-
11  
-import os
12  
-import sys
13  
-import stat
14  
-import shutil
15  
-import tarfile
16  
-import subprocess
17  
-from tempfile import mkdtemp
18  
-
19  
-from pkg_resources import Requirement
20  
-from setuptools.package_index import PackageIndex
21  
-
22  
-# Get the version from setup.py.
23  
-try:
24  
-    version_line = [f for f in open('setup.py').readlines() if 'version' in f][0]
25  
-except IndexError:
26  
-    sys.stderr.write("Error: Found no version string in 'setup.py'\n")
27  
-    sys.exit(1)
28  
-
29  
-i = version_line.find("'")
30  
-j = version_line.rfind("'")
31  
-if i==-1 or j==-1 or j-i<3:
32  
-    sys.stderr.write(
33  
-        "Error: Illegal version string in 'setup.py':\n    %s\n" % version_line)
34  
-    sys.stderr.write("Perhaps no single quotes used?\n")
35  
-    sys.exit(1)
36  
-version = version_line[i + 1:j]
37  
-print "New SageNB version: %s" % version
38  
-
39  
-# Create the source distribution.
40  
-print "Creating the new source tarball..."
41  
-sys.stdout.flush()
42  
-proc = subprocess.Popen([os.path.join(os.path.curdir, 'sdist')], shell=True)
43  
-if proc.wait():
44  
-    sys.stderr.write(
45  
-        "Error: './sdist' failed. See message(s) above for the specific error.\n")
46  
-    sys.exit(1)
47  
-
48  
-# Create the spkg.
49  
-base = 'sagenb-%s' % version
50  
-path = os.path.join('dist', base)
51  
-if os.path.exists(path):
52  
-    shutil.rmtree(path)
53  
-os.makedirs(path)
54  
-
55  
-file = 'sagenb-%s.tar.gz' % version # created by 'sdist' above
56  
-print "Extracting %s..." % file
57  
-sys.stdout.flush()
58  
-t = tarfile.open(os.path.join('dist', file))
59  
-t.extractall(path)
60  
-print "Finished extraction."
61  
-sys.stdout.flush()
62  
-
63  
-os.chdir(path) # cd dist/sagenb-x.y.z
64  
-os.mkdir('src')
65  
-print "Moving new source tree..."
66  
-sys.stdout.flush()
67  
-shutil.move(base, os.path.join('src', 'sagenb')) # mv sagenb-x.y.z src/sagenb
68  
-
69  
-print "Copying 'SPKG.txt'..."
70  
-sys.stdout.flush()
71  
-shutil.copy(os.path.join(os.path.pardir, os.path.pardir, 'SPKG.txt'),
72  
-            os.path.curdir) # cp ../../SPKG.txt . (which is dist/sagenb-x.y.z)
73  
-
74  
-print "Creating 'spkg-install'..."
75  
-sys.stdout.flush()
76  
-spkg_install = os.path.abspath(os.path.join(os.path.curdir, 'spkg-install'))
77  
-spkg_install_fd = open(spkg_install, 'w')
78  
-
79  
-
80  
-def fetch_packages():
81  
-    """
82  
-    Downloads the required dependencies and returns a string
83  
-    which installs the dependencies using easy_install
84  
-    """
85  
-    print "Fetching the required packages"
86  
-    pkg_index = PackageIndex()
87  
-
88  
-    tmp_dir = mkdtemp()
89  
-
90  
-    # in order of dependencies
91  
-    required_packages = ( 'twisted>=11.0.0'
92  
-                        , 'pytz>=2011n'
93  
-                        , 'Babel>=0.9.6'
94  
-                        , 'Werkzeug>=0.8.2'
95  
-                        , 'speaklater>=1.2'
96  
-                        , 'python-openid>=2.2.5'
97  
-                        , 'Flask>=0.8'
98  
-                        , 'Flask-Silk>=0.1.1'
99  
-                        , 'Flask-AutoIndex>=0.4.0'
100  
-                        , 'Flask-Babel>=0.8'
101  
-                        , 'Flask-OpenID>=1.0.1'
102  
-                        , 'dulwich>=0.8.0'
103  
-                        , 'hg-git>=0.3.1'
104  
-                        , 'pyOpenSSL<=0.12'
105  
-                        , 'webassets>=0.7.1'
106  
-                        )
107  
-    pkg_locations = []
108  
-
109  
-    for pkg in required_packages:
110  
-        print "Fetching %s" % pkg
111  
-        dist = pkg_index.fetch_distribution(Requirement.parse(pkg), tmp_dir,
112  
-                                            True, True)
113  
-        pkg_locations.append(os.path.abspath(dist.location))
114  
-
115  
-    install_str = ''
116  
-    for location in pkg_locations:
117  
-        shutil.copy(location, os.path.join('src', os.path.basename(location)))
118  
-
119  
-    return [os.path.basename(location) for location in pkg_locations]
120  
-
121  
-install_dependencies=""
122  
-for pkg_location in fetch_packages():
123  
-    # --allow-hosts=None prevents downloading
124  
-    install_dependencies += r"""
125  
-easy_install --allow-hosts=None %(pkg)s
126  
-if [ $? -ne 0 ]; then
127  
-    echo >&2 "Error installing %(pkg)s."
128  
-    exit 1
129  
-fi
130  
-
131  
-""" % {'pkg': pkg_location}
132  
-
133  
-
134  
-os.chdir(os.path.pardir) # cd .. (now in dist/)
135  
-
136  
-# Write the whole file from a single raw Python string:
137  
-# (The #! has to be on the first line of the script.)
138  
-spkg_install_fd.write(
139  
-r"""#!/usr/bin/env bash
140  
-
141  
-# spkg-install for SageNB, generated by SageNB's spkg-dist
142  
-
143  
-if [ -z "$SAGE_LOCAL" ]; then
144  
-    echo >&2 "SAGE_LOCAL undefined - exiting..."
145  
-    exit 1
146  
-fi
147  
-
148  
-cd src
149  
-[ -z "$CPATH" ] || CPATH="$CPATH":
150  
-[ -z "$LIBRARY_PATH" ] || LIBRARY_PATH="$LIBRARY_PATH":
151  
-export CPATH="$CPATH""$SAGE_LOCAL"/include
152  
-export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib
153  
-
154  
-%(install_dependencies)s
155  
-
156  
-cd sagenb
157  
-python setup.py install
158  
-if [ $? -ne 0 ]; then
159  
-    echo >&2 "Error running 'setup.py install'."
160  
-    exit 1
161  
-fi
162  
-
163  
-mkdir -p "$SAGE_ROOT/devel" # Create if it doesn't already exist
164  
-if [ $? -ne 0 ]; then
165  
-    echo >&2 "Error creating '$SAGE_ROOT/devel'."
166  
-    exit 1
167  
-fi
168  
-
169  
-echo "Copying SageNB package to '$SAGE_ROOT/devel/sagenb-main'..."
170  
-
171  
-if [ -d "$SAGE_ROOT/devel/sagenb-main" ]; then
172  
-    echo "Moving old SageNB package to '$SAGE_ROOT/devel/sagenb-main-old'..."
173  
-    rm -rf "$SAGE_ROOT/devel/sagenb-main-old"
174  
-    mv "$SAGE_ROOT/devel/sagenb-main" "$SAGE_ROOT/devel/sagenb-main-old"
175  
-    if [ $? -ne 0 ]; then
176  
-        echo >&2 "Error moving the old 'sagenb-main' branch."
177  
-        exit 1
178  
-    fi
179  
-fi
180  
-
181  
-rm -f "$SAGE_ROOT/devel/sagenb" # Delete just the link itself (if it exists)
182  
-
183  
-cd .. # Back to sagenb-x.y.z/src/
184  
-cp -pr sagenb "$SAGE_ROOT/devel/sagenb-main" # Creates new sagenb-main dir
185  
-if [ $? -ne 0 ]; then
186  
-    echo >&2 "Error copying the new SageNB package."
187  
-    exit 1
188  
-fi
189  
-
190  
-# Create an appropriate hgrc file for the target
191  
-cat > "$SAGE_ROOT/devel/sagenb-main/.hg/hgrc" <<"HEREDOC"
192  
-[diff]
193  
-git = true
194  
-
195  
-[extensions]
196  
-hggit=
197  
-
198  
-[git]
199  
-intree = True
200  
-HEREDOC
201  
-
202  
-cd "$SAGE_ROOT/devel"
203  
-ln -s sagenb-main sagenb # Create new symbolic link (deleted above)
204  
-if [ $? -ne 0 ]; then
205  
-    echo >&2 "Error creating symbolic link to '$SAGE_ROOT/devel/sagenb-main'."
206  
-    exit 1
207  
-fi
208  
-
209  
-# We use relative paths for relocatability.
210  
-cd "$SAGE_ROOT/devel/sagenb"
211  
-python setup.py develop --egg-path ../../../../devel/sagenb
212  
-if [ $? -ne 0 ]; then
213  
-    echo >&2 "Error running 'setup.py develop'."
214  
-    exit 1
215  
-fi
216  
-
217  
-cd "$SAGE_ROOT/local/lib/python/site-packages"
218  
-# Use >/dev/null instead of grep -q (which doesn't work on Solaris)
219  
-if ! grep sagenb easy-install.pth >/dev/null; then
220  
-    # Ugly work-around, we haven't found the real cause yet (see #10176):
221  
-    echo "No sagenb path found in 'easy-install.pth'"'!'
222  
-    echo "Adding relative sagenb path to 'easy-install.pth'..."
223  
-    sed -e '$ i \../../../../devel/sagenb' easy-install.pth > easy-install.pth.$$
224  
-    if [ $? -ne 0 ]; then
225  
-        echo >&2 "Error adding relative sagenb path to 'easy-install.pth'."
226  
-        exit 1
227  
-    fi
228  
-else
229  
-    echo "Making sagenb path in 'easy-install.pth' relative..."
230  
-    sed 's/^.*sagenb.*$/..\/..\/..\/..\/devel\/sagenb/' easy-install.pth > easy-install.pth.$$
231  
-    if [ $? -ne 0 ]; then
232  
-        echo >&2 "Error patching 'easy-install.pth' to have relative path to SageNB."
233  
-        exit 1
234  
-    fi
235  
-fi
236  
-
237  
-# Print paths for debugging
238  
-echo "Old path: '`grep sagenb easy-install.pth`'"
239  
-echo "New path: '`grep sagenb easy-install.pth.$$`'"
240  
-
241  
-# The following fails only on wrong file permissions etc.:
242  
-mv -f easy-install.pth.$$ easy-install.pth
243  
-if [ $? -ne 0 ]; then
244  
-    echo >&2 "Error overwriting original 'easy-install.pth'."
245  
-    exit 1
246  
-fi
247  
-""" % {'install_dependencies': install_dependencies})
248  
-spkg_install_fd.close()
249  
-os.chmod(spkg_install, stat.S_IRWXU | stat.S_IRGRP | stat.S_IXGRP |
250  
-         stat.S_IROTH | stat.S_IXOTH) # -rwxr-xr-x
251  
-
252  
-# We are still in dist/, now package sagenb-x.y.z/ :
253  
-print "Running 'sage -pkg %s'..."%base
254  
-sys.stdout.flush()
255  
-if subprocess.call(['sage', '-pkg', base]):
256  
-    sys.stderr.write('Error: "sage -pkg %s" failed.\nSee message(s) above for the specific error.\n'%base)
257  
-    sys.exit(1)
51  util/fetch_deps.py
... ...
@@ -0,0 +1,51 @@
  1
+#!/usr/bin/env python
  2
+"""
  3
+Downloads certain (sub)dependencies of sagenb into the given directory
  4
+and returns a list of their names. This script is run from dist.sh, and
  5
+need not be run independently.
  6
+"""
  7
+# These are linearly ordered such that no package depends on something
  8
+# lower than it in the list (or not in the list).
  9
+#
  10
+# TODO: Automatically generate this (using distribute internals ?)
  11
+required_packages = [ 'twisted>=11.0.0'
  12
+                    , 'pytz>=2011n'
  13
+                    , 'Babel>=0.9.6'
  14
+                    , 'Werkzeug>=0.8.2'
  15
+                    , 'speaklater>=1.2'
  16
+                    , 'python-openid>=2.2.5'
  17
+                    , 'Flask>=0.8'
  18
+                    , 'Flask-Silk>=0.1.1'
  19
+                    , 'Flask-AutoIndex>=0.4.0'
  20
+                    , 'Flask-Babel>=0.8'
  21
+                    , 'Flask-OpenID>=1.0.1'
  22
+                    , 'pyOpenSSL<=0.12'
  23
+                    , 'webassets>=0.7.1'
  24
+                    ]
  25
+
  26
+import os, shutil, sys
  27
+from pkg_resources import Requirement
  28
+from setuptools.package_index import PackageIndex
  29
+
  30
+def die(message):
  31
+    sys.stderr.write(message)
  32
+    sys.exit(1)
  33
+
  34
+if __name__ == '__main__':
  35
+    if len(sys.argv) < 2:
  36
+        print "Please supply a destination directory for the fetched packages!"
  37
+        sys.exit(1)
  38
+    dest_dir = os.path.abspath(sys.argv[1])
  39
+
  40
+    print "Fetching packages:"
  41
+    pkg_index = PackageIndex()
  42
+    with open(os.path.join(dest_dir, 'install_order'), 'w') as fd:
  43
+        for pkg in required_packages:
  44
+            print "(---  Processing requirement '{0}'".format(pkg)
  45
+            dist = pkg_index.fetch_distribution(Requirement.parse(pkg),
  46
+                                                dest_dir, True, True)
  47
+            if dist is None:
  48
+                die("Couldn't find package satisfying '{0}'!".format(pkg))
  49
+            print " ---) Fetched {0} {1}".format(dist.project_name,
  50
+                    dist.version)
  51
+            fd.write(os.path.basename(dist.location) + '\n')
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.