Skip to content
This repository has been archived by the owner on Jun 18, 2020. It is now read-only.

Rewrite packaging system, rip out Mercurial #63

Merged
merged 19 commits into from Jun 20, 2012
Merged

Conversation

kini
Copy link
Collaborator

@kini kini commented Jun 14, 2012

I've considerably simplified the packaging system. The new SPKG is
uncompressed, and has a Mercurial repository tracking spkg-install et
al. like all other SPKGs do. The src/ directory consists of a bunch of
source tarballs of dependencies, an sdist tarball of sagenb itself,
and (optionally) a copy of the git repository of sagenb suitable for
beginning development from.

References to Mercurial, dependencies on Mercurial tools, etc. are also
removed.

kini added 14 commits June 12, 2012 11:33
depdist.sh will be used to create a dist/ directory which should be
moved to the sagenb spkg directory and renamed to src/ .

fetch_deps.py is a helper script for depdist.sh which uses distribute
internals to fetch package tarballs from PyPI.
`setup.py sdist` doesn't copy empty directories, as I found out after
a lot of head scratching. This breaks git repos, so we can't trust sdist
to copy our git repo into the source tarball of sagenb. So we'll store
the git repo separately as sagenb.git , and move it into place when
running spkg-install.
@kini kini mentioned this pull request Jun 14, 2012
@kini
Copy link
Collaborator Author

kini commented Jun 14, 2012

Here's an example of a sagenb SPKG created by dist.sh after these commits: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

You will need to take a look at it in order to review this branch because it contains the new SPKG repo I made for sagenb.

Apparently you can't set a branch to track a remote tracking branch that
you haven't fetched yet.
@kini
Copy link
Collaborator Author

kini commented Jun 14, 2012

SPKG updated according to the latest commit: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg (same link)

@jhpalmieri
Copy link
Member

Would it make any sense to omit the git repository and instead document how to install it for people who want to work on sagenb development?

@kini
Copy link
Collaborator Author

kini commented Jun 15, 2012

In my opinion, yes, absolutely. Sage currently ships its repositories inside both source and binary tarballs, which is unorthodox to say the least. The stated goal is to make the barrier of entry to development extremely low in order to gather mathematicians who are not experienced software engineers and open-source development collaborators, but who nonetheless have been hacking on Sage for their own purposes/research, and entice them to give us their code by making it extremely easy to do so. But I don't think this goal makes much sense in the context of sagenb, which is a web app and doesn't really seek contributions from inexperienced programmers who are experts in some other field.

kini added 3 commits June 15, 2012 16:56
This old advice for release management of sagenb was in the SPKG.txt and
was generally out of date. Since the SPKG will now only contain a bunch
of tarballs, a lot of those instructions don't make much sense there, so
I've moved them to the sagenb README for now.
New behavior: `./dist.sh` prepares tarballs of sagenb and its
dependencies in dist/ , and `./dist.sh -g` additionally places an
optimized copy of the git repository in dist/
@kini
Copy link
Collaborator Author

kini commented Jun 16, 2012

OK, so packaging the git repo is now optional. Pass -g to dist.sh if you want a git repo.

New SPKG is up at the same URL: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

@jhpalmieri
Copy link
Member

When I run dist.sh, I see the following:

Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!

Then everything seems to get downloaded successfully. I also see

warning: sdist: standard file not found: should have one of README, README.txt

I have no idea what this means.

Otherwise, everything looks good, and the new SPKG.txt file has good directions for how to use dist.sh. Should SPKG.txt list the non-included dependencies also, like Python, to record where it should go in SAGE_ROOT/spkg/standard/deps?

@jhpalmieri
Copy link
Member

Oh, it might be a good idea to provide instructions (in SPKG.txt? at the top of dist.sh?) for how to use the git repository in the directory sagenb.git, if you run dist.sh -g. (I think you should unpack sagenb-0.9.1.tar.gz and then move sagenb.git to sagenb-0.9.1/.git. Is that right?)

@kini
Copy link
Collaborator Author

kini commented Jun 18, 2012

When I run dist.sh, I see the following:

Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!
Download error: [Errno 61] Connection refused -- Some packages may not be found!

Those don't appear on a stable connection, such as on sage.math.

Then everything seems to get downloaded successfully. I also see

warning: sdist: standard file not found: should have one of README, README.txt

I have no idea what this means.

To make sure github renders it correctly, our readme is called README.rst, so setuptools complains - this can be ignored (it's only a warning).

Otherwise, everything looks good, and the new SPKG.txt file has good directions for how to use dist.sh. Should SPKG.txt list the non-included dependencies also, like Python, to record where it should go in SAGE_ROOT/spkg/standard/deps?

Good idea.

@jhpalmieri
Copy link
Member

Those don't appear on a stable connection, such as on sage.math.

I just tested, and I also see them on sage.math (although with "Errno 111" instead of "Errno 61"). Also, they are completely reproducible: I see four of these "Download error" messages every time I run the script. Maybe you could add a print statement print "Fetching {0} ...".format(pkg) at the start of the loop, so that the later message "Fetched ..." will reassure the user that the desired package was retrieved successfully.

@kini
Copy link
Collaborator Author

kini commented Jun 19, 2012

Oh, it might be a good idea to provide instructions (in SPKG.txt? at the top of dist.sh?) for how to use the git repository in the directory sagenb.git, if you run dist.sh -g. (I think you should unpack sagenb-0.9.1.tar.gz and then move sagenb.git to sagenb-0.9.1/.git. Is that right?)

No, the instructions are correct - spkg-install will install the git repo correctly if sagenb.git is present in the src directory.

@kini
Copy link
Collaborator Author

kini commented Jun 19, 2012

Those don't appear on a stable connection, such as on sage.math.

I just tested, and I also see them on sage.math (although with "Errno 111" instead of "Errno 61"). Also, they are completely reproducible: I see four of these "Download error" messages every time I run the script. Maybe you could add a print statement print "Fetching {0} ...".format(pkg) at the start of the loop, so that the later message "Fetched ..." will reassure the user that the desired package was retrieved successfully.

Good idea. I'll add it when I'm back in front of a computer in a day or so :)

@jhpalmieri
Copy link
Member

No, the instructions are correct - spkg-install will install the git repo correctly if sagenb.git is present in the src directory.

Oh, right. Sorry.

@kini
Copy link
Collaborator Author

kini commented Jun 20, 2012

OK, I've made the changes you suggested - the ones to the dist scripts in the above commit, and the ones to SPKG.txt in the new SPKG I've just uploaded to the same URL as before: http://wstein.org/home/keshav/files/sagenb-0.9.1.spkg

@jhpalmieri
Copy link
Member

I think this looks good now. I've reviewed the changes and tried out dist.sh. Everything seems to work. Does that suffice for a positive review?

@kini
Copy link
Collaborator Author

kini commented Jun 20, 2012

Yup! Thanks for helping me out with this!

kini added a commit that referenced this pull request Jun 20, 2012
Rewrite packaging system, rip out Mercurial
@kini kini merged commit b0ce4e7 into sagemath:master Jun 20, 2012
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants