upgrade ZODB to version 3.10.5 #10352

williamstein · 2010-11-28T21:37:08Z

Upgrade ZODB to the latest upstream version.

The new spkg is here:

http://www.stp.dias.ie/~vbraun/Sage/spkg/zodb3-3.10.5.spkg

NOTE ABOUT spkg-check

I tried adding an spkg-check, but the fact is that the ZODB test suite fails miserably all over the place. This is true both of the version in Sage (3.7.0) and the version posted here. The failures start with:

(sage subshell) deep:src wstein$ python test.py
Running tests from build/lib.macosx-10.6-i386-2.6
test.py:44: DeprecationWarning: zope.testing.testrunner is deprecated in favour of zope.testrunner.

but there is no zope.testrunner so the DeprecationWarning is wrong!! And it gets worse from there. Tons of key errors, other failures, then a total hang.

Regarding spkg-check, here's what it should look like:

#!/usr/bin/env bash
cd src/
python test.py

williamstein · 2010-11-28T21:50:32Z

comment:2

ZODB is used by the database_cremona optional spkg. Thus a way to test this package is to install that optional package, then do:

sage: EllipticCurve("30001a1")
Elliptic Curve defined by y^2 + y = x^3 + x^2 - 171*x - 6408 over Rational Field
sage: CremonaDatabase().largest_conductor()
130000

and not get an error. Just that is enough to show that the ZODB is doing pretty much all we use it for in Sage. Running the test suite for ZODB is unfortunately not possible :-(.

Also, be sure to test the "devel/sage/sage/databases" directory, and you of course should do "make test" in SAGE_ROOT to test the whole library.

williamstein · 2010-11-29T01:08:59Z

comment:4

I just tried installing this and at the end got:


Finished processing dependencies for ZODB3==3.7.0

real	0m20.027s
user	0m14.322s
sys	0m2.886s
Successfully installed zodb3-3.10.1

so somehow setuptools is still thinking partly that it needs to install ZODB 3.7.0. So screw it. I'm changing this to needs work and not messing with it anymore.

cschwan · 2011-10-03T10:28:46Z

comment:6

Ticket #1634 should be marked as a duplicate.

simon-king-jena · 2012-12-28T22:43:09Z

comment:7

I am bumping it up to "major", since it turns out that zodb screws up refcounts, which makes the debug version of Sage (#13864) fail.

simon-king-jena · 2012-12-28T23:10:01Z

comment:8

Can we change this ticket and update to zodb3-3.10.5? Namely, in contrast to 3.10.1, version 3.10.5 does fix one of the problems from #13864.

nbruin · 2012-12-28T23:15:35Z

comment:9

Replying to @simon-king-jena:

Can we change this ticket and update to zodb3-3.10.5? Namely, in contrast to 3.10.1, version 3.10.5 does fix one of the problems from #13864.

Will do. Got an spkg ready here.

nbruin · 2012-12-28T23:22:08Z

comment:10

For review purposes, I've just taken William's original upgrade to 3.10.1 and dumped 3.10.5 in instead.
http://sage.math.washington.edu/home/wstein/patches/zodb3-3.10.1.spkg

As it turns out, our present ZODB3 version does not support Python 2.7, so upgrading is very much recommended.

nbruin · 2012-12-28T23:22:35Z

Author: William Stein, Nils Bruin

kiwifb · 2012-12-29T08:50:06Z

comment:13

Updating could be done but some of us would rather ditch zodb altogether if someone with the know how would work on converting conway polynomial at #12205 which may have been done already and that I missed. See also #10353.

jdemeyer · 2012-12-30T12:27:12Z

comment:14

There are some version mismatches, this needs

http://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.0.2.tar.gz
http://pypi.python.org/packages/source/z/zope.exceptions/zope.exceptions-4.0.4.zip

nbruin · 2012-12-30T18:59:57Z

comment:15

Thanks for catching! spkg updated.

vbraun · 2013-01-01T11:09:07Z

comment:18

How about

zope.interface will be included in the sagenb spkg
We fix up the conway polynomials (rewrite conway polynomial spkg and code in Sage library to not use ZODB #12205)
The fixed zodb spkg from this ticket will be an optional spkg
The zodb spkg will be a dependency for any optional spkg that needs it

simon-king-jena · 2013-01-01T11:58:30Z

comment:19

Replying to @vbraun:

How about

zope.interface will be included in the sagenb spkg

Why should an independent module only be shipped as part of another independent module?

Even if zope.interface was currently only used by sagenb, I'd still think zope.interface should be an separate spkg.

We fix up the conway polynomials (rewrite conway polynomial spkg and code in Sage library to not use ZODB #12205)

The fixed zodb spkg from this ticket will be an optional spkg

The zodb spkg will be a dependency for any optional spkg that needs it

That sounds good to me.

vbraun · 2013-01-01T12:38:53Z

comment:20

The sagenb spkg contains already a dozen of python dependencies:

sagenb-0.10.2/src/Flask-OpenID-1.0.1.tar.gz
sagenb-0.10.2/src/Werkzeug-0.8.3.tar.gz
sagenb-0.10.2/src/Flask-AutoIndex-0.4.1.tar.gz
sagenb-0.10.2/src/Twisted-12.1.0.tar.bz2
sagenb-0.10.2/src/Babel-0.9.6.zip
sagenb-0.10.2/src/Flask-Silk-0.1.1.tar.gz
sagenb-0.10.2/src/pytz-2012d.zip
sagenb-0.10.2/src/python-openid-2.2.5.zip
sagenb-0.10.2/src/Flask-Babel-0.8.tar.gz
sagenb-0.10.2/src/Flask-0.9.tar.gz
sagenb-0.10.2/src/speaklater-1.3.tar.gz
sagenb-0.10.2/src/webassets-0.7.1.tar.gz

jdemeyer · 2013-01-01T12:46:48Z

comment:21

Not sure exactly what changed in the new spkg, but there are still version mismatches:

http://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.0.3.tar.gz
http://pypi.python.org/packages/source/z/zope.exceptions/zope.exceptions-4.0.5.tar.gz

simon-king-jena · 2013-01-01T15:24:15Z

comment:22

Replying to @vbraun:

The sagenb spkg contains already a dozen of python dependencies:

Is sagenb a system that should be able to run independent of another system, or is sagenb supposed to be installed on top of another system (such as Sage)? In the former case, I could understand why sagenb comes with batteries included. But if sagenb is intended to be a small brick in a large wall, then it seems to me that a small brick shouldn't attempt to contain other small bricks...

But I am not a software engineer. Anyway, the fact that sagenb contains a lot of dependencies (taking the risk that the same dependency is provided twice, perhaps even in different versions) should not force us to add yet another module to sagenb.

williamstein · 2013-01-01T15:29:49Z

comment:23

Replying to @simon-king-jena:

Replying to @vbraun:

The sagenb spkg contains already a dozen of python dependencies:

Is sagenb a system that should be able to run independent of another system?

Yes, it is supposed to be something that can run without Sage (it is supposed to only require python). I don't know to what extent that is currently the case.

William

nbruin · 2013-01-01T17:47:21Z

comment:24

Replying to @jdemeyer:

Not sure exactly what changed in the new spkg, but there are still version mismatches:
http://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.0.3.tar.gz
http://pypi.python.org/packages/source/z/zope.exceptions/zope.exceptions-4.0.5.tar.gz

These are different versions than you reported before. I put the files in there you requested last time.

to any zope version expert: please put versions in there that make Jeroen happy.

jdemeyer · 2013-01-02T09:13:59Z

comment:25

Perhaps the problem is simply the order of packages installed. transaction is installed before the zope stuff, but transaction requires zope.interface.

vbraun · 2013-01-04T11:26:21Z

comment:26

Can somebody actually post a log from a failed install attempt? It works for me. The version mismatches shouldn't matter either, distutils is checking that each package's dependencies are satisfied. You don't need matching versions of all zope* packages (indeed, they are not in sync).

Also I don't understand what Jeroen posted, the spkg from the ticket issue contains zope.exceptions-4.0.4.zip and zope.interface-4.0.2.tar.gz and not the versions you mention.

jdemeyer · 2013-01-04T11:38:15Z

comment:27

Replying to @vbraun:

Can somebody actually post a log from a failed install attempt? It works for me. The version mismatches shouldn't matter either, distutils is checking that each package's dependencies are satisfied. You don't need matching versions of all zope* packages (indeed, they are not in sync).

The problem is this:

Processing transaction-1.1.1.tar.gz
Running transaction-1.1.1/setup.py -q bdist_egg --dist-dir /tmp/release/easy_install-FMO44V/transaction-1.1.1/egg-dist-tmp-B7YtEa
Adding transaction 1.1.1 to easy-install.pth file

Installed /release/merger/sage-5.6.beta2-debug/local/lib/python2.7/site-packages/transaction-1.1.1-py2.7.egg
Processing dependencies for transaction==1.1.1
Searching for zope.interface
Reading http://pypi.python.org/simple/zope.interface/
Best match: zope.interface 4.0.3
Downloading http://pypi.python.org/packages/source/z/zope.interface/zope.interface-4.0.3.tar.gz#md5=1ddd308f2c83703accd1696158c300eb
Processing zope.interface-4.0.3.tar.gz
Running zope.interface-4.0.3/setup.py -q bdist_egg --dist-dir /tmp/release/easy_install-86Axhl/zope.interface-4.0.3/egg-dist-tmp-2mKFn_
Adding zope.interface 4.0.3 to easy-install.pth file

Installed /release/merger/sage-5.6.beta2-debug/local/lib/python2.7/site-packages/zope.interface-4.0.3-py2.7-linux-x86_64.egg
Finished processing dependencies for transaction==1.1.1

When transaction is installing, zope.interface hasn't been installed yet and must be downloaded from the internet.

jpflori · 2013-01-04T12:08:18Z

comment:28

I guess you can indeed easily reproduce that by shutting down your internet connection and making sure no previous installation of zope.interface is available (e.g. do not upgrade the zodb spkg on an existing sage install).

vbraun · 2013-01-04T13:27:34Z

Changed author from William Stein, Nils Bruin to William Stein, Nils Bruin, Volker Braun

vbraun · 2013-01-04T13:27:34Z

comment:29

upgraded to zope.interface-4.0.3
using easy_install --index-url dependencies <package> to use the tarballs from the dependencies directory

I've built sage-5.6.beta2 from scratch with the new spkg and verified that zodb is not downloading anything (that is, no "Downloading" or urls in the spkg log).

jdemeyer · 2013-01-07T09:51:15Z

comment:30

Looks good to me (but it's going to be removed anyway, right?)

jdemeyer · 2013-01-07T09:51:15Z

Reviewer: Jeroen Demeyer

vbraun · 2013-01-07T11:46:38Z

comment:31

Its probably premature to remove it in sage-5.6, let's first fix the sagenb dependency. I've created sagemath/sagenb#126 to raise awareness on the sagenb side.

jdemeyer · 2013-01-07T20:55:18Z

Merged: sage-5.6.beta3

kcrisman · 2015-01-06T14:48:41Z

comment:33

Just as a point of information, this was indeed added to upstream, see also #10353 where that was confirmed.

williamstein added this to the sage-5.6 milestone Nov 28, 2010

williamstein added c: packages: standard labels Nov 28, 2010