upgrade sympy to version 0.7.5 #16624

rwst · 2014-07-08T09:34:59Z

It's at https://github.com/sympy/sympy/releases/download/sympy-0.7.5/sympy-0.7.5.tar.gz

For starters the better diophantine solver abilities are needed by #16590

Reported as sympy/sympy#8022

Upstream: Reported upstream. Developers acknowledge bug.

CC: @kiwifb

Component: packages: standard

Keywords: upgrade, mpmath

Author: Ralf Stephan, Sergey B Kirpichev

Branch/Commit: e6c24d3

Reviewer: Volker Braun

Issue created by migration from https://trac.sagemath.org/ticket/16624

The text was updated successfully, but these errors were encountered:

rwst · 2014-07-09T13:16:19Z

Branch: u/rws/upgrade_sympy_to_version_0_7_5

rwst · 2014-07-09T13:18:23Z

Author: Ralf Stephan

rwst · 2014-07-09T13:18:23Z

comment:2

No idea what's the matter here:

Traceback (most recent call last):
  File "/home/ralf/sage/src/bin/sage-runtests", line 87, in <module>
    err = DC.run()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 904, in run
    self.run_doctests()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/control.py", line 674, in run_doctests
    self.dispatcher = DocTestDispatcher(self)
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 1335, in __init__
    init_sage()
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 113, in init_sage
    from sympy.printing.pretty.stringpict import stringPict
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py", line 32, in <module>
    from .core import *
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/__init__.py", line 8, in <module>
    from .expr import Expr, AtomicExpr
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/expr.py", line 7, in <module>
    from .evalf import EvalfMixin, pure_complex
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/core/evalf.py", line 9, in <module>
    import sympy.mpmath.libmp as libmp
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/__init__.py", line 6, in <module>
    from .ctx_mp import MPContext
  File "/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/ctx_mp.py", line 47, in <module>
    from sage.libs.mpmath.ext_main import Context as BaseMPContext
  File "ext_main.pyx", line 38, in init sage.libs.mpmath.ext_main (build/cythonized/sage/libs/mpmath/ext_main.c:28725)
  File "/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/__init__.py", line 6, in <module>
    from .ctx_mp import MPContext
  File "/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py", line 49, in <module>
    import sage.libs.mpmath.ext_main as _mpf_module
AttributeError: 'module' object has no attribute 'ext_main'
make: *** [ptestlong] Error 1

New commits:

`3cc6ca3`	`16624: sympy version 0.7.5`

rwst · 2014-07-09T13:18:23Z

Commit: 3cc6ca3

sagetrac-git · 2014-07-11T07:59:38Z

Changed commit from 3cc6ca3 to c280b72

sagetrac-git · 2014-07-11T07:59:38Z

Branch pushed to git repo; I updated commit sha1. New commits:

`c280b72`	`16624: fix import issue`

rwst · 2014-07-11T08:00:06Z

comment:4

Even with sympy-0.7.4 in a fresh Sage I get

sage:     import sage.libs.mpmath.ext_main as _mpf_module
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-06f00c8c1fbc> in <module>()
----> 1 import sage.libs.mpmath.ext_main as _mpf_module

/home/ralf/sage/ext_main.pyx in init sage.libs.mpmath.ext_main (build/cythonized/sage/libs/mpmath/ext_main.c:28725)()

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/__init__.py in <module>()
      4 
      5 from .ctx_fp import FPContext
----> 6 from .ctx_mp import MPContext
      7 from .ctx_iv import MPIntervalContext
      8 

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py in <module>()
     47     from sage.libs.mpmath.ext_main import Context as BaseMPContext
     48     # pickle hack
---> 49     import sage.libs.mpmath.ext_main as _mpf_module
     50 else:
     51     from .ctx_mp_python import PythonMPContext as BaseMPContext

AttributeError: 'module' object has no attribute 'ext_main'

This and the 0.7.5 failure both can be cured for example with

--- mpmath/libmp/backend.py~    2013-08-08 14:49:58.000000000 +0200
+++ mpmath/libmp/backend.py     2014-07-11 08:54:28.759388954 +0200
@@ -71,6 +71,7 @@
     try:
         import sage.all
         import sage.libs.mpmath.utils as _sage_utils
+        import sage.libs.mpmath.ext_main as _mpf_module
         sage = sage.all
         sage_utils = _sage_utils
         BACKEND = 'sage'

So it seems ext_main has to be imported in the right place and there was somewhere an import in sympy-0.7.4 that made that version work but that was changed in 0.7.5. It does not work to simply catch the error and import again, while it works when in a fresh Sage with 0.7.4.

kiwifb · 2014-08-12T22:01:16Z

comment:7

The whole thing seems to go in circle. I am wondering if the real problem is that first a function is imported from ext_main

/home/ralf/sage/local/lib/python2.7/site-packages/mpmath/ctx_mp.py in <module>()
     47     from sage.libs.mpmath.ext_main import Context as BaseMPContext

and then you want to import the whole ext_main

     48     # pickle hack
---> 49     import sage.libs.mpmath.ext_main as _mpf_module

I wonder what they mean by pickle hack? Presumably if we add the import in backend.py the import in ctx_mp.py becomes redundant?

rwst · 2014-09-12T07:46:10Z

comment:8

I tried this patch but the results are the same:

diff --git a/build/pkgs/sympy/patches/import-extmain.patch b/build/pkgs/sympy/patches/import-extmain.patch
new file mode 100644
index 0000000..af222c1
--- /dev/null
+++ b/build/pkgs/sympy/patches/import-extmain.patch
@@ -0,0 +1,15 @@
+--- sympy/mpmath/ctx_mp.py~   2014-02-22 20:13:32.000000000 +0100
++++ sympy/mpmath/ctx_mp.py    2014-09-11 18:09:11.877599866 +0200
+@@ -44,9 +44,9 @@
+                          r'(?P<im>[\+\-]?\d*\.?\d*(e[\+\-]?\d+)?j)?\)?$')
+ 
+ if BACKEND == 'sage':
+-    from sage.libs.mpmath.ext_main import Context as BaseMPContext
+     # pickle hack
+-    import sage.libs.mpmath.ext_main as _mpf_module
++    from sage.libs.mpmath import ext_main as _mpf_module
++    from sage.libs.mpmath.ext_main import Context as BaseMPContext
+ else:
+     from .ctx_mp_python import PythonMPContext as BaseMPContext
+     from . import ctx_mp_python as _mpf_module
+

To be specific, at the moment it's not possible to import anything from sympy. What happens is that at the first attempt the following is raised:

----> 1 from sympy import sympify

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py in <module>()
     33 from .logic import *
     34 from .assumptions import *
---> 35 from .polys import *
     36 from .series import *
     37 from .functions import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import polytools
      6 __all__.extend(polytools.__all__)
      7 from .polytools import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/polytools.py in <module>()                                                                                      
     50 import sympy.mpmath
     51 
---> 52 from sympy.polys.domains import FF, QQ
     53 from sympy.polys.constructor import construct_domain
     54 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/__init__.py in <module>()
     76 QQ_gmpy = GMPYRationalField
     77 
---> 78 RR = RealField()
     79 CC = ComplexField()
     80 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/realfield.py in __init__(self, prec, dps, tol)
     52 
     53         self.dtype = context.mpf
---> 54         self.zero = self.dtype(0)
     55         self.one = self.dtype(1)
     56 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/ctx_mp_python.py in __new__(cls, val, **kwargs)
     75         else:
     76             v = new(cls)
---> 77             v._mpf_ = mpf_pos(cls.mpf_convert_arg(val, prec, rounding), prec, rounding)
     78             return v
     79 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/mpelements.py in _set_mpf(self, val)
     23         tol = self.context.tol
     24 
---> 25         if mpf_lt(mpf_abs(val, prec, rounding), tol):
     26             self.__mpf__ = fzero
     27         else:

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/mpmath/libmp/libmpf.py in mpf_abs(s, prec, rnd)
    662             return (0, man, exp, bc)
    663         return s
--> 664     return normalize1(0, man, exp, bc, prec, rnd)
    665 
    666 def mpf_sign(s):

TypeError: Argument 'man' has incorrect type (expected sage.rings.integer.Integer, got long)

and at any further attempt this one:

----> 1 from sympy.solvers.diophantine import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/__init__.py in <module>()
     33 from .logic import *
     34 from .assumptions import *
---> 35 from .polys import *
     36 from .series import *
     37 from .functions import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import polytools
      6 __all__.extend(polytools.__all__)
      7 from .polytools import *

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/polytools.py in <module>()                                                                                      
     50 import sympy.mpmath
     51 
---> 52 from sympy.polys.domains import FF, QQ
     53 from sympy.polys.constructor import construct_domain
     54 

/home/ralf/sage/local/lib/python2.7/site-packages/sympy/polys/domains/__init__.py in <module>()
      3 __all__ = []
      4 
----> 5 from . import domain
      6 __all__.extend(domain.__all__)
      7 from .domain import *

ImportError: cannot import name domain

rwst · 2014-09-12T08:15:52Z

Upstream: Reported upstream. No feedback yet.

skirpichev · 2014-09-30T16:14:19Z

comment:10

Attachment: 0001-Update-the-sympy-package.patch.gz

If you include patch to unbundle mpmath (see attached patch) - it will solve your problem.

skirpichev · 2014-09-30T16:16:25Z

Changed upstream from Reported upstream. No feedback yet. to Reported upstream. Developers acknowledge bug.

kiwifb · 2014-10-02T08:50:48Z

comment:12

Replying to @skirpichev:

If you include patch to unbundle mpmath (see attached patch) - it will solve your problem.

I am a bit surprised by this assertion. Gentoo and by extension sage-on-gentoo unbundle mpmath (I was the person who introduced unbundling of mpmath from sympy on Gentoo). Yet I can see this bug with sympy 0.7.4.1 and mpmath 0.19. Your patch must do something else apart from unbundling.

skirpichev · 2014-10-02T10:02:25Z

comment:13

I am a bit surprised by this assertion.

It does work for me. Can you reproduce the bug with this patch?

Gentoo and by extension sage-on-gentoo unbundle mpmath

This:
http://dev.gentoo.org/~bicatali/distfiles/sympy-0.7.4.1-system-mpmath.patch.gz
?

Perhaps, it's wrong patch and it just break sympy. Can you post the traceback for failure in this version?

Your patch must do something else apart from unbundling.

Sure. "Something else" - SPKG.txt changes (e.g. Homepage field, changelog and so on). But the only real change is unbundling mpmath.

kiwifb · 2014-10-03T02:09:36Z

comment:14

Your big patch is unreadable - too big. most of it is the removal of mpmath sources from sympy. This could be achieved in spkg-src by doing a "rm -rf" in the appropriate place. Once that's out of the way we can look at the patch to do the rest and it won't be 2.1MB and we'll be able to see something.

rwst · 2014-10-03T08:09:17Z

comment:15

It looks like the only file not being deleted is mpmath/tests/__init__.py.

rwst · 2014-10-03T08:09:17Z

Changed commit from c280b72 to none

rwst · 2014-10-03T08:09:17Z

Changed branch from u/rws/upgrade_sympy_to_version_0_7_5 to public/sympy075

rwst · 2014-10-03T08:09:17Z

Changed author from Ralf Stephan to Ralf Stephan, Sergey B Kirpichev

rwst · 2014-10-03T08:09:17Z

Changed keywords from none to upgrade, mpmath

sagetrac-git · 2014-10-03T09:51:31Z

Branch pushed to git repo; I updated commit sha1. New commits:

`6365073`	`shortened patch`

sagetrac-git · 2014-10-03T09:51:31Z

Commit: 6365073

rwst · 2014-10-03T09:52:51Z

comment:17

More can be deleted (mpmath docs). This passes make ptestlong.

sagetrac-git · 2014-10-03T13:06:57Z

Branch pushed to git repo; I updated commit sha1. New commits:

`04d8d62`	`remove doc/src/modules/mpmath from patch; remove files via spkg-install`

sagetrac-git · 2014-10-03T13:06:57Z

Changed commit from 6365073 to 04d8d62

rwst · 2014-10-03T13:08:43Z

comment:19

 build/pkgs/sympy/patches/01_no-mpmath.patch | 841 +++++++++++++++++++++++++++++++++++

Now that's more like it. Please review.

skirpichev · 2014-10-03T18:22:15Z

comment:20

Your big patch is unreadable - too big.

It's upstream patch. Does it harm your patch utility, what's wrong?)

I don't see points to modify it, unless you want break this patch just like gentoo one...

remove files via spkg-install

remove sympy/mpmath/tests/init.py too

sagetrac-git · 2014-10-04T12:57:11Z

Changed commit from 04d8d62 to 4c9f288

sagetrac-git · 2014-10-04T12:57:11Z

Branch pushed to git repo; I updated commit sha1. New commits:

`4c9f288`	`16624: remove mpmath/tests/__init__.py too`

vbraun · 2014-11-02T17:29:37Z

comment:22

lgtm

vbraun · 2014-11-02T17:29:43Z

Reviewer: Volker Braun

jdemeyer · 2014-11-03T09:10:36Z

comment:24

When applying patches, please use the code from
http://www.sagemath.org/doc/developer/packaging.html#patching-sources

Otherwise patching will break needlessly if there no patches.

sagetrac-git · 2014-11-03T09:30:52Z

Changed commit from 4c9f288 to e6c24d3

sagetrac-git · 2014-11-03T09:30:52Z

Branch pushed to git repo; I updated commit sha1. New commits:

`60f9af7`	`Merge branch 'develop' into t/16624/public/sympy075`
`e6c24d3`	`16624: use standard patch script in spkg-install`

vbraun · 2014-11-14T21:01:27Z

Changed branch from public/sympy075 to e6c24d3

rwst added this to the sage-6.3 milestone Jul 8, 2014

rwst added c: packages: standard labels Jul 8, 2014

sagetrac-vbraun-spam mannequin modified the milestones: sage-6.3, sage-6.4 Aug 10, 2014

This comment has been minimized.

Sign in to view

rwst added the s: needs info label Sep 12, 2014

skirpichev mannequin added s: needs review and removed s: needs info labels Sep 30, 2014

vbraun added s: positive review and removed s: needs review labels Nov 2, 2014

jdemeyer added s: needs work and removed s: positive review labels Nov 3, 2014

rwst added s: positive review and removed s: needs work labels Nov 3, 2014

vbraun removed the s: positive review label Nov 14, 2014

vbraun closed this as completed in 6807f53 Nov 14, 2014

upgrade sympy to version 0.7.5 #16624

upgrade sympy to version 0.7.5 #16624

Comments

rwst commented Jul 8, 2014

rwst commented Jul 9, 2014

rwst commented Jul 9, 2014

rwst commented Jul 9, 2014

rwst commented Jul 9, 2014

sagetrac-git mannequin commented Jul 11, 2014

sagetrac-git mannequin commented Jul 11, 2014

rwst commented Jul 11, 2014

kiwifb commented Aug 12, 2014

rwst commented Sep 12, 2014

This comment has been minimized.

rwst commented Sep 12, 2014

skirpichev mannequin commented Sep 30, 2014

skirpichev mannequin commented Sep 30, 2014

kiwifb commented Oct 2, 2014

skirpichev mannequin commented Oct 2, 2014

kiwifb commented Oct 3, 2014

rwst commented Oct 3, 2014

rwst commented Oct 3, 2014

rwst commented Oct 3, 2014

rwst commented Oct 3, 2014

rwst commented Oct 3, 2014

sagetrac-git mannequin commented Oct 3, 2014

sagetrac-git mannequin commented Oct 3, 2014

rwst commented Oct 3, 2014

sagetrac-git mannequin commented Oct 3, 2014

sagetrac-git mannequin commented Oct 3, 2014

rwst commented Oct 3, 2014

skirpichev mannequin commented Oct 3, 2014

sagetrac-git mannequin commented Oct 4, 2014

sagetrac-git mannequin commented Oct 4, 2014

vbraun commented Nov 2, 2014

vbraun commented Nov 2, 2014

jdemeyer commented Nov 3, 2014

sagetrac-git mannequin commented Nov 3, 2014

sagetrac-git mannequin commented Nov 3, 2014

vbraun commented Nov 14, 2014