Upgrade cypari2 package #23518

rharron · 2017-07-22T14:45:31Z

Tarball: https://pypi.python.org/packages/1c/ff/a9a0c9087c08d14807214930bed13ccfb6b2198f38c6992d89c63eb5870b/cypari2-1.1.1.tar.gz

CC: @jdemeyer @defeo @videlec @kiwifb

Component: packages: standard

Keywords: cypari2, sd87

Author: Jeroen Demeyer

Branch: 749c534

Reviewer: Frédéric Chapoton

Issue created by migration from https://trac.sagemath.org/ticket/23518

The text was updated successfully, but these errors were encountered:

fchapoton · 2017-08-07T16:41:33Z

comment:1

any release coming soon ?

fchapoton · 2017-08-18T10:54:35Z

comment:3

ping ? this is the current blocking point in my experimental python3 build.

rharron · 2017-08-18T19:39:10Z

comment:5

I don't really know about what kind of automated stuff happens on trac anymore, so let me ask this perhaps naive question: why is this a block for anything? I only created this ticket because of #15829 which itself is still under development (has never even been marked as "needs review").

fchapoton · 2017-08-18T19:41:25Z

comment:6

Sorry, I have used your ticket for my own purposes. I need (for something having to do with making sage works with python3) to have a new release of cypari2. So I would just embark in this ticket, which is also asking for a release of cypari2.

rharron · 2017-08-18T19:43:45Z

comment:7

Aha! I understand now. It may be worth updating the ticket description: it might encourage people to work on this ticket!

jdemeyer · 2017-08-22T14:58:37Z

Author: Jeroen Demeyer

jdemeyer · 2017-08-23T10:22:58Z

Branch: u/jdemeyer/upgrade_cypari2_package

jdemeyer · 2017-08-23T10:33:33Z

Commit: 75cd924

jdemeyer · 2017-08-23T10:33:33Z

comment:14

Something very strange happens while testing the package:

[cypari-1.1.0] Testing cypari2.pari_instance
[cypari-1.1.0] Traceback (most recent call last):
[cypari-1.1.0]   File "tests/rundoctest.py", line 24, in <module>
[cypari-1.1.0]     test = doctest.testmod(mod, optionflags=doctest.ELLIPSIS|doctest.REPORT_NDIFF|doctest.IGNORE_EXCEPTION_DETAIL)
[cypari-1.1.0]   File "/usr/local/src/sage-config/local/lib/python2.7/doctest.py", line 1911, in testmod
[cypari-1.1.0]     runner.run(test)
[cypari-1.1.0]   File "/usr/local/src/sage-config/local/lib/python2.7/doctest.py", line 1454, in run
[cypari-1.1.0]     return self.__run(test, compileflags, out)
[cypari-1.1.0]   File "/usr/local/src/sage-config/local/lib/python2.7/doctest.py", line 1368, in __run
[cypari-1.1.0]     exc_info)
[cypari-1.1.0]   File "/usr/local/src/sage-config/local/lib/python2.7/doctest.py", line 1235, in report_unexpected_exception
[cypari-1.1.0]     'Exception raised:\n' + _indent(_exception_traceback(exc_info)))
[cypari-1.1.0]   File "/usr/local/src/sage-config/local/lib/python2.7/doctest.py", line 1251, in _failure_header
[cypari-1.1.0]     return '\n'.join(out)
[cypari-1.1.0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 16: ordinal not in range(128)

This happens only when testing the Sage package. Any other way of running the testsuite works fine.

I have absolutely no idea why and I hope that somebody else can fix it.

New commits:

`75cd924`	`Upgrade cypari2 to version 1.1.0`

fchapoton · 2017-08-23T11:34:12Z

comment:15

How can I run this testsuite ?

jdemeyer · 2017-08-23T11:45:12Z

comment:16

The failing testsuite is ./sage -i -c cypari

But like I said, the strange thing is that manually running spkg-check works.

fchapoton · 2017-08-23T11:57:29Z

comment:17

I can confirm the failure of ./sage -i -c cypari, just the same as what you got.

How can I run manually "spkg-check" ?

fchapoton · 2017-08-23T12:06:02Z

comment:18

wiht python3-sage, I get something else

[cypari-1.1.0] ****************************************************
[cypari-1.1.0] Uninstalling cypari2-1.0.0:
[cypari-1.1.0]   Successfully uninstalled cypari2-1.0.0
[cypari-1.1.0] Installing package cypari2 using pip3
[cypari-1.1.0] Ignoring indexes: https://pypi.python.org/simple
[cypari-1.1.0] Processing /home/chapoton/sage3/local/var/tmp/sage/build/cypari-1.1.0/src
[cypari-1.1.0]   Running setup.py (path:/tmp/pip-yihktdba-build/setup.py) egg_info for package from file:///home/chapoton/sage3/local/var/tmp/sage/build/cypari-1.1.0/src
[cypari-1.1.0]     Running command python setup.py egg_info
[cypari-1.1.0]     running egg_info
[cypari-1.1.0]     creating pip-egg-info/cypari2.egg-info
[cypari-1.1.0]     writing pip-egg-info/cypari2.egg-info/PKG-INFO
[cypari-1.1.0]     writing dependency_links to pip-egg-info/cypari2.egg-info/dependency_links.txt
[cypari-1.1.0]     writing top-level names to pip-egg-info/cypari2.egg-info/top_level.txt
[cypari-1.1.0]     writing manifest file 'pip-egg-info/cypari2.egg-info/SOURCES.txt'
[cypari-1.1.0]     Generating PARI functions: (!_) (#_) (%) (%#) (+_) (-_) Catalan Col Colrev (DEBUGLEVEL) Euler I List Map MatTraceback (most recent call last):
[cypari-1.1.0]       File "<string>", line 1, in <module>
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/setup.py", line 84, in <module>
[cypari-1.1.0]         cmdclass=dict(build_ext=build_ext, bdist_egg=no_egg)
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/distutils/core.py", line 148, in setup
[cypari-1.1.0]         dist.run_commands()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
[cypari-1.1.0]         self.run_command(cmd)
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/distutils/dist.py", line 974, in run_command
[cypari-1.1.0]         cmd_obj.run()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 279, in run
[cypari-1.1.0]         self.find_sources()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 294, in find_sources
[cypari-1.1.0]         mm.run()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 521, in run
[cypari-1.1.0]         self.add_defaults()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 557, in add_defaults
[cypari-1.1.0]         sdist.add_defaults(self)
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 36, in add_defaults
[cypari-1.1.0]         self._add_defaults_ext()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/site-packages/setuptools/command/py36compat.py", line 119, in _add_defaults_ext
[cypari-1.1.0]         build_ext = self.get_finalized_command('build_ext')
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/distutils/cmd.py", line 299, in get_finalized_command
[cypari-1.1.0]         cmd_obj.ensure_finalized()
[cypari-1.1.0]       File "/home/chapoton/sage3/local/lib/python3.6/distutils/cmd.py", line 107, in ensure_finalized
[cypari-1.1.0]         self.finalize_options()
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/setup.py", line 50, in finalize_options
[cypari-1.1.0]         rebuild()
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/autogen/__init__.py", line 26, in rebuild
[cypari-1.1.0]         G()
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/autogen/generator.py", line 336, in __call__
[cypari-1.1.0]         self.handle_pari_function(**v)
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/autogen/generator.py", line 239, in handle_pari_function
[cypari-1.1.0]         self.gen_file, doc, obsolete)
[cypari-1.1.0]       File "/tmp/pip-yihktdba-build/autogen/generator.py", line 313, in write_method
[cypari-1.1.0]         print(s, file=file)
[cypari-1.1.0]     UnicodeEncodeError: 'ascii' codec can't encode character '\xa0' in position 314: ordinal not in range(128)
[cypari-1.1.0] Cleaning up...
[cypari-1.1.0] Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-yihktdba-build/
[cypari-1.1.0] Exception information:
[cypari-1.1.0] Traceback (most recent call last):
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
[cypari-1.1.0]     status = self.run(options, args)
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/commands/install.py", line 324, in run
[cypari-1.1.0]     requirement_set.prepare_files(finder)
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
[cypari-1.1.0]     ignore_dependencies=self.ignore_dependencies))
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/req/req_set.py", line 634, in _prepare_file
[cypari-1.1.0]     abstract_dist.prep_for_dist()
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
[cypari-1.1.0]     self.req_to_install.run_egg_info()
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/req/req_install.py", line 439, in run_egg_info
[cypari-1.1.0]     command_desc='python setup.py egg_info')
[cypari-1.1.0]   File "/home/chapoton/sage3/local/lib/python3.6/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
[cypari-1.1.0]     % (command_desc, proc.returncode, cwd))
[cypari-1.1.0] pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-yihktdba-build/
[cypari-1.1.0] Error: installing with pip3 failed
[cypari-1.1.0] 
[cypari-1.1.0] real	0m4.142s
[cypari-1.1.0] user	0m2.116s
[cypari-1.1.0] sys	0m0.224s
[cypari-1.1.0] ************************************************************************
[cypari-1.1.0] Error installing package cypari-1.1.0
[cypari-1.1.0] ************************************************************************

jdemeyer · 2017-08-23T12:22:31Z

comment:19

Can I please have

from __past__ import bytes_literals

fchapoton · 2017-08-23T12:24:09Z

comment:20

Sorry, I do not understand..

EDIT: oh, some kind of joke ?

jdemeyer · 2017-08-23T12:36:56Z

comment:21

Yes, a joke of course.

Let me just say that I'm not a fan of the way how Python deals with unicode encoding issues...

jdemeyer · 2017-08-24T08:04:38Z

comment:22

OK, the issues are due to LC_ALL=C. This is something that will need to be fixed upstream.

embray · 2017-08-24T10:07:03Z

comment:23

I just saw your e-mail about this. Did you figure it out? You mentioned something about LC_ALL=C. And by "fixed upstream" do you mean in cypari? Should I take a look at how cypari is handling encodings?

jdemeyer · 2017-08-24T12:21:28Z

comment:24

Replying to @embray:

I just saw your e-mail about this. Did you figure it out?

Yes. The issue is that Python's open() and print() functions depend on the locale:

$ LC_ALL=en_US.UTF-8 python3 -c 'print(open("/dev/null"))'
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>
$ LC_ALL=C python3 -c 'print(open("/dev/null"))'
<_io.TextIOWrapper name='/dev/null' mode='r' encoding='ANSI_X3.4-1968'>

It turns out that cypari2 was assuming that files were opened in UTF-8 mode. I now made this explicit with open(...., encoding="utf-8") in Python 3.

embray · 2017-08-24T12:32:10Z

comment:25

By the way, this is a useful guide handling encodings when reading text files:

http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#text-file-processing

I would suggest also adding errors='surrogateescape'. This way if any file happens to contain non-ASCII characters that were written using any encoding other than UTF-8, it will at least handle the ASCII parts correctly and not blow up with a UnicodeDecodeError.

sagetrac-git · 2017-08-24T12:59:53Z

Changed commit from 75cd924 to 749c534

sagetrac-git · 2017-08-24T12:59:53Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`749c534`	`Upgrade cypari2 to version 1.1.1`

jdemeyer · 2017-08-24T13:05:27Z

comment:27

Replying to @embray:

I would suggest also adding errors='surrogateescape'. This way if any file happens to contain non-ASCII characters that were written using any encoding other than UTF-8, it will at least handle the ASCII parts correctly and not blow up with a UnicodeDecodeError.

This is not applicable here. In this case, we know that all the files we deal with are UTF-8. The input files come from the PARI documentation which is written in UTF-8. And the output files contain auto-generated Cython code which has a # -*- coding: utf-8 -*- header.

jdemeyer · 2017-08-24T13:07:20Z

comment:28

This version builds and passes its testsuite. I did not yet run Sage doctests.

embray · 2017-08-24T13:08:53Z

comment:29

That's good, that makes it easy then.

jdemeyer · 2017-08-24T13:12:02Z

comment:30

Side comment: the # -*- coding: utf-8 -*- header doesn't actually matter since Cython supports PEP 3120.

fchapoton · 2017-08-24T13:39:43Z

comment:31

not yet smooth for python3:

[cypari-1.1.1]     Running setup.py install for cypari2: finished with status 'done'
[cypari-1.1.1]   Removing source in /tmp/pip-qsn9psry-build
[cypari-1.1.1] Successfully installed cypari2-1.1.1
[cypari-1.1.1] Cleaning up...
[cypari-1.1.1] 
[cypari-1.1.1] real	3m51.177s
[cypari-1.1.1] user	3m42.104s
[cypari-1.1.1] sys	0m5.896s
[cypari-1.1.1] Successfully installed cypari-1.1.1
[cypari-1.1.1] Running the test suite for cypari-1.1.1...
[cypari-1.1.1] make[2]: Entering directory '/home/chapoton/sage3/local/var/tmp/sage/build/cypari-1.1.1/src'
[cypari-1.1.1] python tests/rundoctest.py
[cypari-1.1.1] Traceback (most recent call last):
[cypari-1.1.1]   File "tests/rundoctest.py", line 15, in <module>
[cypari-1.1.1]     cypari2.string_utils.encoding = "utf-8"
[cypari-1.1.1] AttributeError: 'module' object has no attribute 'string_utils'
[cypari-1.1.1] Makefile:16: recipe for target 'check' failed
[cypari-1.1.1] make[2]: *** [check] Error 1
[cypari-1.1.1] make[2]: Leaving directory '/home/chapoton/sage3/local/var/tmp/sage/build/cypari-1.1.1/src'
[cypari-1.1.1] 
[cypari-1.1.1] real	0m0.346s
[cypari-1.1.1] user	0m0.120s
[cypari-1.1.1] sys	0m0.012s
[cypari-1.1.1] ************************************************************************

jdemeyer · 2017-08-24T13:53:23Z

comment:32

Sigh... but Python 3 works fine outside of Sage in Travis CI, running the same commands (pip install . and make check).

fchapoton · 2017-08-24T14:16:09Z

comment:33

Well, I am using "export SAGE_PYTHON3=yes", pull the branch here and "sage -c -f cypari" (in a dedicated sage install for py3 experimental work)

Maybe it is because the python3 general setup is not firmly in place..

jdemeyer · 2017-08-24T15:33:56Z

comment:34

Doctests pass (on Python 2).

I don't understand why this fails with SAGE_PYTHON3=yes, especially given that the error seems to be not specific to Python 3 and given that it works with Python 3 outside of Sage. Maybe there is simply a problem with the particular setup of chapoton.

In any case, I suggest that this should not be a blocker for merging this ticket. It doesn't make the Python 3 situation any worse than before and it helps people who need to update cypari2 for other reasons.

fchapoton · 2017-08-24T15:35:50Z

comment:35

ok, ok. But still, if somebody could try to reproduce that error that I find..

EDIT: I do not have a particular setup, I think.

embray · 2017-08-24T16:17:49Z

comment:36

I'll have a look at the Python 3 issue. I agree with Jeroen it needn't hold up this issue but I can try to investigate ASAP (I need to do a SAGE_PYTHON3=yes build anyways to help with other issues on that).

fchapoton · 2017-08-29T12:09:36Z

comment:37

ok, let this go in. We will see later what to do for tests to pass in python3.

fchapoton · 2017-08-29T12:09:36Z

Reviewer: Frédéric Chapoton

vbraun · 2017-09-02T09:34:04Z

Changed branch from u/jdemeyer/upgrade_cypari2_package to 749c534

jdemeyer · 2017-09-05T13:54:57Z

comment:39

Any news from the PYTHON3 front?

jdemeyer · 2017-09-05T13:54:57Z

Changed commit from 749c534 to none

fchapoton · 2017-09-05T13:59:31Z

comment:40

Not an answer on the cypari2 relation to python3..

Well, this ticket fixes the issue that I had. One of the first blocking point for PYTHON3 is now the division problem in #22875. If you feel like handling this one, it would be great, because I do not understand exactly what you ask for there.

And #23785 would be helpful too.

rharron mannequin added this to the sage-8.1 milestone Jul 22, 2017

rharron mannequin added c: packages: standard labels Jul 22, 2017

fchapoton added the s: needs info label Aug 18, 2017

This comment has been minimized.

Sign in to view

videlec changed the title ~~Update cypari2 with version including idealmoddivisor~~ Updgrade cypari2 package Aug 19, 2017

This comment has been minimized.

Sign in to view

jdemeyer changed the title ~~Updgrade cypari2 package~~ Upgrade cypari2 package Aug 22, 2017

This comment has been minimized.

Sign in to view

jdemeyer added s: needs work and removed s: needs info labels Aug 23, 2017

This comment has been minimized.

Sign in to view

jdemeyer added s: needs review and removed s: needs work labels Aug 24, 2017

fchapoton added s: positive review and removed s: needs review labels Aug 29, 2017

vbraun removed the s: positive review label Sep 2, 2017

vbraun closed this as completed in dd8e806 Sep 2, 2017

Upgrade cypari2 package #23518

Upgrade cypari2 package #23518

Comments

rharron mannequin commented Jul 22, 2017

fchapoton commented Aug 7, 2017

fchapoton commented Aug 18, 2017

rharron mannequin commented Aug 18, 2017

fchapoton commented Aug 18, 2017

rharron mannequin commented Aug 18, 2017

This comment has been minimized.

This comment has been minimized.

jdemeyer commented Aug 22, 2017

This comment has been minimized.

jdemeyer commented Aug 23, 2017

jdemeyer commented Aug 23, 2017

jdemeyer commented Aug 23, 2017

fchapoton commented Aug 23, 2017

jdemeyer commented Aug 23, 2017

fchapoton commented Aug 23, 2017

fchapoton commented Aug 23, 2017

jdemeyer commented Aug 23, 2017

fchapoton commented Aug 23, 2017

jdemeyer commented Aug 23, 2017

jdemeyer commented Aug 24, 2017

embray commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

This comment has been minimized.

embray commented Aug 24, 2017

sagetrac-git mannequin commented Aug 24, 2017

sagetrac-git mannequin commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

embray commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

fchapoton commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

fchapoton commented Aug 24, 2017

jdemeyer commented Aug 24, 2017

fchapoton commented Aug 24, 2017

embray commented Aug 24, 2017

fchapoton commented Aug 29, 2017

fchapoton commented Aug 29, 2017

vbraun commented Sep 2, 2017

jdemeyer commented Sep 5, 2017

jdemeyer commented Sep 5, 2017

fchapoton commented Sep 5, 2017