Skip to content

Autorat #1470

Merged
merged 7 commits into from Aug 10, 2012

5 participants

@asmeurer
SymPy member
asmeurer commented Aug 7, 2012

This fixes http://code.google.com/p/sympy/issues/detail?id=3182. There is now a -i option to isympy, which allows automatic wrapping of integer literals with Integer. The result is that things like 1/2 will be converted to Rational(1, 2) automatically. For example:

$./bin/isympy  -i
IPython console for SymPy 0.7.1-git (Python 2.7.3-64-bit) (ground types: python)

These commands were executed:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z, t = symbols('x y z t')
>>> k, m, n = symbols('k m n', integer=True)
>>> f, g, h = symbols('f g h', cls=Function)

Documentation can be found at http://www.sympy.org

In [1]: 1/2
Out[1]: 1/2

In [2]: x**(1/2)
Out[2]: 
  ___
╲╱ x 

To get a Python int, you can wrap the literal with int(). This works by preprocessing the code with tokenize and wrapping all non-float number literals with Integer. There was some discussion about the best way to make IPython do this (see the commit message and the discussions linked to there).

Still todo: tests.

asmeurer added some commits Aug 7, 2012
@asmeurer asmeurer Add -i, --int-to-Integer option to isympy
This makes isympy automatically wrap integer literals with Integer.  The
result is that things like 1/2 are computed as Rational(1, 2) instead of 0.5.
For example:

$./bin/isympy -i
IPython console for SymPy 0.7.1-git (Python 2.7.3-64-bit) (ground types:
python)

These commands were executed:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z, t = symbols('x y z t')
>>> k, m, n = symbols('k m n', integer=True)
>>> f, g, h = symbols('f g h', cls=Function)

Documentation can be found at http://www.sympy.org

In [1]: 1/2
Out[1]: 1/2

In [2]: x**(1/2)
Out[2]:
    ___
  ╲╱ x

You can still get a int literal by wrapping the integer with int()

In [3]: int(1)/int(2)
Out[3]: 0.5

Note that this requires IPython.

Also added -I, --interactive, which combines this option with -a,
--auto-symbols (which has been renamed from --auto).  Additional such
interactive modifications should be included in this command line argument.

Note that this had to be done by monkey-patching IPython's run_cell. IPython's
current API for preprocessing lines is not expressive enough to handle
this--in particular, it does not allow modifying multiple line inputs.  See
http://mail.scipy.org/pipermail/ipython-user/2012-August/010846.html and
ipython/ipython#1491 for more information.

Thanks to http://stackoverflow.com/a/11735604/161801 for suggesting the
tokenize method of doing this.

This fixes issue 3182.
a4ebdab
@asmeurer asmeurer Use ast instead of compiler, which was removed in Python 3 a8eb22c
@asmeurer asmeurer Add some more tests for isympy -a 116e910
@asmeurer asmeurer Add some tests for isympy -i 42fdfe2
@asmeurer
SymPy member
asmeurer commented Aug 7, 2012

OK, some tests added.

@travisbot

This pull request fails (merged a8eb22c into 3c2c3c7).

@asmeurer asmeurer Remove a binary literal from a doctest
They are not supported in Python 2.5
f90ea7e
@travisbot

This pull request fails (merged 42fdfe2 into 3c2c3c7).

@asmeurer
SymPy member
asmeurer commented Aug 7, 2012

Huh, apparently binary literals were added in Python 2.6. I did not know that.

@travisbot

This pull request passes (merged f90ea7e into 3c2c3c7).

@certik certik commented on the diff Aug 8, 2012
sympy/interactive/session.py
+ from tokenize import generate_tokens, untokenize, NUMBER, NAME, OP
+ from StringIO import StringIO
+
+ result = []
+ g = generate_tokens(StringIO(s).readline) # tokenize the string
+ for toknum, tokval, _, _, _ in g:
+ if toknum == NUMBER and '.' not in tokval: # replace NUMBER tokens
+ result.extend([
+ (NAME, 'Integer'),
+ (OP, '('),
+ (NUMBER, tokval),
+ (OP, ')')
+ ])
+ else:
+ result.append((toknum, tokval))
+ return untokenize(result)
@certik
SymPy member
certik added a note Aug 8, 2012

Looks simple enough and should work in Python 3.x as well:

http://docs.python.org/dev/library/tokenize.html#examples

The only difference in there is the use of BytesIO instead of StringIO.

@asmeurer
SymPy member
asmeurer added a note Aug 8, 2012

I guess 2to3 changes this automatically, because I tested in Python 3 and it works.

@asmeurer
SymPy member
asmeurer added a note Aug 8, 2012

Actually, just checked, and it changes it to from io import StringIO.

@certik
SymPy member
certik added a note Aug 8, 2012

Cool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@certik certik commented on the diff Aug 8, 2012
sympy/interactive/session.py
+ # Check the cell for syntax errors. This way, the syntax error
+ # will show the original input, not the transformed input. The
+ # downside here is that IPython magic like %timeit will not work
+ # with transformed input (but on the other hand, IPython magic
+ # that doesn't expect transformed input will continue to work).
+ ast.parse(cell)
+ except SyntaxError:
+ pass
+ else:
+ cell = int_to_Integer(cell)
+ old_run_cell(cell, *args, **kwargs)
+
+ if hasshell:
+ app.shell.run_cell = my_run_cell
+ else:
+ app.run_cell = my_run_cell
@certik
SymPy member
certik added a note Aug 8, 2012

Again, simple enough to maintain the monkey patch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@certik certik commented on the diff Aug 8, 2012
sympy/interactive/session.py
+ Example
+ =======
+
+ >>> from sympy.interactive.session import int_to_Integer
+ >>> from sympy import Integer
+ >>> s = '1.2 + 1/2 - 0x12 + a1'
+ >>> int_to_Integer(s)
+ '1.2 +Integer (1 )/Integer (2 )-Integer (0x12 )+a1 '
+ >>> s = 'print (1/2)'
+ >>> int_to_Integer(s)
+ 'print (Integer (1 )/Integer (2 ))'
+ >>> exec(s) #doctest: +SKIP
+ 0.5
+ >>> exec(int_to_Integer(s))
+ 1/2
+ """
@certik
SymPy member
certik added a note Aug 8, 2012

This doctest is the only test that we have for this function. Can you also add a regular test for this?

@asmeurer
SymPy member
asmeurer added a note Aug 8, 2012

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@smichr smichr commented on the diff Aug 8, 2012
bin/isympy
@@ -129,6 +129,27 @@ COMMAND LINE OPTIONS
example, if you define a function in isympy with an undefined
Symbol, it will not work.
+ See also the -i and -I options.
+
+-i, --int-to-Integer
+
+ Automatically wrap int literals with Integer. This makes it so that
+ things like 1/2 will come out as Rational(1, 2), rather than 0.5. This
+ works by preprocessing the source and wrapping all int literals with
+ Integer. Note that this will not change the behavior of int literals
+ assigned to variables, and it also won't change the behavior of functions
+ that return int literals.
@smichr
SymPy member
smichr added a note Aug 8, 2012

Should a note be added that this overrides "from future import division"? It seems pretty clear that this is the case, though.

@asmeurer
SymPy member
asmeurer added a note Aug 8, 2012

It doesn't, though. It just wraps integer literals in Integer. int(1)/int(2) will still give 0.5 (instead of 0).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@travisbot

This pull request passes (merged c1aaae1 into 3c2c3c7).

@Krastanov
SymPy member

SymPy Bot Summary: ✳️ All tests have passed.

Test command: setup.py test
master hash: bd49e6a
branch hash: c1aaae1

Interpreter 1: ✳️ All tests have passed.

Interpreter: /usr/local/bin/python2.5 (2.5.6-final-0)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYwrYiDA

Interpreter 2: ✳️ All tests have passed.

Interpreter: /usr/bin/python2.7 (2.7.3-candidate-2)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYnu0iDA

Interpreter 3: ✳️ All tests have passed.

Interpreter: /usr/bin/python3.2 (3.2.3-candidate-2)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYjJQjDA

Build HTML Docs: ✳️ All tests have passed.

Docs build command: make html-errors
Sphinx version: 1.1.3

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYm_AhDA

Automatic review by SymPy Bot.

@asmeurer
SymPy member
asmeurer commented Aug 8, 2012

SymPy Bot Summary: 🔴 There were test failures (merged asmeurer/autorat (c1aaae1) into master (bd49e6a)).

@asmeurer: Please fix the test failures.

Interpreter 1: 🔴 There were test failures.
Interpreter: None (2.5.0-final-0)
Architecture: Darwin (32-bit)

Interpreter 2: 🔴 There were test failures.
Interpreter: None (2.6.6-final-0)
Architecture: Darwin (32-bit)

Interpreter 3: 🔴 There were test failures.
Interpreter: None (2.7.2-final-0)
Architecture: Darwin (32-bit)

Interpreter 4: ✳️ All tests have passed.
Interpreter: /sw/bin//python2.6 (2.6.8-final-0)
Architecture: Darwin (64-bit)

Interpreter 5: ✳️ All tests have passed.
Interpreter: /sw/bin//python2.7 (2.7.3-final-0)
Architecture: Darwin (64-bit)

Interpreter 6: ✳️ All tests have passed.
Interpreter: None (3.2.2-final-0)
Architecture: Darwin (32-bit)

Interpreter 7: 🔴 There were test failures.
Interpreter: None (3.3.0-beta-1)
Architecture: Darwin (32-bit)

Interpreter 8: ✳️ All tests have passed.
Interpreter: /sw/bin//python3.2 (3.2.3-final-0)
Architecture: Darwin (64-bit)

Interpreter 9: 🔴 There were test failures.
Interpreter: None (3.3.0-beta-1)
Architecture: Darwin (64-bit)

Build HTML Docs: 🔴 There were test failures.
Sphinx version: 1.1.3

@certik
SymPy member
certik commented Aug 8, 2012

There is currently a big bug:

In [1]: e = 1e-100

In [2]: e                                                                      
Out[2]: 0

The reason being (by looking into ipython history):


In [1]: e =Integer (1e-100 )

In [2]: e                                                                      
Out[2]: 0

This bit me today...

@certik certik commented on the diff Aug 8, 2012
sympy/interactive/tests/test_interactive.py
@@ -0,0 +1,14 @@
+import sys
+
+from sympy.interactive.session import int_to_Integer
+
+def test_int_to_Integer():
+ assert int_to_Integer("1 + 2.2 + 0x3 + 40") == \
+ 'Integer (1 )+2.2 +Integer (0x3 )+Integer (40 )'
+ if sys.version_info[0] == 2:
+ assert int_to_Integer("1l") == 'Integer (1l )'
+ if sys.version_info[1] > 5 or sys.version_info[0] == 3:
+ # Binary literals were added in Python 2.6
+ assert int_to_Integer("0b101") == 'Integer (0b101 )'
+ assert int_to_Integer("ab1 + 1 + '1 + 2'") == "ab1 +Integer (1 )+'1 + 2'"
+ assert int_to_Integer("(2 + \n3)") == '(Integer (2 )+\nInteger (3 ))'
@certik
SymPy member
certik added a note Aug 8, 2012

It's missing a test for "1e-100", which will fail.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@asmeurer
SymPy member
asmeurer commented Aug 8, 2012

Ahh, good catch. I made the assumption that a floating point literal must contain a ., but this is clearly not true. I guess I also missed complex literals. Can you think of any other cases of a float literal? Or maybe there is something in the standard library that automates this.

@asmeurer
SymPy member
asmeurer commented Aug 8, 2012

Pushed a fix.

@travisbot

This pull request passes (merged 64366c2 into 3c2c3c7).

@asmeurer
SymPy member
asmeurer commented Aug 8, 2012

I guess I can just eval it and check the type. Let me see if that is not too slow.

@asmeurer
SymPy member
asmeurer commented Aug 8, 2012

The way I'm doing it seems to be much faster, so I'll just stick with it for now.

@Krastanov
SymPy member

SymPy Bot Summary: ✳️ All tests have passed.

Test command: setup.py test
master hash: 15c2c89
branch hash: 64366c2

Interpreter 1: ✳️ All tests have passed.

Interpreter: /usr/local/bin/python2.5 (2.5.6-final-0)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYzrYiDA

Interpreter 2: ✳️ All tests have passed.

Interpreter: /usr/bin/python2.7 (2.7.3-candidate-2)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sY7dUiDA

Interpreter 3: ✳️ All tests have passed.

Interpreter: /usr/bin/python3.2 (3.2.3-candidate-2)
Architecture: Linux (64-bit)
Cache: yes

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYrO0iDA

Build HTML Docs: ✳️ All tests have passed.

Docs build command: make html-errors
Sphinx version: 1.1.3

Test results html report: http://reviews.sympy.org/report/agZzeW1weTNyDAsSBFRhc2sYyo8iDA

Automatic review by SymPy Bot.

@smichr

from http://stackoverflow.com/questions/379906/python-parse-string-to-float-or-int

try:
    i = int(num)
except ValueError:
    return False
return True

The same link suggests using ast.literal_eval so you could do return type(ast.literal_eval(num)) is int.

SymPy member

That doesn't work for non-base ten literals. See http://stackoverflow.com/questions/11874087/determine-the-kind-int-float-of-number-a-python-numeric-literal-represents. I left it as it is because it's faster this way (and also ast is not in Python 2.5, but I just realized this won't work in Python 2.5 anyway because IPython is 2.6+).

@certik
SymPy member
certik commented Aug 10, 2012

This looks good enough, so I am merging it. We can improve upon the master if needed.

@certik certik merged commit 08617ed into sympy:master Aug 10, 2012

1 check passed

Details default The Travis build passed
@asmeurer
SymPy member

Thanks. I also added this to the release notes for the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.