matrix() command should accept some Matlab-style inputs

[dandrake]

In https://groups.google.com/d/msg/sage-devel/p-nrpKUBMm8/LUMAfoXsz-UJ Jason Grout mentioned that it would be convenient if Sage accepted the following Matlab syntax for creating matrices:

sage: matrix("1 2 3; 4 5 6; 7 8 9")
[1 2 3]
[4 5 6]
[7 8 9]

This should be an easy change to the matrix() command, and would be very convenient for users, even those with no Matlab background.

Depends on #24742

CC: @kini @kcrisman @mforets

Component: linear algebra

Author: Dan Drake

Branch/Commit: u/ddrake/string-matrix-11699 @ 5d006ef

Issue created by migration from https://trac.sagemath.org/ticket/11699

[jasongrout]

comment:1

see also http://www.mathworks.com/help/techdoc/math/f1-84864.html#f1-84906

[jasongrout]

comment:2

In particular, elements can be separated by either commas or spaces (or both), and rows are separated by semicolons. So we should be able to easily separate by splitting the string:

rows = a.split(';')
for r in rows:
    elts = r.replace(',', ' ').split()
    # do something to interpret the elts, like map(sage_eval, elts)

[jasongrout]

comment:3

Or probably even faster, do the ','->' ' translation once at the beginning to avoid lots of small strings being created:

# if a was the string passed in as the list of elements
a=a.replace(',', ' ')
rows = a.split(';')
elements=[map(sage_eval, r.split()) for r in rows]
# now continue matrix processing with "elements" as the list of elements

This would just be inserted in the first time the elements of a matrix were accessed and would convert the string to a list of elements like it expects.

[jasongrout]

comment:4

In fact, we should support multiple lines here too, like Matlab, so you could do:

matrix("""
1 2 3
4 5 6
7 8 9
""")

To do that, do

# if a was the string passed in as the list of elements
a=a.replace(',', ' ').replace(';','\n')
rows = a.split('\n')
# throw away empty rows
rows = [r for r in rows if not r.trim()]
elements=[map(sage_eval, r.split()) for r in rows]
# now continue matrix processing with "elements" as the list of elements

At this point, it might be better to use regular expressions. It's certainly worth a timing test. Note the following matlab examples:

>> a=[1 2 3
4 5 6]

a =

     1     2     3
     4     5     6

>> a=[
1 2 3
4 5 6
]

a =

     1     2     3
     4     5     6

>> a=[  
1 2 3;
4 4 4
]

a =

     1     2     3
     4     4     4

>> a = [
1 2 3; ;
4 5 6]

a =

     1     2     3
     4     5     6

>> a = [;;1 2;;3 4] 

a =

     1     2
     3     4

>> a=[

;
1 2 3]

a =

     1     2     3

[jasongrout]

comment:5

Oops, that should r.strip() rather than r.trim() above.

[jasongrout]

comment:6

Slightly faster than the double replace, if s is defined at module creation time:

import string
s=string.maketrans(';,','\n ')

Then the double replace above becomes: a.translate(s)

[jasongrout]

comment:7

Okay, here's a new iteration, especially since I flipped logic above with the r.strip()

# if a was the string passed in as the list of elements
a=a.replace(',', ' ').replace(';','\n') # or a.translate(s) if we can initialize s
rows = a.split('\n')
# throw away empty rows
rows = [r for r in rows if r.strip()]
elements=[map(sage_eval, r.split()) for r in rows]
# now continue matrix processing with "elements" as the list of elements

[dandrake]

comment:8

Okay, try this. I think it still needs some optimization with the string handling stuff, and it could probably use some more doctests, but it's basically ready for review.

[dandrake]

Author: Dan Drake

[dandrake]

comment:9

Hrm, Robert Bradshaw just tossed some cold water on my simple string-handling: https://groups.google.com/d/msg/sage-devel/p-nrpKUBMm8/_4DL5li7NcQJ

The quick version: what about 1 + sin(x) (for splitting in spaces) and [1, 2; pi, f(3, 4)] for splitting on commas? Hrm.

[jasongrout]

comment:10

quick comment: you should probably use isinstance(args[0], basestring), as that is forward-compatible with unicode strings and python3.

[dandrake]

comment:11

Another problem: sage_eval evaluates its arguments in a different context than the one in which matrix() gets called. So even though you may have done x = var('x'), it doesn't know about x.

If we do put this in, I think the docstring should be explicit that using strings is limited, and only really works for number literals and similar things. Then later someone can write a full parser and figure out the evaluation context stuff.

[jasongrout]

comment:13

Maybe on another ticket, but can I ask for one more enhancement on this? Can we make the following work:

matrix("""
[1 2]
[3 4]
""")

(i.e., any "]" or "[" next to row delimiters are stripped). That would mean that I could cut and paste output from Sage back into Sage to make a matrix.

[dandrake]

comment:14

Replying to @jasongrout:

Maybe on another ticket, but can I ask for one more enhancement on this? Can we make the following work:

matrix("""
[1 2]
[3 4]
""")

(i.e., any "]" or "[" next to row delimiters are stripped). That would mean that I could cut and paste output from Sage back into Sage to make a matrix.

Sure, no problem. Although you'll still have the split-on-whitespace and context limitations. But plain numerical matrices are common enough so that I think it's worth it.

[dandrake]

apply to main repo

[jasongrout]

comment:15

Attachment: trac_11699_matlab_input_to_matrix.patch.gz

See #12354 for another take on this as preparser syntax.

[loefflerd]

comment:17

This patch causes a doctest to fail in sageinspect.py (see patchbot logs) -- looks like somebody helpfully wrote a doctest which relies on the docstring of "matrix()" being exactly 34 lines long. (There is also another failure on the most recent run, but I suspect that is just a random glitch.)

[jasongrout]

comment:18

argh, I was trying to use this today, thinking it was already in...

[dandrake]

Commit: ddc2e81

[dandrake]

Branch: u/ddrake/string-matrix-11699

[dandrake]

Changed work issues from failing doctest to failing doc build

[dandrake]

comment:24

I've ported this over to 6.9.beta3. It builds, works, and passes doctests, but the documentation doesn't build -- I have doctests with \n in them, and these are not interpreted by Sphinx correctly, and then it complains about indentation being messed up. Advice for how to fix that welcome.

---

New commits:

ddc2e81

add Matlab-style strings for matrix() constructor

[jdemeyer]

comment:25

Replying to @dandrake:

Advice for how to fix that welcome.

Use

r"""
docstring
"""

instead of

"""
docstring
"""

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5d006ef

raw string for docstrings so Sphinx handles \n -- thanks Jeroen Demeyer

[sagetrac-git]

Changed commit from ddc2e81 to 5d006ef

[dandrake]

comment:27

Replying to @jdemeyer:

Use

r"""
docstring
"""

instead of

"""
docstring
"""

That's it! Thank you.

[dandrake]

Changed work issues from failing doc build to none

[videlec]

comment:29

Failing doctest

sage -t --long src/sage/misc/sageinspect.py
**********************************************************************
File "src/sage/misc/sageinspect.py", line 1959, in sage.misc.sageinspect.sage_getsourcelines
Failed example:
    sage_getsourcelines(matrix)[1]
Expected:
    732
Got:
    867
**********************************************************************
1 item had failures:
   1 of  28 in sage.misc.sageinspect.sage_getsourcelines
    [284 tests, 1 failure, 28.10 s]
----------------------------------------------------------------------
sage -t --long src/sage/misc/sageinspect.py  # 1 doctest failed
----------------------------------------------------------------------

[jdemeyer]

Dependencies: #24742

[jdemeyer]

comment:32

I don't plan to work on this, but if anybody does: it should be on top of #24742.