Fix doctests that want write access to $SAGE_ROOT

[sagetrac-mabshoff]

All doctests in Sage should pass when they are run as a user that does not have write access to the Sage install. To do that, run the doctests on a Sage install that isn't owned by the user:

$ ./sage -tp 4 --long devel/sage/doc/common devel/sage/doc/en devel/sage/sage

On sage-5.4.beta1, this causes:

The following tests failed:

        sage -t --long devel/sage/sage/interfaces/qepcad.py # 3 doctests failed
        sage -t --long devel/sage/sage/misc/inline_fortran.py # 3 doctests failed
        sage -t --long devel/sage/sage/tests/cmdline.py # 2 doctests failed

Apply:

1. attachment: 5155_sage_location.patch to the scripts repository.
2. attachment: 5155_root.patch to the SAGE_ROOT repository.
3. attachment: 5155_sagelib.patch to the Sage library.

Depends on #13157
Depends on #13397
Depends on #13452
Depends on #13407
Depends on #13459
Depends on #13457
Depends on #13123
Depends on #13887

CC: @lftabera @nexttime

Component: scripts

Author: Mike Hansen, Jeroen Demeyer, John Palmieri

Reviewer: François Bissey

Merged: sage-5.7.beta0

Issue created by migration from https://trac.sagemath.org/ticket/5155

[mwhansen]

Author: Mike Hansen

[jhpalmieri]

comment:2

Before applying the patch, I get two doctest failures (with a non-writeable Sage install). Applying the patch "trac_5155.patch" fixes both. So positive review for that.

The scripts patch wasn't necessary to get those tests to pass, and it also doesn't apply cleanly to 4.3.1:

applying /Users/palmieri/Downloads/scripts_5155.patch
patching file sage-doctest
Hunk #1 FAILED at 55
1 out of 1 hunks FAILED -- saving rejects to file sage-doctest.rej
unable to find 'sage-dsage-trial' for patching
1 out of 1 hunks FAILED -- saving rejects to file sage-dsage-trial.rej
patching file sage-maketest
Hunk #1 succeeded at 1 with fuzz 1 (offset -1 lines).
patching file sage-test
Hunk #1 FAILED at 40
1 out of 1 hunks FAILED -- saving rejects to file sage-test.rej
sage-dsage-trial: No such file or directory
abort: patch failed to apply

The changes there look fine in principle, though. So once it's rebased, positive review there, too.

[lftabera]

comment:3

Some of the doctest are ok nowadays but there are other new fails. See for example #9965

I can confirm that, with sage 4.5.3 I get the following doctest failures:

    sage -t  -long "devel/sage/doc/common/builder.py"
    sage -t  -long "devel/sage/sage/interfaces/qepcad.py"
    sage -t  -long "devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py"
    sage -t  -long "devel/sage/sage/modular/hecke/submodule.py"
    sage -t  -long "devel/sage/sage/modular/abvar/abvar.py"
    sage -t  -long "devel/sage/sage/lfunctions/sympow.py"

All except quecad are trying to be solved in #9965

[lftabera]

comment:4

qepcad failures

sage -t -long "devel/sage/sage/interfaces/qepcad.py"        
**********************************************************************
File "/opt/SAGE/sage/devel/sage/sage/interfaces/qepcad.py", line 638:
    sage: _rewrite_qepcadrc()
Exception raised:
    Traceback (most recent call last):
      File "/opt/SAGE/sage/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/opt/SAGE/sage/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/opt/SAGE/sage/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_3[3]>", line 1, in <module>
        _rewrite_qepcadrc()###line 638:
    sage: _rewrite_qepcadrc()
      File "/opt/SAGE/sage/local/lib/python/site-packages/sage/interfaces/qepcad.py", line 660, in _rewrite_qepcadrc
        open(fn, 'w').write(text)
    IOError: [Errno 13] Permission denied: '/opt/SAGE/sage/local//default.qepcadrc'
**********************************************************************
File "/opt/SAGE/sage/devel/sage/sage/interfaces/qepcad.py", line 689:
    sage: Qepcad_expect(memcells=100000, logfile=sys.stdout)
Exception raised:
    Traceback (most recent call last):
      File "/opt/SAGE/sage/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/opt/SAGE/sage/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/opt/SAGE/sage/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_6[3]>", line 1, in <module>
        Qepcad_expect(memcells=Integer(100000), logfile=sys.stdout)###line 689:
    sage: Qepcad_expect(memcells=100000, logfile=sys.stdout)
      File "/opt/SAGE/sage/local/lib/python/site-packages/sage/interfaces/qepcad.py", line 692, in __init__
        _rewrite_qepcadrc()
      File "/opt/SAGE/sage/local/lib/python/site-packages/sage/interfaces/qepcad.py", line 660, in _rewrite_qepcadrc
        open(fn, 'w').write(text)
    IOError: [Errno 13] Permission denied: '/opt/SAGE/sage/local//default.qepcadrc'
**********************************************************************
2 items had failures:
   1 of   6 in __main__.example_3
   1 of   4 in __main__.example_6
***Test Failed*** 2 failures.
For whitespace errors, see the file /home/usuario/.sage//tmp/.doctest_qepcad.py
         [1.9 s]

[lftabera]

comment:5

Another issue, if you compile sage but the very first access of sage is made by a user without write permissions, you get the following error:

---

| Sage Version 4.5.3, Release Date: 2010-09-04 |
| Type notebook() for the GUI, and license() for information. |

---

Traceback (most recent call last):
File "/opt/SAGE/sage-4.5.3/local/bin/sage-location", line 174, in
t, R = install_moved()
File "/opt/SAGE/sage-4.5.3/local/bin/sage-location", line 18, in install_moved
write_flags_file()
File "/opt/SAGE/sage-4.5.3/local/bin/sage-location", line 82, in write_flags_file
open(flags_file,'w').write(get_flags_info())
IOError: [Errno 13] Permission denied: '/opt/SAGE/sage-4.5.3/local/lib/sage-flags.txt'

[jhpalmieri]

comment:6

With Sage 4.7.2.alpha2, I see problems with qepcad and sympow. I think the qepcad problem should be easy to solve, basically as mhansen did before:

diff --git a/sage/interfaces/qepcad.py b/sage/interfaces/qepcad.py
--- a/sage/interfaces/qepcad.py
+++ b/sage/interfaces/qepcad.py
@@ -636,14 +636,14 @@ def _rewrite_qepcadrc():
     EXAMPLES:
         sage: from sage.interfaces.qepcad import _rewrite_qepcadrc
         sage: _rewrite_qepcadrc()
-        sage: from sage.misc.misc import SAGE_LOCAL
-        sage: open('%s/default.qepcadrc'%SAGE_LOCAL).readlines()[-1]
+        sage: from sage.misc.misc import DOT_SAGE
+        sage: open('%s/default.qepcadrc'%DOT_SAGE).readlines()[-1]
         'SINGULAR .../local//bin'
     """
     global _rewrote_qepcadrc
     if _rewrote_qepcadrc: return
 
-    SL = sage.misc.misc.SAGE_LOCAL
+    SL = sage.misc.misc.DOT_SAGE
     fn = '%s/default.qepcadrc'%SL
     text = \
 """# THIS FILE IS AUTOMATICALLY GENERATED -- DO NOT EDIT

Sympow will be harder to deal with, because of how the spkg is written: it tries to write files to SAGE_LOCAL/lib/sympow. See [#9703 comment:9] for a possible fix.

I'm attaching a patch to try to deal with the situation when you run Sage for the very first time as a user without write permissions.

Finally, there may be other issues if you compile Sage but don't run it, and then run doctests as a user without write permissions (the first time doctests get run, they might write some files which don't need to be written later). These issues should be fixed, too.

[jdemeyer]

comment:7

I'm not sure this patch is a good idea or needed, considering #11926. If the Sage install was moved, it is probably good to bail out with an error if there is no write access.

[jdemeyer]

Dependencies: #11926

[jhpalmieri]

comment:14

A few quick comments:

• you have a typo in line 93, "eyactly"
• on line 583, # optional - qepcad, note "algeraic" [sic], this is not the way to format an optional doctest: it will only run if you do "sage -t -optional-only=qepcad,not,algeraic,sic" or something like that.
• same on line 1220

For the scripts patch:

• why not apply os.path.abspath to SAGE_ROOT?

[jdemeyer]

Attachment: 5155_root.patch.gz

Attachment: 5155_sage_location.patch.gz

[jdemeyer]

comment:39

The Fortran patch needed to be rebased because of #13579. Apart from that, things look fine.

[jdemeyer]

comment:40

Some doctest failures...

[jdemeyer]

comment:41

Attachment: 5155_sagelib.patch.gz

[jdemeyer]

Changed dependencies from #13157, #13397, #13452, #13407, #13459, #13457, #13123 to #13157, #13397, #13452, #13407, #13459, #13457, #13123, #13887

[kiwifb]

comment:43

SAGE_LOCAL is a strange location for default.qepcadrc I suppose it really needs to be changed in the spkg first but SAGE_LOCAL/etc is a better location for this kind of file.

[kiwifb]

Reviewer: François Bissey

[kiwifb]

comment:44

My only objection is the location of default.qepcadrc but since it is for an experimental spkg and would require another ticket to change it I am giving this a positive review. Moving the default.qepcadrc location should be the object of a follow up ticket.

[jdemeyer]

comment:45

Replying to @kiwifb:

My only objection is the location of default.qepcadrc

Also, you shouldn't blame this ticket for that, it has always been like that.

Thanks for the review!

[jdemeyer]

Merged: sage-5.7.beta0

[jdemeyer]

comment:47

I made a patch for the analogous problem for $HOME at #13985. It would be nice if somebody could review that.