New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Doctest failures caused by non-working sympow on 32-bit Solaris x86 and 32-bit OpenSolaris #9703
Comments
comment:1
The same issue is observed on a 32-bit build on OpenSolaris. |
This comment has been minimized.
This comment has been minimized.
comment:3
By generating the data files directly, with $ ./sage -sh
> cd SAGE_LOCAL/lib/sympow
> ./new_data sh gp '-sp 2' For the example $ ./sage -sh
> cd SAGE_LOCAL/lib/sympow
> ./new_data sh gp '-sp 1 -dv 0'
> ./new_data sh gp '-sp 1 -dv 1'
> ./new_data sh gp '-sp 1 -dv 2' It seems that part of the Sage interface to SYMPOW is broken. But I don't know if missing data files caused the failures reported here and at #9166. |
comment:4
Replying to @qed777:
Or maybe |
comment:5
On the other hand, there is this (after running "sage -sh" and "cd $SAGE_LOCAL/lib/sympow"):
On my mac or on sage.math, this produces valid output, including the line
which is what is parsed by the (Furthermore, regardless of the C code, I think that sympow.py would never be accepted into Sage today: too many untested methods.) |
comment:6
I wasn't clear enough above: the line |
comment:7
Replying to @qed777:
Try that on the global installation of sage on sage.math - not your own private one. I'm told part of the problem is that SYMPOW is trying to write data files below its installation directory. So it fails on a global installation of Sage, unless the user the user has root access. But there's very little error checking, so the fact the data files are not created is not reported.
If it was only that!
To the best of my knowledge, not all examples need data files. When they do, a message is printed telling one how to create the data files. So that is not all the reason. That is certainly applied by the README file in the source code. I've run the It's not just the C code that is bad. There's a My preference would be to remove this, or at least move it to experimental, but William seems keen to keep it as a standard package. It seems a bit strange to me given.
That seems an alful lot of problems I'm cc'ing a few people that I know have looked at SYMPOW at some point in the past. They might have some comments. Perhaps William has some comments to he would like to add to the ticket. Dave |
comment:8
Replying to @sagetrac-drkirkby:
I'll comment. I am the one who told that to David. I have shamelessly sat on this info for about 3 years. I noticed it when I first tried to package sympow for Gentoo as part of my sage porting drive. At the time I had to allow the directory where the sympow scripts are installed to be world writable.... That no one really noticed means that no one using a global sage install has been using sympow for their work. The only people using it must have used a private installation. We now have a fix for this problem in sage-on-gentoo, Christopher, the other sage-on-gentoo dev, has figured a way of getting sympow to write in $HOME/.sympow in the last month or so. I think there is a case for trying to replace sympow. It seems like the original author has abandoned it - the sympow homepage has gone. Sage could become sympow That's my opinion. Francois |
comment:9
Replying to @kiwifb:
I prepared a small patch which should fix the directory/permission problems. I did not fully test it because I got little time but you should be able see its basic idea - it includes the sed fixes from our sage-on-gentoo overlay: http://github.com/cschwan/sage-on-gentoo/blob/master/sci-mathematics/sympow/sympow-1.018-r1.ebuild (look at it for more detailed comments on what the patch does). To successfully use it, apply it to sympow's source and do the following (would go into spkg-install):
With the patch you also should be able to create the precomputed data which is needed for some optional tests. Christopher |
This comment has been minimized.
This comment has been minimized.
Upstream: Reported upstream. Little or no feedback. |
Author: David Kirkby |
comment:11
I now have a better understanding of why SYMPOW is failing to work correctly on Cygwin and Solaris on x86 chips. SYMPOW makes use of the Quad Double code which relies on IEE-754 double-precision maths, with a 53-bit mantissa and 64-bits in total. However, the default mode of the x86 CPUs is to use extended precision where 64-bits are used for the mantissa and 80 bits in total. It is necessary to change the mode of operation of the floating point processor in order that the Quad double code will work at all. SYMPOW had code for this in a file fpu.c, but it only worked on Linux. It made use of Linux-specific header files too. The main change necessary was to add some inline assembly code to alter the values of bits 8 and 9 in the 16-bit control register of the floating point processor. I got the relevant information about the exact bits needed from an old (1987) copy of the Intel 80387 Programmer's Reference Manual. Here's a revised package which builds and passes all relevant tests on
It was not possible to test on Solaris 10 x86, as access to the Skynet machines is currently unavailable, but I do not believe ll present any problems, since it builds fine on OpenSolaris as a 32-bit application. http://boxen.math.washington.edu/home/kirkby/patches/sympow-1.018.1.p8.spkg Apart from a couple of minor changes, it is the same code which Mike Hansen tested on Cygwin and said it works there too. I've not implemented Christopher's changes which are necessary to get this to pass all the optional tests. Someone who is more keen than me can open a ticket for that if they wish. I just want to see the back of this rather annoying package. A note to reviewers (mainly aimed at Leif!) I know there are 1001 things wrong with this package, but I have no desire to clean them all up. Please review on the basis of the actual changes I made - not what it might be desirable to make. Otherwise it will never get reviewed. Test results a Sun Ultra 27 running OpenSolaris 06/2009 (Originally SYMPOW failed all tests on this workstation)
Test results on Ubunta Linux (sage.math)
== Test results on OS X (bsd.math) ==
== Test results on a Sun T5240 SPARC running Solaris 10 update 7 (t2.math) ==
Dave |
Changes the FPU to double-precession IEEE-754 from Intel's expended precisiion. |
comment:12
Attachment: 9703-SYMPOW-fix-for-Solaris.patch.gz If #9722 is ready to merge soon and this ticket is also positively reviewed, I'll consider including this in a 4.5.3.alpha2. |
comment:14
Replying to @qed777:
Thank you. That would be great. That should give all doc tests passing on Solaris 10 x86, OpenSolaris and Solaris 10 SPARC. I just need someone to change it to positive. I think I've provided quite a bit of evidence that it passes the tests on every platform I can test on. |
Attachment: testfpu.c.gz Test whether FPU is 53-bit or more |
comment:15
Maybe the attached program might be useful. It runs a simple computation to check whether or not the FPU is 53-bit or more. For example, on a x86_64 Intel machine, I get:
You could use to check
|
comment:16
Sage 4.6.prealpha3, which includes this ticket, passed all tests ( |
comment:17
Sage 4.6.prealpha3, which includes this ticket, passed all tests (ptestlong) on a Sun UltraSPARC with Solaris 10. |
comment:18
Replying to @jdemeyer:
It is an interesting little program. It does show the behavior someone (probably you) suspected, that in 64-bit builds, the behaviour changes.
I don't personally feel there is a need to do this.
I think if the result is incorrect (i.e. for some reason the FPU remained in the wrong precision), the doctests will pick this up very quickly. This was the reason the program built, but failed tests on Cygwin and Solairs x86. Dave |
comment:19
Replying to @nexttime:
Thank you for the Fedora confirmation. I had already tested on Ubunta, but not Fedora. |
comment:20
Replying to @jdemeyer:
Note the SPARC that Jeroen used (a Sun Blade 1000 belonging to me), was not the same as as I used, which was the Sun T5240 (t2.,math). So that's confirmation on two different Sun systems. One running the 2005 release of Solaris 10, the other running a much more recent release. Dave |
comment:21
Replying to @sagetrac-drkirkby:
It doesn't. I just added those options to show how the behaviour changes. My idea would be to run testfpu.c without any options and check the result.
I believe that my program is a cleaner solution. For example, are you sure you don't need the patch on Darwin? Also, the following code looks very suspicious:
|
comment:22
By the way, my comment should not prevent this patch from being included in 4.5.3 if that was the plan. If this patch works, I'm certainly happy with that. I just wanted to indicate that there might be better ways to solve the issue. |
comment:23
Replying to @jdemeyer:
The:
was there before - I only added The fpu patch has never been applied on OS X - note the original code specifically checked the platform was Linux. But since the C code used Linux-specific header files, I could not simply apply it everywhere. Given SYMPOW is going to be withdrawn from Sage since it's poorly written and unmaintained, I think this is enough. You may notice today that Bill said on I really want to avoid adding things to this code, since no matter what I do, it is still going to be very poor. This solves the only remaining Solaris x86 problem and is one less problem for the Cygwin port. Dave |
Reviewer: Jeroen Demeyer, Mitesh Patel |
comment:24
I'm testing http://boxen.math.washington.edu/home/kirkby/patches/sympow-1.018.1.p8.spkg on bsd, redhawk, sage, and t2.math. The patch looks good to me, except that I'm not qualified to review the new Jeroen, can you comment on |
comment:26
Can we close #9166 if/when this ticket is merged? |
comment:27
Replying to @qed777:
It should be possible, but you should wait for confirmation from Mike Hansen. Mike confirmed the earlier version I wrote had worked on Cygwin and passed all tests. However, I had did make a small change since then. The change was only the way I wrote the numbers in fpu.c. Initially I wrote them as binary using So the only changes from the patch that Mike said worked were
Dave |
comment:28
Oops, I should have previewed that once more.
|
comment:29
Replying to @qed777:
I've asked on sage-devel if someone can comment on it. I suspect Jeroen can. Here's the and here's the changed version One obvious difference is the original code just writes a number to the control word, but I first read the control word, set two bits, then write the control word. The reason I did that was that one of the bits in that control word is reserved by Intel, so I did not think it wise to write a number to the register directly. Having carefully read the documentation that comes with the later version of quad-double, the only changes that should be made to that code is the precision control bits, which is all I change. Dave |
comment:30
I've read and understood the new and old versions of fpu.c, and I confirm that the new version is equivalent to the old version under Linux but is more portable in general. So positive review for fpu.c . (I'll go ahead and mark as positive review, relying on mpatel's review for everything except fpu.c .) |
comment:31
Thanks, Carl! Replying to @qed777:
The long doctests pass on these machines. |
Changed reviewer from Jeroen Demeyer, Mitesh Patel to Jeroen Demeyer, Mitesh Patel, Carl Witty |
Merged: sage-4.5.3.alpha2 |
John Palmieri has built most of Sage on the host fulvia, but there are a number of tests related to sympow that are failing. The summary at the end shows:
of these test failures, the following three are attributed to SYMPOW
Looking at the source code, it is not valid C, so it's quite possible the code gets mis-compiled. In fact, IMHO, gcc should reject the code - just as the Sun compiler does.
I'll try to work out what was intended and see if the code can be re-written in a way that compiles with the Sun compiler, in which case gcc should have more chance of generating correct code.
Upstream: Reported upstream. Little or no feedback.
CC: @jhpalmieri @jaapspies @nexttime @peterjeremy @kiwifb @williamstein @JohnCremona @mwhansen
Component: build
Author: David Kirkby
Reviewer: Jeroen Demeyer, Mitesh Patel, Carl Witty
Merged: sage-4.5.3.alpha2
Issue created by migration from https://trac.sagemath.org/ticket/9703
The text was updated successfully, but these errors were encountered: