port Sage to SPARC Solaris 11

[dimpase]

We have access to SPARC T4 system

SunOS xenos-sagetest 5.11 11.3 sun4v sparc sun4v

We track here the resulting issues and tickets.

• no need to pre-build any tools, apparently. Everything is available via Solaris pkg or pkgutil (i.e. opencsw).

• MPFR configure fails with MPIR. Can be resolved by using GMP instead. Upstream: issue 242. For details and a possible fix, see enable MPIR on Solaris #24674.

• Various packages need CFLAGS="-D_XPG6"

• To take care of the above, set the following before building

export PATH=/opt/csw/gnu:$PATH
export CFLAGS="-O2 -g -D_XPG6"
export CXXFLAGS="$CFLAGS"        # Maybe not strictly needed

./configure --with-mp=gmp

• OpenBLAS fails to auto-detect SPARC CPU: Fix OpenBLAS build on SPARC #24638

• Properly set LD_LIBRARY_PATH in spkg-build for python2/3 on SunOS #24597 fixes building issues for python2/3

• building fplll and singular fails due to a silly C++ (g++ 5.4.0)
  error, which cannot properly compile templated pow() from
  cmath, e.g. the following fails:

#include <cmath>
int main()
{
int i,j,k;
i = 2;
j = 10;
k = pow(i,j);
return k;
}

This appears to be platform-specific. Same error with g++ 7.2.0. Adding std:: to pow, or adding using namespace std; results in successful compilation of this example (and most probably this also will work for Singular).
** this problem went away by itself after some changes to the linker used, I don't really know why **

• iconv fails due to a wrong assumption on the standard in the
  source code, fixed in the latest stable version. upgrade iconv to 1.15 #24602 is the
  update to do here.

• gcc 7.2 builds fplll, whereas gcc 5.4 cannot do it. See fplll fails to build on SunOS #24618

• linking of Singular is tentatively fixed by Singular fails to build on SunOS #24611

Depends on #23733
Depends on #24638
Depends on #24765

CC: @dimpase

Component: porting: Solaris

Branch/Commit: u/jdemeyer/port_sage_to_sparc_solaris_11 @ 760cfbb

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/24596

[dimpase]

comment:5

I've managed to build gcc 7.2.0 from source configuring as follows:

./configure --prefix=/datapool/dima --with-gmp-include=/usr/include/gmp --with-mpfr-include=/usr/include/mpfr --with-mpc=/usr --without-gnu-ld --with-ld=/usr/bin/ld --with-gnu-as --enable-languages=c,c++,fortran  --enable-shared --enable-initfini-array  --with-system-zlib  --with-as=/usr/gnu/bin/as --build=sparcv9-sun-solaris2.11

using system-provided gmp, mpfr, and mpc.
Building it in Sage did not work, it failed in building gfrortan with some obscure MPFR-related linking error.

[dimpase]

Dependencies: #24602

[dimpase]

Changed dependencies from #24602 to #24597, #24602

[jdemeyer]

comment:9

Lots of issues mentioned in the ticket description do not have a ticket yet. Should I create tickets for each?

[dimpase]

comment:10

One more issue is just reported here: http://www.singular.uni-kl.de:8002/trac/ticket/816

Singular uses specific -Wl linker syntax in libpolys/polys/Makefile.am
which nukes linking (as Solaris linker does not accept dynamic_lookup parameter).
(By right, it's autotools that should generate such platform-specific arguments, it's asking for trouble to give them explicitly).

Feel free to open as many tickets as needed- making sure that we don't duplicate each other.

[dimpase]

Changed dependencies from #24597, #24602 to #24597, #24602, 24611

[dimpase]

comment:11

Opened #24611 to deal with Singular linking problems

[jdemeyer]

Changed dependencies from #24597, #24602, 24611 to #24597, #24602, #24611

[dimpase]

comment:13

the next snag is R, a bus error during installation of 'tools' package:

...
gcc -shared -L../../../../lib -m64 -L/datapool/dima/Sage/sagetrac-mirror/local/lib -Wl,-rpath,/datapool/dima/Sage/sagetrac-mirror/local/lib -o tools.so text.o init.o Rmd5.o md5.o signals.o install.o getfmts.o http.o gramLatex.o gramRd.o -L../../../../lib -lR
make[9]: Entering directory '/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/r-3.4.3/src/src/library/tools/src'
mkdir -p -- ../../../../library/tools/libs
/src'
make[7]: Leaving directory '/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/r-3.4.3/src/src/library/tools'
make[7]: Entering directory '/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/r-3.4.3/src/src/library/tools'
installing 'sysdata.rda'
make[7]: Leaving directory '/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/r-3.4.3/src/src/library/tools'

 *** caught bus error ***
address 101556b6c, cause 'invalid alignment'

Traceback:
 1: strsplit(x, "\n[ \t\n]*\n", perl = TRUE)
 2: is.vector(X)
 3: lapply(strsplit(x, "\n[ \t\n]*\n", perl = TRUE), strsplit, "[ \t\n]",     perl = TRUE)
 4: strwrap(paste0(x, ": ", y), exdent = indent, width = width, simplify = FALSE)
 5: formatDL(rep.int(tag, length(val)), val, style = "list", width = width,     indent = indent)
 6: fmt(nmx[j], xj[i], foldable[j])
 7: write.dcf(rbind(x), dfile, keep.white = c(.keep_white_description_fields,     "Maintainer", "BugReports"))
 8: .write_description(db, file.path(outDir, "DESCRIPTION"))
 9: tools:::.install_package_description(".", "../../../library/tools")
An irrecoverable exception occurred. R is aborting now ...
/bin/sh: 13985: Bus error(coredump)
make[6]: *** [Makefile:36: all] Bus Error

[dimpase]

comment:14

The complete log of R installation is here. http://users.ox.ac.uk/~coml0531/tmp/r-343-solaris11.log

I suppose this is a compiler bug (with gcc 7.2.0). I'll try different gcc versions.

[jdemeyer]

comment:15

To avoid the MPIR trouble, I'm trying ./configure --with-mp=gmp.

[jdemeyer]

Changed dependencies from #24597, #24602, #24611 to #24597, #24602, #24611, #23733

[dimpase]

comment:19

Replying to @dimpase:

The complete log of R installation is here. http://users.ox.ac.uk/~coml0531/tmp/r-343-solaris11.log

I suppose this is a compiler bug (with gcc 7.2.0). I'll try different gcc versions.

Same error with gcc 7.3.0. Will try gcc 5.4 now.

[jdemeyer]

Branch: u/jdemeyer/port_sage_to_sparc_solaris_11

[dimpase]

comment:21

the problem of R is in pcre (that's where it crashes), and pcre fails its self-tests:

[pcre-8.40.p1] make[5]: Entering directory '/datapool/dima/Sage/sagetrac-mirror/local/var/tmp/sage/build/pcre-8.40.p1/src'
[pcre-8.40.p1] PASS: pcre_stringpiece_unittest
[pcre-8.40.p1] PASS: pcre_scanner_unittest
[pcre-8.40.p1] PASS: pcrecpp_unittest
[pcre-8.40.p1] ./test-driver: line 107: 25794 Bus Error               (core dumped) "$@" > $log_file 2>&1
[pcre-8.40.p1] FAIL: pcre_jit_test
[pcre-8.40.p1] FAIL: RunTest
[pcre-8.40.p1] FAIL: RunGrepTest
[pcre-8.40.p1] ============================================================================
[pcre-8.40.p1] Testsuite summary for PCRE 8.40
[pcre-8.40.p1] ============================================================================
[pcre-8.40.p1] # TOTAL: 6
[pcre-8.40.p1] # PASS:  3
[pcre-8.40.p1] # SKIP:  0
[pcre-8.40.p1] # XFAIL: 0
[pcre-8.40.p1] # FAIL:  3
[pcre-8.40.p1] # XPASS: 0
[pcre-8.40.p1] # ERROR: 0

---

New commits:

364ec35	LD_LIBRARY_PATH="." everywhere except OSX
6eeec43	trac 23733: deprecate SAGE64 and CFLAG64
0ad41ab	Stop supporting SAGE64
b4d1eb6	Merge commit '0ad41abda20e2c5fac12e2f78a12e66ac10f6d1b' into t/24596/port_sage_to_sparc_solaris_11

[dimpase]

Commit: b4d1eb6

[dimpase]

comment:57

Are you sure it's a good idea not to pass -D_XPG6 to g++, too?

[jdemeyer]

comment:58

Replying to @dimpase:

Are you sure it's a good idea not to pass -D_XPG6 to g++, too?

It seems in particular needed to compile C99 code. But I guess it doesn't hurt to add it to CXXFLAGS.

[jdemeyer]

Changed dependencies from #24597, #24602, #24611, #23733, #24618, #24619, #24628, #24629, #24638, #24642, #24643, #23696, #24650, #24651 to #24597, #24602, #24611, #23733, #24618, #24619, #24628, #24629, #24638, #24643, #23696, #24650, #24651, #24667

[sagetrac-git]

Changed commit from f613de5 to ee3d7f8

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

9a662f7	trac 23733: deprecate SAGE64 and CFLAG64
ded617f	Stop supporting SAGE64 except in Numpy
3e34cd3	Fix building fplll on Solaris
8d93e0c	Fix giac build on SunOS
db16bc9	Build PCRE without JIT if it does not work
97ac0ac	Fix R build on Solaris
e350b30	Fix OpenBLAS build on SPARC
3df9f9f	correctly linking Flint on SunOS and FreeBSD
93de16e	Clean up partitions_c.cc
ee3d7f8	Merge commit '364ec35f1cbdab920e52c44341b8c19c8183b818'; commit 'e6237fbfe56870290b8524458d2740ee85b2668a'; commit '504dbaf654efa16170bfbc8cf4c76568d35c5caa'; commit 'ded617f86815657dded89ab7e5801d383cf67ecf'; commit '3e34cd3b08480d42761cad93b2db51650e54d2cc'; commit '8d93e0c2b69163c82762b5680e8ef615930f202c'; commit 'db16bc954c742ea82878da606e8737e57230865b'; commit '97ac0ac61e926e78482d0006f5c3ca6ce4f486a4'; commit 'e350b306612946bcc407efeebee331441d5728c5'; commit '3df9f9ffabb2b7ec588646160c337521089c30bd'; commit '93de16e381c300c875604494092027d086a83f50' into t/24596/port_sage_to_sparc_solaris_11

[sagetrac-git]

Changed commit from ee3d7f8 to 760cfbb

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

f82683c	Use pyconfig.h from Python 3 to build subprocess32
760cfbb	Merge commit 'f82683c27fda13c9fb6b19f19ca8b26ca04fbae4' into t/24596/port_sage_to_sparc_solaris_11

[jdemeyer]

Changed dependencies from #24597, #24602, #24611, #23733, #24618, #24619, #24628, #24629, #24638, #24643, #23696, #24650, #24651, #24667 to #23733, #24638, #24643, #24650, #24651, #24667

[dimpase]

comment:64

testing:

sage -t src/sage/plot/plot3d/list_plot3d.py  # 1 doctest failed
sage -t src/sage/plot/plot3d/base.pyx  # Killed due to bus error
sage -t src/sage/modular/abvar/abvar.py  # Timed out
sage -t src/sage/modular/abvar/lseries.py  # Timed out
sage -t src/sage/modular/modform/element.py  # Timed out
sage -t src/sage/modular/modform_hecketriangle/element.py  # Timed out
sage -t src/sage/modular/modform_hecketriangle/readme.py  # Timed out
sage -t src/sage/schemes/elliptic_curves/ell_point.py  # 1 doctest failed
sage -t src/sage/functions/wigner.py  # 1 doctest failed
sage -t src/sage/rings/bernmm.pyx  # 2 doctests failed
sage -t src/sage/rings/number_field/number_field.py  # Timed out
sage -t src/sage/rings/padics/padic_extension_leaves.py  # Timed out
sage -t src/sage/geometry/riemannian_manifolds/surface3d_generators.py  # 1 doctest failed
sage -t src/sage/misc/cython.py  # 1 doctest failed
sage -t src/sage/misc/compat.py  # 1 doctest failed
sage -t src/sage/tests/parigp.py  # Timed out
sage -t src/sage/tests/cmdline.py  # 1 doctest failed
sage -t src/sage/lfunctions/dokchitser.py  # Timed out
sage -t src/sage/graphs/generators/smallgraphs.py  # 1 doctest failed
sage -t src/sage/repl/image.py  # Killed due to bus error
sage -t src/sage/matrix/matrix2.pyx  # Killed due to bus error
sage -t src/sage/dynamics/complex_dynamics/mandel_julia.py  # Killed due to bus error
sage -t src/sage/dynamics/complex_dynamics/mandel_julia_helper.pyx  # Killed due to bus error
sage -t src/doc/en/thematic_tutorials/explicit_methods_in_number_theory/l_series.rst  # Timed out

here is the log for failing tests:
http://users.ox.ac.uk/~coml0531/tmp/test5.log

[dimpase]

comment:65

see #24765 for a GAP problem.

[dimpase]

Changed dependencies from #23733, #24638, #24643, #24650, #24651, #24667 to #23733, #24638, #24643, #24650, #24651, #24667, 24765

[dimpase]

Changed dependencies from #23733, #24638, #24643, #24650, #24651, #24667, 24765 to #23733, #24638, #24643, #24650, #24651, #24667, #24765

[jdemeyer]

Changed dependencies from #23733, #24638, #24643, #24650, #24651, #24667, #24765 to #23733, #24638, #24765

[mkoeppe]

comment:68

Do "we" have still access to the system? Can it be configured as a self-hosted runner for GitHub actions? https://help.github.com/en/actions/hosting-your-own-runners

[dimpase]

comment:69

no access since last year, no reply from the sysadmin.
I can offer an OpenBSD box as a replacement :-)

[mkoeppe]

comment:70

Outdated, should be closed

[dimpase]

Reviewer: Dima Pasechnik