Update and clean up openssl #13126

jhpalmieri · 2012-06-17T17:50:58Z

The current openssl package doesn't build on OS X Lion and OS X PowerPC. Here is an spkg which does; it also updates openssl from 1.0.1a to 1.0.1c:

http://sage.math.washington.edu/home/palmieri/SPKG/openssl-1.0.1c.p0.spkg

Component: packages: optional

Keywords: openssl PyOpenSSL ssl.h SageNB notebook 0.9.0

Author: John Palmieri, Jeroen Demeyer

Reviewer: Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman, Benjamin Jones

Issue created by migration from https://trac.sagemath.org/ticket/13126

jhpalmieri · 2012-06-17T17:55:59Z

comment:1

The previous version of openssl had no Mercurial repository (!), so I created one, then made the changes relevant to this ticket. I'm attaching the diff showing those changes. This also adds an spkg-check script.

OS X Lion: builds and passes tests
sage.math: builds but fails tests
cleo: builds but fails tests
taurus: builds but fails tests
silius: builds but fails tests

mark, hawk: the build fails, but it did with the old version also.

I don't have the time to track down the test failures right now. I'm guessing that they would have happened with the old spkg as well, but I haven't actually tried that. For OS X, I don't have access to any non-Lion machines.

jdemeyer · 2012-06-20T07:39:39Z

comment:2

Two immediate details: mention the Trac number in SPKG.txt, use $MAKE instead of make in spkg-check.

jdemeyer · 2012-06-20T07:45:18Z

comment:3

Could you make it support OS X PPC also (removing -arch ppc analogously).

jdemeyer · 2012-06-20T07:50:00Z

comment:4

Could you please post the error message of the old OpenSSL spkg on OS X Lion?

jdemeyer · 2012-06-20T07:55:07Z

comment:5

Good news! Unlike the old spkg, the new one does build on Skynet iras.

jdemeyer · 2012-06-20T18:56:06Z

comment:6

The openssl config script is fairly broken, causing it to fail to build on Solaris.

jdemeyer · 2012-06-20T19:54:12Z

Attachment: config.patch.gz

Patch to the openssl sources to fix building on Solaris

jdemeyer · 2012-06-20T19:55:16Z

comment:7

After putting attachment: config.patch in patches, the build works on Solaris. Please add this to the spkg.

jhpalmieri · 2012-06-20T21:34:44Z

comment:8

Could you please post the error message of the old OpenSSL spkg on OS X Lion?

gcc -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch i386 -O3 -fomit-frame-pointer -DL_ENDIAN -DOPENSSL_BN_ASM_PART_WORDS -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM   -c -o cryptlib.o cryptlib.c
gcc: error: i386: No such file or directory
gcc: error: unrecognized option ‘-arch’
make[1]: *** [cryptlib.o] Error 1

I've now added config.patch to the spkg, applying it in all cases (not just on Solaris). Is this what you meant? For me, this still builds on OS X Lion (and passes tests), builds on sage.math (and fails tests), builds on mark, but doesn't build on hawk:

make[4]: Entering directory `/export/home/palmieri/testing/sage-5.1.beta3/spkg/build/openssl-1.0.1c.p0/src'
ld: fatal: file /export/home/palmieri/testing/sage-5.1.beta3/local/bin/../lib/gcc/i386-pc-solaris2.11/4.6.3/crtbegin.o: wrong ELF class: ELFCLASS32
ld: fatal: file processing errors. No output written to libcrypto.so.1.0.0
collect2: ld returned 1 exit status
make[4]: *** [link_a.solaris] Error 1
make[4]: Leaving directory `/export/home/palmieri/testing/sage-5.1.beta3/spkg/build/openssl-1.0.1c.p0/src'

This is from a version of Sage built with SAGE_INSTALL_GCC=yes, but I think my environment variables are messed up on hawk. I'll try again later.

jhpalmieri · 2012-06-21T01:05:23Z

comment:9

Okay, with a better Sage installation on hawk, this openssl spkg builds for me. It fails self-tests, but that seems to be common...

kcrisman · 2012-06-27T15:53:37Z

comment:11

if [ "$SAGE_LOCAL" = "" ]; then 
   echo "SAGE_LOCAL undefined ... exiting";

etc., probably ok, but the developer guide has

if [ -z "$SAGE_LOCAL" ]; then
    echo >&2 "SAGE_LOCAL undefined ... exiting"
    echo >&2 "Maybe run 'sage --sh'?"
    exit 1
fi

not that I want to hold up #11080 at all.

Also, I really didn't understand the very long discussion about licensing at #11080 at all - but I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

jdemeyer · 2012-06-27T16:11:28Z

comment:12

Replying to @kcrisman:

I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

OpenSSL would be required both to build and run Sage. The spkg is optional only in the sense that you don't need it if the operating system provides it. Since OpenSSL is truly required for Sage, it's dubious that Sage is obeying the GPL, but IANAL.

kcrisman · 2012-06-27T16:43:20Z

comment:13

I assume that this will be an optional spkg, only needed if one's computer doesn't have openssl dev "headers" and wants to build from scratch and/or use the OpenID logins?

OpenSSL would be required both to build and run Sage. The spkg is optional only in the sense that you don't need it if the operating system provides it. Since OpenSSL is truly required for Sage, it's dubious that Sage is obeying the GPL, but IANAL.

Right, I just meant that under ordinary circumstances this spkg would not be required.

This actually takes a while to build, I'm a little surprised... did it overwrite my existing openssl headers, by the way? Hopefully that wouldn't be a problem :(

Built ok on OS X 10.6 Intel and OS X 10.4 PPC. I did not run self-tests, pursuant to John's comment above and since it's an optional spkg.

So I guess it's ok that -fno-common:-arch ppc and friends are still in there, since it worked? I notice that in the "darwin64-x86_64-cc" line, all references to arch were removed...

What is the status here? John tested the Solaris patch from Jeroen. If Jeroen tested on Lion, then this should be mostly okay, possibly except my comment about arch and slight changing of the spkg-install (probably pointless).

kcrisman · 2012-06-27T16:43:20Z

Reviewer: Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman

kcrisman · 2012-06-27T16:43:20Z

Changed author from John Palmieri to John Palmieri, Jeroen Demeyer

jhpalmieri · 2012-06-27T17:05:10Z

comment:14

I modified the patch to make the SAGE_LOCAL check the same as in the developer's guide, and I also changed a few other error messages from echo "blah" to echo >&2 "blah".

If it builds on OS X 10.6 Intel and OS X 10.4 PPC, that's good news, since the patches were affecting those builds and I don't have access to those platforms to test.

kcrisman · 2012-06-27T17:18:15Z

comment:15

I modified the patch to make the SAGE_LOCAL check the same as in the developer's guide, and I also changed a few other error messages from echo "blah" to echo >&2 "blah".

Like I said, not crucial, but if it wasn't too much effort.

If it builds on OS X 10.6 Intel and OS X 10.4 PPC, that's good news, since the patches were affecting those builds and I don't have access to those platforms to test.

Full disclosure; the PPC one still was using the Apple gcc, because I hadn't had access to it for a while (no keyboard) and it's still building the latest beta. So maybe that's a false alarm, though I have no reason to suspect not - I can try again whenever that finishes if it's really important. But the 10.6 was with Sage 5.1.beta3, so that should have the Sage-provided gcc in use.

So no worries about the other arch locations? Jeroen, have you been able to test on a Lion machine? I can't.

jhpalmieri · 2012-06-27T17:25:51Z

comment:16

For what it's worth, I've tested on two Lion machines, and it has not only built, but passed self-tests.

jdemeyer · 2012-06-27T19:49:16Z

comment:17

Interesting, for me it fails on OS X 10.4 PPC:

****************************************************
Host system:
Darwin moufang.ugent.be 8.11.0 Darwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC Power Macintos
h powerpc
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/Users/jdemeyer/sage-5.1.beta5/local/libexec/gcc/powerpc-apple-darwin8.11.0/4.6.3/lto-wrapper
Target: powerpc-apple-darwin8.11.0
Configured with: ../src/configure --prefix=/Users/jdemeyer/sage-5.1.beta5/local --with-local-prefix=/Users/jdemeyer/sage-5.1.beta5/local -
-with-gmp=/Users/jdemeyer/sage-5.1.beta5/local --with-mpfr=/Users/jdemeyer/sage-5.1.beta5/local --with-mpc=/Users/jdemeyer/sage-5.1.beta5/
local --with-system-zlib --disable-multilib
Thread model: posix
gcc version 4.6.3 (GCC)
****************************************************
Patching openssl...
patching file Configure
patching file config

Configuring openssl...
Operating system: ppc-apple-darwinDarwin Kernel Version 8.11.0: Wed Oct 10 18:26:00 PDT 2007; root:xnu-792.24.17~1/RELEASE_PPC
WARNING! If you wish to build 64-bit library, then you have to
         invoke './Configure darwin64-ppc-cc' *manually*.
Configuring for darwin-ppc-cc
Configuring for darwin-ppc-cc
[...]
if [ -n "libcrypto.1.0.0.dylib libssl.1.0.0.dylib" ]; then \
        (cd ..; make -j1 libcrypto.1.0.0.dylib); \
fi
[ -z "" ] || gcc -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -O3 -DB_ENDIAN -Wa,-force_cpusubtype_ALL -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DAES_ASM -Iinclude \
        -DFINGERPRINT_PREMAIN_DSO_LOAD -o fips_premain_dso  \
        fips_premain.c fipscanister.o \
        libcrypto.a
gcc: error: ppc: No such file or directory
gcc: error: unrecognized option '-arch'
make[4]: *** [link_a.darwin] Error 1
make[3]: *** [do_darwin-shared] Error 2
make[2]: *** [libcrypto.1.0.0.dylib] Error 2
make[1]: *** [shared] Error 2
make: *** [build_crypto] Error 1
Error building openssl.

kcrisman · 2012-06-27T20:02:21Z

comment:18

I think that my build on my machine is far enough along that I can check this now. which gcc in sage -sh returns the correct one there, so now we just have to wait 20 minutes...

gcc: error: unrecognized option '-arch'

Hmm. I'm not sure where that comes from, after the patch - were there other instances in the configuration?

jhpalmieri · 2012-06-27T20:07:07Z

Attachment: trac_13126-openssl.patch.gz

patch for openssl spkg; for review only

jhpalmieri · 2012-06-27T20:08:10Z

comment:19

Please try this version, in which I've removed a few more instances of -arch from src/Configure.

kcrisman · 2012-06-27T20:20:45Z

comment:20

Please try this version, in which I've removed a few more instances of -arch from src/Configure.

This may have to wait a bit for me (my computer is still finishing the previous one, and I have to go home soon), but maybe Jeroen can ask moufang to take a stab at it. Otherwise I'll try in a day or two.

kcrisman · 2012-06-27T20:25:18Z

comment:21

Okay, I got the same error, so this is undoubtedly the problem - remember, before you only removed the fno-common occurrences on one architecture. I'm starting it now, but I won't be here to see it finish right away. But I don't know what else it could be.

kcrisman · 2012-06-28T02:01:25Z

comment:22

Luckily, I had a 5.0.1 release candidate built on a very similar (but faster) computer at home, with the gcc spkg and so forth. Result of latest spkg posted here...

real    14m19.061s
user    8m29.944s
sys     2m46.075s
Successfully installed openssl-1.0.1c.p0
Deleting temporary build directory
/Users/crisman/Desktop/sage-5.0.1.rc1/spkg/build/openssl-1.0.1c.p0
Making Python scripts relocatable...
Finished installing openssl-1.0.1c.p0.spkg

jdemeyer · 2012-06-28T07:34:21Z

comment:23

Yep, the latest spkg does compile on OS X 10.4.

kcrisman · 2012-06-28T13:10:13Z

comment:24

Okay, then it only remains for someone other than John to test this on Lion, I guess - the spkg still looks correct.

benjaminfjones · 2012-06-28T23:57:06Z

Changed reviewer from Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman to Jeroen Demeyer, John Palmieri, Karl-Dieter Crisman, Benjamin Jones

benjaminfjones · 2012-06-28T23:57:06Z

comment:25

Builds on Lion (Mac OS X 10.7.4) (Darwin Kernel Version 11.4.0) (sage-5.1.beta6).

Here is the build log: http://sage.math.washington.edu/home/bjones/openssl-1.0.1c.p0.log

Since this is the last issue, I'll give the ticket a positive review.

haraldschilly · 2012-07-02T12:35:25Z

comment:27

moved spkg to the servers

nexttime · 2012-07-16T00:31:58Z

comment:29

For the search engines:

In file included from OpenSSL/crypto/crypto.h:17:0,
                 from OpenSSL/crypto/crypto.c:16:
OpenSSL/crypto/x509.h:17:25: fatal error: openssl/ssl.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1
Error installing pyOpenSSL-0.12.tar.gz.
Error installing package sagenb-0.9.0

nexttime · 2012-07-16T00:31:58Z

Changed keywords from openssl to openssl PyOpenSSL ssl.h SageNB notebook 0.9.0

jhpalmieri added this to the sage-5.1 milestone Jun 17, 2012

jhpalmieri added c: packages: optional labels Jun 17, 2012

jhpalmieri added the s: needs review label Jun 17, 2012

This comment has been minimized.

Sign in to view

jdemeyer changed the title ~~update and clean up openssl~~ Update and clean up openssl Jun 20, 2012

jdemeyer added s: needs work and removed s: needs review labels Jun 20, 2012

This comment has been minimized.

Sign in to view

jhpalmieri added s: needs review and removed s: needs work labels Jun 21, 2012

benjaminfjones added s: positive review and removed s: needs review labels Jun 28, 2012

jdemeyer modified the milestones: sage-5.1, sage-5.2 Jun 29, 2012

jdemeyer removed the s: positive review label Jul 2, 2012

jdemeyer closed this as completed Jul 2, 2012

jasongrout mentioned this issue Apr 15, 2016

move notebook to flask/wsgi-based notebook #11080

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update and clean up openssl #13126

Update and clean up openssl #13126

jhpalmieri commented Jun 17, 2012

jhpalmieri commented Jun 17, 2012

jdemeyer commented Jun 20, 2012

This comment has been minimized.

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

This comment has been minimized.

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jhpalmieri commented Jun 20, 2012

jhpalmieri commented Jun 21, 2012

kcrisman commented Jun 27, 2012

jdemeyer commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

jdemeyer commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 28, 2012

jdemeyer commented Jun 28, 2012

kcrisman commented Jun 28, 2012

benjaminfjones commented Jun 28, 2012

benjaminfjones commented Jun 28, 2012

haraldschilly commented Jul 2, 2012

nexttime mannequin commented Jul 16, 2012

nexttime mannequin commented Jul 16, 2012

Update and clean up openssl #13126

Update and clean up openssl #13126

Comments

jhpalmieri commented Jun 17, 2012

jhpalmieri commented Jun 17, 2012

jdemeyer commented Jun 20, 2012

This comment has been minimized.

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

This comment has been minimized.

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jdemeyer commented Jun 20, 2012

jhpalmieri commented Jun 20, 2012

jhpalmieri commented Jun 21, 2012

kcrisman commented Jun 27, 2012

jdemeyer commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

jdemeyer commented Jun 27, 2012

kcrisman commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

jhpalmieri commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 27, 2012

kcrisman commented Jun 28, 2012

jdemeyer commented Jun 28, 2012

kcrisman commented Jun 28, 2012

benjaminfjones commented Jun 28, 2012

benjaminfjones commented Jun 28, 2012

haraldschilly commented Jul 2, 2012

nexttime mannequin commented Jul 16, 2012

nexttime mannequin commented Jul 16, 2012