Apache2 Segfault with AuthnRequestsSigned

[aadlung]

My SAML Implementation basically works - when I disable the Signing of the Authn Request. I am using Debian 9.8 with the apache package of the distribution, and the mod_auth_mellon package from the test repository which has version 0.14.

As soon as I add the AuthnRequestsSigned="true" parameter to my Metadata file, the apache processes get a sigchld and a process exits.

Is there any possibility to debug further why this happens, or is there any special requirement for the signing to work?

# strace -p 20066
strace: Process 20066 attached
semop(25985024, [{0, -1, SEM_UNDO}], 1) = 0
epoll_wait(14, [{EPOLLIN, {u32=3553572296, u64=140066732062152}}], 2, 10000) = 1
accept4(6, {sa_family=AF_INET6, sin6_port=htons(41113), inet_pton(AF_INET6, "::ffff:91.204.194.30", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [128->28], SOCK_CLOEXEC) = 15
semop(25985024, [{0, 1, SEM_UNDO}], 1)  = 0
getsockname(15, {sa_family=AF_INET6, sin6_port=htons(443), inet_pton(AF_INET6, "::ffff:10.192.240.81", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, [128->28]) = 0
openat(AT_FDCWD, "/dev/urandom", O_RDONLY|O_CLOEXEC) = 16
read(16, "\3323P\17|\341p8@\274\331\257\23\224.....(truncated)>\345"..., 512) = 512
close(16)                               = 0
fcntl(15, F_GETFL)                      = 0x2 (flags O_RDWR)
fcntl(15, F_SETFL, O_RDWR|O_NONBLOCK)   = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f63d46ea6d0) = 20135
wait4(20135, [{WIFSIGNALED(s) && WTERMSIG(s) == SIGSEGV}], 0, NULL) = 20135
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=20135, si_uid=33, si_status=SIGSEGV, si_utime=0, si_stime=0} ---
openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = 16
fstat(16, {st_mode=S_IFREG|0644, st_size=2237, ...}) = 0
fstat(16, {st_mode=S_IFREG|0644, st_size=2237, ...}) = 0
read(16, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0\0\0\7\0\0\0\0"..., 4096) = 2237
lseek(16, -1419, SEEK_CUR)              = 818
read(16, "TZif2\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0\0\0\7\0\0\0\0"..., 4096) = 1419
close(16)                               = 0
getpid()                                = 20066
write(2, "[Tue Apr 16 08:02:51.996518 2019"..., 88) = 88
exit_group(1)                           = ?
+++ exited with 1 +++

The Metadata itself is quite simple

<EntityDescriptor entityID="https://saml-test.example.com" xmlns="urn:oasis:names:tc:SAML:2.0:metadata" xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<SPSSODescriptor AuthnRequestsSigned="true" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
    <KeyDescriptor use="signing">
      <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
        <ds:X509Data>
<ds:X509Certificate>MIIC.....==</ds:X509Certificate>
        </ds:X509Data>
      </ds:KeyInfo>
    </KeyDescriptor>
    <SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://saml-test.example.com/mellon/logout"/>
    <NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:persistent</NameIDFormat>
    <AssertionConsumerService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://saml-test.example.com/mellon/postResponse" index="0"/>
  </SPSSODescriptor>
</EntityDescriptor>

[haraldhh]

This sounds suspiciously exacltly why I'm here for the first time ever trying to get help. Also debian 9.8 and self-compiled 0.14.2 mod_auth_mellon (to get diagnostics support).

(gdb) run -X -k start
Starting program: /usr/sbin/apache2 -X -k start
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff3a33a1e in RSA_sign () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2
(gdb) bt
#0 0x00007ffff3a33a1e in RSA_sign () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2
#1 0x00007ffff54b11ea in ?? () from /usr/lib/liblasso.so.3
#2 0x00007ffff54f66d0 in ?? () from /usr/lib/liblasso.so.3
#3 0x00007ffff54f6894 in ?? () from /usr/lib/liblasso.so.3
#4 0x00007ffff54f4d74 in ?? () from /usr/lib/liblasso.so.3
#5 0x00007ffff54f54ac in ?? () from /usr/lib/liblasso.so.3
#6 0x00007ffff54fb628 in ?? () from /usr/lib/liblasso.so.3
#7 0x00007ffff54d312a in lasso_login_build_authn_request_msg () from /usr/lib/liblasso.so.3
#8 0x00007ffff5eb6b8d in am_init_authn_request_common (r=r@entry=0x7fffddfae0a0, login_return=login_return@entry=0x7fffffffdea0,
idp=idp@entry=0x7fffddfaa770 "http://adfs.arcada.fi/adfs/services/trust", http_method=http_method@entry=LASSO_HTTP_METHOD_REDIRECT,
destination_url=destination_url@entry=0x555555bd31a0 "https://adfs.arcada.fi/adfs/ls/",
assertion_consumer_service_url=assertion_consumer_service_url@entry=0x555555bb7840 "https://asta.arcada.fi/endpoint/postResponse",
return_to_url=0x7fffddfaa5f0 "https://asta.arcada.fi/", is_passive=0) at auth_mellon_handler.c:2945
#9 0x00007ffff5eb77b4 in am_send_login_authn_request (r=r@entry=0x7fffddfae0a0, idp=0x7fffddfaa770 "http://adfs.arcada.fi/adfs/services/trust",
return_to_url=return_to_url@entry=0x7fffddfaa5f0 "https://asta.arcada.fi/", is_passive=0) at auth_mellon_handler.c:3151
#10 0x00007ffff5eb8f92 in am_handle_login (r=0x7fffddfae0a0) at auth_mellon_handler.c:3282
#11 am_handler (r=0x7fffddfae0a0) at auth_mellon_handler.c:3540
#12 0x00005555555abd60 in ap_run_handler (r=r@entry=0x7fffddfae0a0) at config.c:170
#13 0x00005555555ac2f6 in ap_invoke_handler (r=r@entry=0x7fffddfae0a0) at config.c:434
#14 0x00005555555c3f33 in ap_process_async_request (r=0x7fffddfae0a0) at http_request.c:436
#15 0x00005555555c4040 in ap_process_request (r=r@entry=0x7fffddfae0a0) at http_request.c:471
#16 0x00005555555c00fd in ap_process_http_sync_connection (c=0x7fffe58be290) at http_core.c:210
#17 ap_process_http_connection (c=0x7fffe58be290) at http_core.c:251
#18 0x00005555555b5bd0 in ap_run_process_connection (c=c@entry=0x7fffe58be290) at connection.c:42
#19 0x00005555555b6120 in ap_process_connection (c=c@entry=0x7fffe58be290, csd=) at connection.c:226
#20 0x00007fffeaf456bf in child_main (child_num_arg=child_num_arg@entry=0, child_bucket=child_bucket@entry=0) at prefork.c:723
#21 0x00007fffeaf458da in make_child (s=0x7ffff7fc34a0, slot=slot@entry=0) at prefork.c:768
#22 0x00007fffeaf46dfd in prefork_run (_pconf=, plog=0x7ffff7fbe028, s=0x7ffff7fc34a0) at prefork.c:975
#23 0x000055555558f0fe in ap_run_mpm (pconf=0x7ffff7ff0028, plog=0x7ffff7fbe028, s=0x7ffff7fc34a0) at mpm_common.c:94
#24 0x0000555555587cfd in main (argc=, argv=) at main.c:783

Is there a way for me to disable signing of metadata from the config, and not having to maintain a metadata file just for the test?

[haraldhh]

I put AuthnRequestsSigned="false" in a copy of the generated metadata. My site now works.

[aadlung]

@haraldhh correct, as I know (although I'm not experienced with mod_auth_mellon), this is the only possibility to disable signing.
I would like to enable it again, but I just don't know if it's a mod_auth_mellon issue, a liblasso issue, or anything else...

[haraldhh]

I'm no expert but my backtrace seems to indicate liblasso (again).

[aadlung]

@haraldhh I also installed the liblasseo from the Debian SID repository (version 2.6), and on my first tests, I do not get a segfault any more, and I see a Signature header in the SAML Request..

I'm not totally sure why the request to the IdP is a GET request with all parameters as Query Strings (I would expect a POST request with data in the body), but I could successfully test it with a signature verification on the IdP side as well.

[haraldhh]

Hopefully the problems found with liblasso3 would be fixed and backported back to Debian, one might have to open a ticket over there as soon as someone is able to verify that my hunch is right.

[olavmrk]

Hi,

have you set the MellonSPPrivateKeyFile and MellonSPCertFile options? To be able to sign authentication requests and logout messages, those options must be provided. Though I agree that crashing if they are absent is a bit unfortunate.

[haraldhh]

Yes, they are set and verified as readable. I don't think the crash is caused by this.

I have to try with a newer liblasso3 when I have the opportunity. I really like to have signing as well.

[haraldhh]

I have another server on which I tried installing liblasso3 from debian backports, and signing works as it should now. I'll raise a bug on debian.

[olavmrk]

Closing this issue as part of archiving this project. See the announcement for details:

https://github.com/Uninett/mod_auth_mellon/blob/info/README.md