CONNECT_ERROR in 3rd TLS handshake on Fedora (NSS backend)

[tiran]

PR #42 introduced a test for SASL EXTERNAL auth with TLS client certs. The test case works fine on Travis (Ubuntu). It's also passing on Fedora 27 when executed alone. But when executed with the rest of the test case, slapd refuses STARTTLS with CONNECT_ERROR. PR #59 disables the test temporarily.

I need to investigate why the test is failing. It might be related to the fact that Fedora uses NSS instead of OpenSSL.

[tiran]

I was able to track this down to a problem with NSS layer in openldap. It looks like SECMOD_RestartModules() call fails. I also wonder why tlsm_deferred_ctx_init is called again. It's protected by PR_CallOnceWithArg() so it should only be initialized once.

Breakpoint 2, alloc_handle (ctx_arg=0x55f297b0e480, is_server=0) at tls2.c:280
280     {
(gdb) n
284             if ( ctx_arg ) {
(gdb) p ctx
$6 = <optimized out>
(gdb) n
292             ssl = tls_imp->ti_session_new( ctx, is_server );
(gdb) p tls_imp
$7 = <optimized out>
(gdb) p tls_imp->ti_session_new
value has been optimized out
(gdb) s
tlsm_session_new (ctx=0x55f297b0e480, is_server=0) at tls_m.c:2769
warning: Source file is more recent than executable.
2769    {
(gdb) n
2776            c->tc_is_server = is_server;
(gdb) n
2777            LDAP_MUTEX_LOCK( &tlsm_init_mutex );
(gdb) n
2778            status = PR_CallOnceWithArg( &c->tc_callonce, tlsm_deferred_ctx_init, c );
(gdb) n
2779            LDAP_MUTEX_UNLOCK( &tlsm_init_mutex );
(gdb) n
2778            status = PR_CallOnceWithArg( &c->tc_callonce, tlsm_deferred_ctx_init, c );
(gdb) n
2779            LDAP_MUTEX_UNLOCK( &tlsm_init_mutex );
(gdb) n
2780            if ( PR_SUCCESS != status ) {
(gdb) p status
$8 = PR_FAILURE

Next session

Breakpoint 3, tlsm_deferred_ctx_init (arg=0x55f297b0e480) at tls_m.c:2329
2329    {
(gdb) n
2339            if ( tlsm_deferred_init( ctx ) ) {
(gdb) s
tlsm_deferred_init (arg=<optimized out>) at tls_m.c:1836
1836            struct ldaptls *lt = ctx->tc_config;
(gdb) n
1855            if ( SECFailure == ( rc = SECMOD_RestartModules(PR_FALSE /* do not force */) ) ) {
(gdb) n
tlsm_deferred_ctx_init (arg=0x55f297b0e480) at tls_m.c:2329
2329    {
(gdb) n
2339            if ( tlsm_deferred_init( ctx ) ) {
(gdb) n
2340                Debug( LDAP_DEBUG_ANY,
(gdb) n
2656                            return -1;
(gdb) n
2682    }

[tiran]

Fedora issue: https://bugzilla.redhat.com/show_bug.cgi?id=1519167

[tiran]

It was an issue in OpenLDAP's NSS backend, see https://bugzilla.redhat.com/show_bug.cgi?id=1520990 for more details. Matus Honek has pushed fixes:

• openldap-2.4.45-2.fc26 https://bodhi.fedoraproject.org/updates/FEDORA-2017-ca4946dac6
• openldap-2.4.45-4.fc27 https://bodhi.fedoraproject.org/updates/FEDORA-2017-3789b9721a