Kerberos tests failing with ubi8 image

[vdesabou]

Current kerberos tests are broken with UBI8 image (5.4.0-1-ubi8)

Could not configure server because SASL configuration did not allow the  ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: No password provided

[vdesabou]

zookeeper:

[2020-03-04 16:32:38,948] WARN No password found for user: null (org.apache.zookeeper.server.auth.SaslServerCallbackHandler)
[2020-03-04 16:32:38,963] ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.IOException: Could not configure server because SASL configuration did not allow the  ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: No password provided
	at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:243)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646)
	at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143)
	at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106)
	at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64)
	at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128)
	at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)

broker:

[main-SendThread(zookeeper.kerberos-demo.local:2181)] WARN org.apache.zookeeper.SaslClientCallbackHandler - Could not login: the Client is being asked for a password, but the ZooKeeper Client code does not currently support obtaining a password from the user. Make sure that the Client is configured to use a ticket cache (using the JAAS configuration setting 'useTicketCache=true)' and restart the Client. If you still get this message after that, the TGT in the ticket cache has expired and must be manually refreshed. To do so, first determine if you are using a password or a keytab. If the former, run kinit in a Unix shell in the environment of the user who is running this Zookeeper Client using the command 'kinit <princ>' (where <princ> is the name of the Client's Kerberos principal). If the latter, do 'kinit -k -t <keytab> <princ>' (where <princ> is the name of the Kerberos principal, and <keytab> is the location of the keytab file). After manually refreshing your cache, restart this Client. If you continue to see this message after manually refreshing your cache, ensure that your KDC host's clock is in sync with this host's clock.
[main-SendThread(zookeeper.kerberos-demo.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No password provided Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main] ERROR io.confluent.admin.utils.ClusterStatus - Error occurred while connecting to Zookeeper server[zookeeper.kerberos-demo.local:2181]. Authentication failed.

UBI8:

[appuser@zookeeper tmp]$ java Ciphers
Default Cipher

•   TLS_DHE_DSS_WITH_AES_128_CBC_SHA
  

•   TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
  

•   TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
  

•   TLS_DHE_DSS_WITH_AES_256_CBC_SHA
  

•   TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
  

•   TLS_DHE_DSS_WITH_AES_256_GCM_SHA384
  

•   TLS_DHE_RSA_WITH_AES_128_CBC_SHA
  

•   TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
  

•   TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
  

•   TLS_DHE_RSA_WITH_AES_256_CBC_SHA
  

•   TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
  

•   TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
  

•   TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
  

•   TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
  

•   TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  

•   TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
  

•   TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
  

•   TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  

•   TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
  

•   TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
  

•   TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  

•   TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
  

•   TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
  

•   TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  

•   TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
  

•   TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
  

•   TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
  

•   TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
  

•   TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
  

•   TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
  

•   TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
  

•   TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
  

•   TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
  

•   TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
  

•   TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
  

•   TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
  

•   TLS_EMPTY_RENEGOTIATION_INFO_SCSV
  

•   TLS_RSA_WITH_AES_128_CBC_SHA
  

•   TLS_RSA_WITH_AES_128_CBC_SHA256
  

•   TLS_RSA_WITH_AES_128_GCM_SHA256
  

•   TLS_RSA_WITH_AES_256_CBC_SHA
  

•   TLS_RSA_WITH_AES_256_CBC_SHA256
  

•   TLS_RSA_WITH_AES_256_GCM_SHA384
  

[vdesabou]

That's probably caused by owner being root whereas appuser is now used:

[appuser@zookeeper ~]$ cd /var/lib/secret/
[appuser@zookeeper secret]$ ls -lrt
total 36
-rw------- 1 root root 204 Mar  4 16:31 broker.key
-rw------- 1 root root 206 Mar  4 16:31 broker2.key
-rw------- 1 root root 218 Mar  4 16:31 zookeeper.key
-rw------- 1 root root 154 Mar  4 16:31 zookeeper-client.key
-rw------- 1 root root 524 Mar  4 16:31 kafka-client.key
-rw------- 1 root root 170 Mar  4 16:31 kafka-admin.key
-rw------- 1 root root 152 Mar  4 16:31 kafka-connect.key
-rw------- 1 root root 166 Mar  4 16:31 kafka-schemaregistry.key
-rw------- 1 root root 164 Mar  4 16:31 kafka-controlcenter.key

[vdesabou]

still failing with

[2020-03-04 17:12:13,002] ERROR Unexpected exception, exiting abnormally (org.apache.zookeeper.server.ZooKeeperServerMain)
java.io.IOException: Could not configure server because SASL configuration did not allow the  ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: Message stream modified (41)
        at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:243)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:646)
        at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:143)
        at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106)
        at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)

whereas I set

        -Djdk.security.allowNonCaAnchor=true
        -Dsun.security.krb5.disableReferrals=true

[vdesabou]

In this case:

        -Djdk.security.allowNonCaAnchor=true
        -Dsun.security.krb5.disableReferrals=true

was not needed
Problem was due to:

renew_lifetime = 7d

in krb5.conf

https://bugs.openjdk.java.net/browse/JDK-8131051