2.8.1 breaks Candlepin AMQP Broker

[evgeni]

Ohai,

the change in 363f0e1 has broken the AMQP Broker part of Candlepin. Tomcat starts, but fails to listen on port 61613. Looking at /var/log/tomcat/localhost.<date>.log one sees the following traceback:

1) Error injecting constructor, java.io.IOException: Could not read key
  at org.candlepin.pki.CertificateReader.<init>(CertificateReader.java:50)
  at org.candlepin.guice.CandlepinModule.bindPki(CandlepinModule.java:303)
  while locating org.candlepin.pki.CertificateReader
    for the 1st parameter of org.candlepin.pki.impl.JSSPKIUtility.<init>(JSSPKIUtility.java:101)
  while locating org.candlepin.pki.impl.JSSPKIUtility
  at org.candlepin.guice.CandlepinModule.bindPki(CandlepinModule.java:302)
  while locating org.candlepin.pki.PKIUtility
Caused by: java.io.IOException: Could not read key
        at org.candlepin.pki.impl.ProviderBasedPrivateKeyReader$PKCS8EncryptedPrivateKeyPemParser.decode(ProviderBasedPrivateKeyReader.java:210)
        at org.candlepin.pki.PrivateKeyReader$PrivateKeyPemParser.decode(PrivateKeyReader.java:47)
        at org.candlepin.pki.impl.ProviderBasedPrivateKeyReader.readPem(ProviderBasedPrivateKeyReader.java:156)
        at org.candlepin.pki.impl.ProviderBasedPrivateKeyReader.read(ProviderBasedPrivateKeyReader.java:82)
        at org.candlepin.pki.impl.ProviderBasedPrivateKeyReader.read(ProviderBasedPrivateKeyReader.java:60)
        at org.candlepin.pki.CertificateReader.readPrivateKey(CertificateReader.java:78)
        at org.candlepin.pki.CertificateReader.<init>(CertificateReader.java:58)
        at org.candlepin.pki.CertificateReader$$FastClassByGuice$$d3225fea.newInstance(<generated>)
        at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:213)
        at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:111)
        at com.google.inject.Guice.createInjector(Guice.java:87)
        at org.jboss.resteasy.plugins.guice.GuiceResteasyBootstrapServletContextListener.contextInitialized(GuiceResteasyBootstrapServletContextListener.java:56)
        at org.candlepin.guice.CandlepinContextListener.contextInitialized(CandlepinContextListener.java:133)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:5127)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5643)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
        at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1260)
        at org.apache.catalina.startup.HostConfig$DeployDirectory.run(HostConfig.java:2002)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.security.NoSuchAlgorithmException: 1.2.840.113549.1.5.13 SecretKeyFactory not available
        at java.base/javax.crypto.SecretKeyFactory.<init>(SecretKeyFactory.java:122)
        at java.base/javax.crypto.SecretKeyFactory.getInstance(SecretKeyFactory.java:168)
        at org.candlepin.pki.impl.ProviderBasedPrivateKeyReader$PKCS8EncryptedPrivateKeyPemParser.decode(ProviderBasedPrivateKeyReader.java:203)
        ... 42 more

I debugged this a bit further and found the following interesting:

• the change of the CRYPTO constant to aes265 is fine
• the change to using openssl genpkey in genServerKey is fine
• the change to using openssl genpkey in genPrivateCaKey is what's breaking it
• I think the issue is the format of /etc/candlepin/certs/candlepin-ca.key (it starts with BEGIN RSA PRIVATE KEY in the working case, and with BEGIN ENCRYPTED PRIVATE KEY in the failing case)
  if I convert the key with openssl rsa -in candlepin-ca.key -passin pass:<password> -passout pass:<password> -aes256 Candlepin starts working again.

I don't really understand why the change to the CA key (which the Artemis broker doesn't even need, IMHO) breaks it, but I guess it's safest to just revert the genpkey part of the change and fix it at a later point.