Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove nsslib from IPA #367

Closed
wants to merge 12 commits into from
Closed

Remove nsslib from IPA #367

wants to merge 12 commits into from

Conversation

stlaz
Copy link
Contributor

@stlaz stlaz commented Jan 4, 2017

This batch of patches removes NSSConnection along with the whole ipapython.nsslib from IPA and replaces it with more standard httplib.HTTPSConnection.

NSSConnection was causing a lot of trouble in the past because it is apparently very fragile when it comes to nss library initialization. On top of that, when NSSConnection is used to set up an HTTPS connection in FIPS, it always requires a password to NSS database as NSS apparently tries to create a temporary private key and store it to the database even though client authentication is not required in the SSL connection.

TODO (will require changes in certmonger/dogatg.c):

  • remove NSSConnection from client modules
  • remove NSSConnection from server modules where it's used to connect to the certificate server
  • remove the nsslib library completely
  • we may probably remove ipaCert from /etc/httpd/alias and stop tracking it with certmonger
  • once ^- is done, track /var/lib/ipa/ra-agent.pem in certmonger instead

https://fedorahosted.org/freeipa/ticket/5695

@tiran
Copy link
Member

tiran commented Jan 4, 2017

  • Ticket 5695 is about FreeIPA on FIPS enabled systems. Moving from NSS to OpenSSL is a big change and should be tracked by its own ticket.
  • Are customers fine with the fact that FreeIPA clients will no longer very CRLs? OpenSSL does not automatically download and verify CRLs. OCSP is not yet supported by Python's ssl module.

ipalib/util.py Outdated
# TLSv1 and later is to setup SSLContext with PROTOCOL_SSLv23
# and then negate the insecure SSLv2 and SSLv3
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ctx.options |= (ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3)
Copy link
Member

@tiran tiran Jan 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • () are not needed
  • Python sets these flags and more by default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • alright
  • I was thinking explicit is better than implicit but I can remove it if you insist

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keep it :)

ipalib/util.py Outdated
# official Python documentation states that the best option to get
# TLSv1 and later is to setup SSLContext with PROTOCOL_SSLv23
# and then negate the insecure SSLv2 and SSLv3
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You want to set secure settings for cipher suites, too. The default settings are rather bad. In fact it would be better if you use ssl.create_default_context instead of ssl.SSLContext.

Copy link
Contributor Author

@stlaz stlaz Jan 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we dropped create_default_context as it was loading system-wide certs which we do not want.

I did not find any mentions in the official documentation about create_default_context() setting different cipher suites from SSLContext, can you elaborate more on which should be used?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create_default_context does not load the system certificates if you pass in a cafile, capath or cadata argument. Does it really makes sense to ignore the system's trust store? How are we going to validate certificates from 3rd parties, when NSSDB is gone? FreeIPA supports 3rd party certificates for FreeIPA UI, e.g. Lets Encrypt.

Contrary to SSLContext, create_default_context sets additional options like better selection and order of cipher suites. (https://docs.python.org/3/library/ssl.html#ssl.create_default_context).

The settings are: PROTOCOL_TLS, OP_NO_SSLv2, and OP_NO_SSLv3 with high encryption cipher suites without RC4 and without unauthenticated cipher suites.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK we are ignoring system's trust store now so I do not think this should be changed. As for 3rd party certificates, I think all these are store in /etc/ipa/ca.crt so we should be safe there (see ipa_certupdate.py:update_client()).

I am not opposed to using create_default_context(), although I thought we may want to have the SSLContext set up exactly according to our needs and in the same way in all Python versions should this ever change.

ipalib/util.py Outdated
ctx.load_verify_locations(cafile)

if client_certfile is not None:
ctx.load_cert_chain(client_certfile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we using unencrypted client certs? Python's ssl module can handle separate key/cert files and encrypted private keys.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are. I guess the question here is whether we'd be able to track it with certmonger.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so:

Usage: ipa-getcert request [options]

Required arguments:
* If using an NSS database for storage:
  -d DIR        NSS database for key and cert
  -n NAME       nickname for NSS-based storage (only valid with -d)
  -t NAME       optional token name for NSS-based storage (only valid with -d)
* If using files for storage:
  -k FILE       PEM file for private key
  -f FILE       PEM file for certificate (only valid with -k)
* If keys are to be encrypted:
  -p FILE       file which holds the encryption PIN
  -P PIN        PIN value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the hint, this may work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about this - do we get anything by encrypting the client cert (ra-agent.pem)? The file is root-readable only, if we encrypt it we're only adding more overhead to reading it as the same user can get to the encryption PIN anyway.

Copy link
Member

@tiran tiran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments

@stlaz
Copy link
Contributor Author

stlaz commented Jan 4, 2017

You're right, I should probably write some design. The current implementation does not check CRL or OSCP, so we're "fine" with this change. There is a plan on doing CRL check in certmonger, though.

@rcritten
Copy link
Contributor

rcritten commented Jan 4, 2017

Did you open a bug against NSS or python-nss regarding the PIN requirement?

@stlaz
Copy link
Contributor Author

stlaz commented Jan 4, 2017

@rcritten I spoke to the NSS people who assured me it's the intended behavior. But thanks for the remainder, I will open a Bugzilla for that as well, I was considering it before Christmas.

edit: https://bugzilla.redhat.com/show_bug.cgi?id=1410143
edit2: The above bug was closed as NOTABUG

self.ca_certificate_nickname = "caCert"
self._read_password()
self.ca_cert = paths.IPA_CA_CRT
self.client_certfile = paths.RA_AGENT_PEM
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to myself: RA_AGENT_PEM will need to move to dot_ipa as well. Testing guide on Wiki will need updating, too.

ipalib/util.py Outdated
# official Python documentation states that the best option to get
# TLSv1 and later is to setup SSLContext with PROTOCOL_SSLv23
# and then negate the insecure SSLv2 and SSLv3
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

create_default_context does not load the system certificates if you pass in a cafile, capath or cadata argument. Does it really makes sense to ignore the system's trust store? How are we going to validate certificates from 3rd parties, when NSSDB is gone? FreeIPA supports 3rd party certificates for FreeIPA UI, e.g. Lets Encrypt.

Contrary to SSLContext, create_default_context sets additional options like better selection and order of cipher suites. (https://docs.python.org/3/library/ssl.html#ssl.create_default_context).

The settings are: PROTOCOL_TLS, OP_NO_SSLv2, and OP_NO_SSLv3 with high encryption cipher suites without RC4 and without unauthenticated cipher suites.

ipalib/util.py Outdated
ctx.load_verify_locations(cafile)

if client_certfile is not None:
ctx.load_cert_chain(client_certfile)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess so:

Usage: ipa-getcert request [options]

Required arguments:
* If using an NSS database for storage:
  -d DIR        NSS database for key and cert
  -n NAME       nickname for NSS-based storage (only valid with -d)
  -t NAME       optional token name for NSS-based storage (only valid with -d)
* If using files for storage:
  -k FILE       PEM file for private key
  -f FILE       PEM file for certificate (only valid with -k)
* If keys are to be encrypted:
  -p FILE       file which holds the encryption PIN
  -P PIN        PIN value

ipalib/util.py Outdated
# TLSv1 and later is to setup SSLContext with PROTOCOL_SSLv23
# and then negate the insecure SSLv2 and SSLv3
ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ctx.options |= (ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keep it :)

ipalib/util.py Outdated
ctx.options |= version

if cafile is not None:
ctx.verify_mode = ssl.CERT_REQUIRED
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the fact that no cafile implicitly means no validation. I'd rather have an explicit argument like verify=False with default setting verify=True. By the way is there any use case to have an unverified connection in FreeIPA?

@tiran
Copy link
Member

tiran commented Jan 5, 2017

ctx = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ctx.options = ssl.OP_ALL | ssl.OP_NO_COMPRESSION | ssl.OP_SINGLE_DH_USE | ssl.OP_SINGLE_ECDH_USE | ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3
try:
    # use Fedora crypto policy
    # https://fedoraproject.org/wiki/Changes/CryptoPolicy
    ctx.set_ciphers("PROFILE=SYSTEM")
except ssl.SSLError:
    # high ciphers without RC4, MD5, TripleDES, pre-shared key and secure remote password
    ctx.set_ciphers("HIGH:!aNULL:!eNULL:!MD5:!RC4:!3DES:!PSK:!SRP")

@stlaz stlaz force-pushed the nsslib-removal branch 3 times, most recently from f313a53 to d423630 Compare January 10, 2017 12:04
@stlaz
Copy link
Contributor Author

stlaz commented Jan 10, 2017

In the last update I added SSLv2 support in IPAHTTPSConnection for backward compatibility (https://goo.gl/images/gqh2D9).
I also removed the Fedora crypto policies ciphers as we are not supporting that right now and if we did, we should do that on server as well. There would perhaps be a ticket required.
Also added a ticket to "Move RA agent certificate file export to a different location" as it fixes an issue with missing /etc/httpd/alias/kra-agent.pem as well.

ipalib/util.py Outdated

# high ciphers without RC4, MD5, TripleDES, pre-shared key
# and secure remote password
ctx.set_ciphers("HIGH:!aNULL:!eNULL:!MD5:!RC4:!3DES:!PSK:!SRP")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about you move the hard coded constants to a config option, e.g. tls_ciphers? It makes it much easier to change the option later or set Fedora crypto policy in /etc/ipa/default.conf.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that the settings in config options are meant to be user-friendly but I don't think a string such as this one is. I would just set the healthy defaults here and have an admin of the IPA server decide which ciphers should be used by setting these in the config files that are already available.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No argue on user friendly-ness here. Yes, the cipher string is awkward. The cipher string is used all over the place, e.g. Apache mod_ssl, nginx and tons of other libraries that use OpenSSL. Admins are familiar with it.

ctx.set_ciphers() overrides any default settings in other config files, e.g. system policies. In order to make the ciphers configurable, you have to offer an API.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can (and IMO should) be done in a subsequent PR.

@stlaz
Copy link
Contributor Author

stlaz commented Jan 11, 2017

I created a design for this effort: http://www.freeipa.org/page/V4/Replace_NSS_with_OpenSSL

@rcritten
Copy link
Contributor

Wait, you added support for SSLv2? Please remove it, it isn't needed even for backwards compatibility and would not be considered a regression.

@tiran
Copy link
Member

tiran commented Jan 12, 2017

@rcritten I wonder if we need to support any version except TLS 1.2 at all. Are there any versions of FreeIPA stack that do not have TLS 1.2 support?

@tiran
Copy link
Member

tiran commented Jan 12, 2017

Let's not make @stlaz jump through more bike-shedding hoops. How about we let him finish this PR, and then address TLS versions, ciphers and other simplifications in another PR?

@stlaz
Copy link
Contributor Author

stlaz commented Jan 12, 2017

@rcritten tls_version_min/max could have been set to "ssl2" just as well as "ssl3" but perhaps it's for the best to remove them. I will try to do the certmonger part and will remove this with it.

@rcritten
Copy link
Contributor

SSLv2 should not be supported, period.

Not that it would work anyway because most SSL libs have completely removed this support, but it is just a terrible idea to even try and allow it.

The rest I'm flexible on.

@stlaz stlaz force-pushed the nsslib-removal branch 2 times, most recently from b587871 to 29381fe Compare February 2, 2017 11:04
@stlaz
Copy link
Contributor Author

stlaz commented Feb 2, 2017

In the latest patchset, the "ipaCert" is removed from the "/etc/httpd/alias/" NSSDB and all the machinery around the certificate is moved accordingly.
I am addressing support of old SSL protocol versions in #396, although that one currently requires some changes.

@stlaz
Copy link
Contributor Author

stlaz commented Feb 27, 2017

All the raised issues should've been addressed in the latest PR. Except for the NSS DB creation, please answer the question in ipaserver/install/server/install.py

@stlaz stlaz force-pushed the nsslib-removal branch 2 times, most recently from 64d6cfc to eaf728d Compare February 27, 2017 12:06
@stlaz
Copy link
Contributor Author

stlaz commented Feb 27, 2017

NSS DB creation removed from server install, did not realize it does not matter anymore.

if os.path.exists(paths.IPA_RADB_DIR):
if newdb.has_nickname('ipaCert'):
if os.path.exists(paths.RA_AGENT_PEM):
if certdb.has_nickname(ra_nick):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if statement is superfluous - the condition will always be true, because it is verified to be true above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will make it go away.

@@ -177,14 +177,14 @@ def install_check(standalone, replica_config, options):
if standalone:
dirname = dsinstance.config_dirname(
installutils.realm_to_serverid(realm_name))
cadb = certs.CertDB(realm_name, subject_base=options._subject_base)
cadb = certs.CertDB(realm_name, paths.HTTPD_ALIAS_DIR,
subject_base=options._subject_base)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same NSS database as below in install() should be used here, i.e. paths.PKI_TOMCAT_ALIAS_DIR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, missed that.

@@ -420,7 +419,8 @@ def __setup_ssl(self):

def __import_ca_certs(self):
# first for the RA DB
db = certs.CertDB(self.realm, subject_base=self.subject_base)
db = certs.CertDB(self.realm, paths.HTTPD_ALIAS_DIR,
subject_base=self.subject_base)
self.import_ca_certs(db, self.ca_is_configured)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this needs to be done twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol 💩

self.sec_dir = paths.IPA_RADB_DIR
self.pwd_file = os.path.join(paths.IPA_RADB_DIR, 'pwdfile.txt')
self.sec_dir = paths.HTTPD_ALIAS_DIR
self.pwd_file = os.path.join(paths.HTTPD_ALIAS_DIR, 'pwdfile.txt')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not make sense anymore, the same as in RestClient.__init__() should be done here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@@ -46,7 +46,7 @@ def PKI_TOMCAT_password_callback():


def HTTPD_password_callback():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this unused now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did not find it anywhere, will remove it.

@HonzaCholasta
Copy link
Contributor

Upgrade from 4.3 fails with:

2017-02-28T07:07:18Z DEBUG Starting external process
2017-02-28T07:07:18Z DEBUG args=/usr/bin/pk12util -d /etc/httpd/alias -o (6, '/etc/httpd/alias/tmpFNEJrK') -n ipaCert -k /etc/httpd/alias/pwdfile.txt
2017-02-28T07:07:18Z DEBUG Process execution failed
2017-02-28T07:07:18Z DEBUG Destroyed connection context.ldap2_139873144635088
2017-02-28T07:07:18Z ERROR Upgrade failed with coercing to Unicode: need string or buffer, tuple found
2017-02-28T07:07:18Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", line 219, in __upgrade
    self.modified = (ld.update(self.files) or self.modified)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 911, in update
    self._run_updates(all_updates)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 883, in _run_updates
    self._run_update_plugin(update['plugin'])
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ldapupdate.py", line 859, in _run_update_plugin
    restart_ds, updates = self.api.Updater[plugin_name]()
  File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 1470, in __call__
    return self.execute(**options)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/plugins/update_ra_cert_store.py", line 47, in execute
    certdb.export_pkcs12(ra_nick, p12file)
  File "/usr/lib/python2.7/site-packages/ipapython/certdb.py", line 232, in export_pkcs12
    ipautil.run(args)
  File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 442, in run
    preexec_fn=preexec_fn)
  File "/usr/lib64/python2.7/subprocess.py", line 390, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.7/subprocess.py", line 1024, in _execute_child
    raise child_exception
TypeError: coercing to Unicode: need string or buffer, tuple found

2017-02-28T07:07:18Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 423, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 413, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/upgradeinstance.py", line 227, in __upgrade
    raise RuntimeError(e)
RuntimeError: coercing to Unicode: need string or buffer, tuple found

2017-02-28T07:07:18Z DEBUG   [error] RuntimeError: coercing to Unicode: need string or buffer, tuple found

@HonzaCholasta
Copy link
Contributor

CA-less to CA-full ipa-ca-install fails with:

2017-02-28T07:24:47Z DEBUG   File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 892, in run_script
    return_value = main_function()

  File "/sbin/ipa-ca-install", line 304, in main
    promote(safe_options, options, filename)

  File "/sbin/ipa-ca-install", line 270, in promote
    install_master(safe_options, options)

  File "/sbin/ipa-ca-install", line 235, in install_master
    ca.install(True, None, options)

  File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 204, in install
    install_step_1(standalone, replica_config, options)

  File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 325, in install_step_1
    config_ipa=True, config_compat=True)

  File "/usr/lib/python2.7/site-packages/ipalib/install/certstore.py", line 410, in put_ca_cert_nss
    config_ipa, config_compat)

  File "/usr/lib/python2.7/site-packages/ipalib/install/certstore.py", line 233, in put_ca_cert
    config_ipa=config_ipa, config_compat=config_compat)

  File "/usr/lib/python2.7/site-packages/ipalib/install/certstore.py", line 160, in update_ca_cert
    subject, issuer_serial, public_key = _parse_cert(dercert)

  File "/usr/lib/python2.7/site-packages/ipalib/install/certstore.py", line 39, in _parse_cert
    raise ValueError("failed to decode certificate: %s" % e)

2017-02-28T07:24:47Z DEBUG The ipa-ca-install command failed, exception: ValueError: failed to decode certificate: Unable to load certificate

@HonzaCholasta
Copy link
Contributor

ipa-replica-install with --setup-ca fails with:

2017-02-28T07:38:41Z DEBUG   File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute
    return_value = self.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 336, in run
    cfgr.run()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 328, in run
    self.execute()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 352, in execute
    for _nothing in self._executor():
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in __runner
    exc_handler(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 423, in _handle_execute_exception
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 413, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 384, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 381, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 618, in _configure
    next(executor)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in __runner
    exc_handler(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 423, in _handle_execute_exception
    self._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 481, in _handle_exception
    self.__parent._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 413, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 478, in _handle_exception
    super(ComponentBase, self)._handle_exception(exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 413, in _handle_exception
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 384, in __runner
    step()
  File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 381, in <lambda>
    step = lambda: next(self.__gen)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from
    six.reraise(*exc_info)
  File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from
    value = gen.send(prev_value)
  File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install
    for _nothing in self._installer(self.parent):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/__init__.py", line 595, in main
    replica_install(self)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 398, in decorated
    func(installer)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1455, in install
    ca.install(False, config, options)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 203, in install
    install_step_0(standalone, replica_config, options)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/ca.py", line 282, in install_step_0
    use_ldaps=standalone)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 478, in configure_instance
    self.start_creation(runtime=210)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 423, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 413, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 289, in wrapper
    ra_cert_retrieval(cls, *args)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 729, in __import_ra_key
    custodia.import_ra_key(self.master_host)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/custodiainstance.py", line 119, in import_ra_key
    cli.fetch_key('ra/ipaCert')
  File "/usr/lib/python2.7/site-packages/ipaserver/secrets/client.py", line 100, in fetch_key
    r.raise_for_status()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 844, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

2017-02-28T07:38:41Z DEBUG The ipa-replica-install command failed, exception: HTTPError: 404 Client Error: Not Found for url: https://vm-226.abc.idm.lab.eng.brq.redhat.com/ipa/keys/ra/ipaCert?type=kem&value=eyJhbGciOiJSU0EtT0FFUCIsImVuYyI6IkEyNTZDQkMtSFM1MTIiLCJraWQiOm51bGx9.UbDCRVumiqN3YDwxdfHGvkzDakB0Isbq3dTZ9tMVe5NK_wuvGmtBYGUfO46IQmBsqto0N28WJUcselfuY8Q3uSOEPb0HximzbmnJm-S1TrF9KMsHgozwhNjXDAXapXSmiFqyKiTPAvxLzx0OKq052oIbnUsprk3s6R_mQ5AWrP52DsVqM9EovOHa6RM3t4xasamhJ_at1eR647TGqS8pamulRkSilK-kxyPQnN7Rphz_cyr0yIlG93xnfQTsCJ9WcodWmVPrPgP3PvKh8OaINHqUsfT-gUR0IR3hXkN5slAYSxOdmNYwLVR-wtc6Yh3jf4LpRgbHhfmuNx8rTU3nuw.-xfhG2UGKLBU7D6UzO4d5g.zG5YCUsskbqiYRYEFGqDwbx0JghL2Yo-oioFM8KBVoxvfVNtrUzN96TU-aQn08WXS2GFvAdXo7-EpOMtrFwPEGiWfXhLDjYAhVCAmu69YYjptCDokSEY8PK1HtUJanVTb0LtcPlp78yNyM-ZGC42-PfMiiG66rlWMMpWtAWPHugxDa8EcV8AlTFdqtqFHwzRYxISXUbuiwGD_h8pht2irYGdSeJ6Aa6Fwk54ZQdQshb24njjBt-MIrgy1YKlTkF4nPhuPhH0o70IWFoQSQ24R7GVfofMc3xnoUCtPlv-QLnaOucnvCDZPUBx33JGRJP5Y7Acpp_MJTkWqmSWvmXlhNouWXtpe3oTakKNYqqurNieJtHaGmIyuyn0yzQcv4w1re8aEn9Zv2TIZvSfq5qxMqUvlhiyhej6ZCRQ1FdLaslPbFuullik95Ik-pF7BBtvn3d5LKnZJgYnQg2n3yzi73zdMdync_rovg5abmWKLAM_SrRVgeoJ9-TQrJ18HvuViIoz1n14-TQyGKaw8hsbNGJZE0vrPx0gVTl4-HJK9PLP8M1jUylSxtVRC-Xv1bny4LkLQotJuV0wbZDKHg77gk3xolAUAZ6ZrWD8xUVprdJlYW423NYyX_t-2-c6HUQBlmZqkv_lYrgaZ5WZ5mw2U1MwVxf_KU6SnwYM3kPQm3a13KkeoK9IdmnZv3YAvrKOjtekmUwCjaItX74FAg2IJA.eoL7I4gO67k_Ejt9ttgKFnw29RC3Uzz1Ykn6-mA4DsM

@stlaz
Copy link
Contributor Author

stlaz commented Feb 28, 2017

The issues should hopefully be fixed

@stlaz
Copy link
Contributor Author

stlaz commented Feb 28, 2017

Fixed another issue with CA-less to CA-full upgrade.

@HonzaCholasta
Copy link
Contributor

Upgrade from 4.4.3 asks for a PKCS#12 file password and then fails:

  Cleanup     : freeipa-server-common-4.4.3-1.fc25.noarch                                                                                                                                                    14/16 
  Cleanup     : freeipa-client-common-4.4.3-1.fc25.noarch                                                                                                                                                    15/16 
  Cleanup     : freeipa-common-4.4.3-1.fc25.noarch                                                                                                                                                           16/16 
Enter password for PKCS12 file: 
Re-enter password: 
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
Unexpected error - see /var/log/ipaupgrade.log for details:
NetworkError: cannot connect to 'ldapi://%2fvar%2frun%2fslapd-ABC-IDM-LAB-ENG-BRQ-REDHAT-COM.socket': 
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
  Verifying   : freeipa-client-4.4.90.dev201703010721+git5bb660e-0.fc25.x86_64                                                                                                                                1/16 
  Verifying   : freeipa-client-common-4.4.90.dev201703010721+git5bb660e-0.fc25.noarch                                                                                                                         2/16 
  Verifying   : freeipa-common-4.4.90.dev201703010721+git5bb660e-0.fc25.noarch                                                                                                                                3/16 

The "ipaCert" nicknamed certificate is not required to be
in /var/lib/ipa/radb NSSDB anymore as we were keeping a copy
of this file in a separate file anyway. Remove it from there
and track only the file. Remove the IPA_RADB_DIR as well as
it is not required anymore.

https://fedorahosted.org/freeipa/ticket/5695
https://fedorahosted.org/freeipa/ticket/6680
@stlaz
Copy link
Contributor Author

stlaz commented Mar 1, 2017

This should now be fixed. In my endless naivety I had thought passing no password to export_pkcs12() would actually mean no password will be set.

@HonzaCholasta
Copy link
Contributor

CA-less to CA-ful conversion still fails:

2017-03-01T09:14:40Z DEBUG Starting external process
2017-03-01T09:14:40Z DEBUG args=/usr/sbin/pkispawn -s CA -f /tmp/tmpgj_Ue4
2017-03-01T09:14:40Z DEBUG Process finished, return code=1
2017-03-01T09:14:40Z DEBUG stdout=Log file: /var/log/pki/pki-ca-spawn.20170301101440.log
Loading deployment configuration from /tmp/tmpgj_Ue4.
Installing CA into /var/lib/pki/pki-tomcat.
Storing deployment configuration into /etc/sysconfig/pki/tomcat/pki-tomcat/ca/deployment.cfg.

Installation failed: Directory '/etc/pki/pki-tomcat' already exists!


2017-03-01T09:14:40Z DEBUG stderr=pkispawn    : ERROR    ....... Directory '/etc/pki/pki-tomcat' already exists!

2017-03-01T09:14:40Z CRITICAL Failed to configure CA instance: Command '/usr/sbin/pkispawn -s CA -f /tmp/tmpgj_Ue4' returned non-zero exit status 1
2017-03-01T09:14:40Z CRITICAL See the installation logs and the following files/directories for more information:
2017-03-01T09:14:40Z CRITICAL   /var/log/pki/pki-tomcat
2017-03-01T09:14:40Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 423, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 413, in run_step
    method()
  File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 611, in __spawn_instance
    nolog_list=(self.dm_password, self.admin_password)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py", line 144, in spawn_instance
    self.handle_setup_error(e)
  File "/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py", line 391, in handle_setup_error
    raise RuntimeError("%s configuration failed." % self.subsystem)
RuntimeError: CA configuration failed.

2017-03-01T09:14:40Z DEBUG   [error] RuntimeError: CA configuration failed.

Not sure if it's caused by the PR or not, but either way it can be fixed later.

@HonzaCholasta
Copy link
Contributor

ipa-replica-install --setup-ca still fails with the same error though.

@stlaz
Copy link
Contributor Author

stlaz commented Mar 1, 2017

@HonzaCholasta I saw this issue as well, once you hit it on a VM no pkispawn will run correctly. I am not sure if it's caused by this PR, my guess is it shouldn't be as pkispawn was not touched at all but I can't be sure.

@HonzaCholasta
Copy link
Contributor

OK. Let's fix it later.

@HonzaCholasta HonzaCholasta added the ack Pull Request approved, can be merged label Mar 1, 2017
@HonzaCholasta HonzaCholasta added the pushed Pull Request has already been pushed label Mar 1, 2017
@stlaz stlaz deleted the nsslib-removal branch September 11, 2017 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ack Pull Request approved, can be merged pushed Pull Request has already been pushed
Projects
None yet
4 participants