Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installer refactoring #232

Closed
wants to merge 77 commits into from
Closed

Installer refactoring #232

wants to merge 77 commits into from

Conversation

HonzaCholasta
Copy link
Contributor

@HonzaCholasta HonzaCholasta commented Nov 11, 2016

This PR contains the installer refactoring work we (@jcholast, @martbab, @mbasti-rh, @stlaz) did in the last 6 weeks.

What is included:

  • Replica install workflows were updated to more closely resemble each other.
  • Some classic replica installation and replica promotion code was merged.
  • Client installer was turned into a module.
  • Command line options shared between multiple installers are now defined in one place.

What is missing:

  • Use GSSAPI authentication for initial replication on domain level 0 (jcholast/freeipa#22).
  • Merge the rest of classic replica installation and replica promotion code.
  • Call into the client installer module instead of executing ipa-client-install in server installers.
  • Update the remaining installers (ipa-replica-prepare, ipa-ca-install, ipa-kra-install, ipa-dns-install) to make use of the single option definition.

Known issues:

  • Ugly help in ipa-server-install, ipa-replica-install and ipa-client-install.

https://fedorahosted.org/freeipa/ticket/6392

Note that the commits in this PR were already reviewed and ACKed over at jcholast/freeipa (list of PRs).

If you find any issues, please file a Trac ticket or leave a comment in this PR.

stlaz and others added 30 commits November 11, 2016 09:48
Removed a redundant restart in server install which was there
only so other methods of dsinstance would not fail as they would
use the wrong connection mentioned above.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Replica populate can be applied with other update plugins.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
To make the code more general, moved the update_dna_shared_config
among other update plugins.

Bugfix: DNA shared config connection protocol was compared to a
method string which would result in a try to always update it
even if there was no need to.

https://fedorahosted.org/389/ticket/48373 causes that two
shared DNA config entries are created instead of one.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
`create_from_pkcs12` method of CertDB was re-creating NSS database files
during PKCS#12 bundle import. This may cause bugs because the file permissions
could be re-set to wrong values causing subtle bugs. Modify the class API so
that the server cert chain can be imported while preserving existing FS
attributes.

https://fedorahosted.org/freeipa/ticket/6429

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In order to reduce coupling between httpinstance and other service installers,
the HTTP installer is now tasked with initialization of /etc/httpd/alias (RA
agent database) in the beginning of server/replica installation

Part of https://fedorahosted.org/freeipa/ticket/6429

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Remote master and CA host names may differ. Always use the remote CA host
name and never the remote master host name in CA replica install.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Remote master and KRA host names may differ. Always use the remote KRA host
name and never the remote master host name in KRA replica install.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Merge CA install code paths use in ipa-server-install, ipa-replica-install
in either domain level and ipa-ca-install into one.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Merge KRA install code paths use in ipa-replica-install in either domain
level and ipa-kra-install into one.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This commit only moves the code from ipa-client-install to module
ipaclient/install/client.py and fixes PEP8.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Function configure_krb5_conf always returns 0 as return state. Remove
the 'return' statement and let exceptions work

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Function always returns return code 0, and this code is even not used
elsewehere.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
We should use as specific import as possible, better for python memory
consumption and speed, and looks better in code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Instead of copy&paste is better to use constant. It makes code shorter
and improves readability, saves resources.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
At this point, httpd is not configured and the restart fails.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Merge all RA cert import code paths into a single code path in CA install.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Merge all KRA agent cert export code paths into a single code path in KRA
install.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
`CertDB.request_service_cert` could re-create NSSDB files if the supplied CA
certificate was not found in database. This could cause subtle bugs since the
files were recreated with wrong permissions. This behavior was removed so that
there are no destructive operations performed by the method.

https://fedorahosted.org/freeipa/ticket/6429

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This functionality will be reused in the DL0 host enrollment

https://fedorahosted.org/freeipa/ticket/6434

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
In order to unify domain-level specific replica installers to a single
workflow some kind of host enrollment must be done also in domain level 0
replica installation.

Here the enrollment is done by directory manager using
one-time password and only krb5.conf is configured to point to master KDC.

Since host keytab is fetched during enrollment KDC installer no longer needs
to request it during replica install.

https://fedorahosted.org/freeipa/ticket/6434

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
There should not be mixed statestore as global variable and as local
function parameter. This commit fixes usage of sysrestore and statestore
as local variables only. In future we may need to change default
statestore and fstore depending on where the functions are called and
this change makes it easier and less error prone.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Move checks from ipa-client-install to clien.install_check

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
client install contained installation check that have been moved to
install_check function

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Checks if uninstallation is possible should be moved to uninstall_check

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
There is no need to have env as parameter because this is used only
once, so it can eb safely moved to client.py module

NOTE: PATH should be overwritten to safe values before we execute any
command
https://www.securecoding.cert.org/confluence/display/c/ENV03-C.+Sanitize+the+environment+when+invoking+external+programs

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
stlaz and others added 24 commits November 11, 2016 09:59
install_check() and promote_check() have some common checks that can
be safely moved to common grounds.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Domain levels 0 and 1 use the same mechanism of checking domain
level correctness. Group them together and make it more general
should there be more domain levels in the future (although lets
hope there won't be).

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Instead of delegating handling of some parameters like fstore to the parent
class, the *Instance installers had the logic copy-pasted in their
constructors. Some other members were also moved to the Service class and the
parent class constructors in children were fixed to modern standards of
initializing parent class in Python.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This will aid further refactoring of service installers, since the user will
be defined only once during parent class initialization.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
The Service class now accepts keytab path and service name part of Kerberos
principal as members. Kerberos principal is turned into a property computed
from service prefix, FQDN and realm. the handling of Kerberos principals and
keytabs in service installers was changed to use class members instead of
copy-pasted constants. This shall aid in the future refactoring of
principal/keytab handling code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Since creation of service principals and keytab retrieval are common
operations, Service class should provide means to add service entry to LDAP,
retrieve its keytab to designated destination and change the owner to service
user.

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
In DL0 directory manager password is bundled in the supplied replica file and
the replica installer can use it to authenticate against master when
retrieving service keytabs.

In DL1, however, DM credentials are generated randomly and used during local
DS instance creation. The proper DM password is imported by custodia much
later to the process. We must not allow the installer to contact the remote
master using this random password since it would fail.

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
DS replica can now use remote API and ipa-getkeytab to create service
principal and fetch the keytab in both domain levels. There is no need to use
KDC installer to do it.

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This is required to enable password extension plugin right away so that
services configured later can use it to request keytabs via ipa-getkeytab.

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
apache keytab is now retrieved using the same method in both domain levels.
The difference lies in the authentication scheme used to retrieve service
keytab:

  * in DL0 passed in DM credentials are used
  * in DL1 GSSAPI is used

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
This functionality was merged to Service class and is not longer used
anywhere.

https://fedorahosted.org/freeipa/ticket/6405

Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Dogtag requires Directory Manager password for its installation.
On Domain Level 1 a special password for Directory Manager is
created and used during the installation. However, by importing
the real DM password from remote LDAP, we can no longer use
the temporary password from the replica installation.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Instead of specifying which knobs should be positional arguments in
cli.install_tool(), do it using a flag in knob definition, where the rest
of CLI configuration is.

As a side effect, the usage string for CLI tools can now be generated
automatically.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Let IPAOptionParser handle parsing of its supported types and use an option
callback only for unsupported types.

Instead of parsing positional arguments manually, parse them using a custom
IPAOptionParser instance, reusing the option parsing code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Add new knob() knob constructor. Keep the old Knob() constructor for
backward compatibility with old code.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Use type(None) rather than bool to define knobs which are represented as
command line flags. This allows declaring both "--option" and
"--option={0,1}"-style command line options.

Use enum.Enum subclasses instead of set literals to declare enumerations.

Use typing.List[T] instead of (list, T) to declare lists. (Note that a
minimal reimplementation of typing.List is used instead of the Python 2
backport of the typing module due to non-technical reasons.)

Use CheckedIPAddress instead of 'ip' and 'ip-local' to declare IP
addresses.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Replace cli_name, cli_short_name and cli_positional knob arguments with a
single cli_names argument, which allows defining one or more CLI names
using the argparse convention ("--option" for long option name, "-o" for
short option name and "argument" for positional argument name).

Also replace cli_aliases with cli_deprecated_names which uses the same
convention.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Declare knob bases explicitly using a keyword argument instead of guessing
if the type argument is a base or a type of the knob.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Add new @group decorator to declare an installer class as a knob group
instead of subclassing Group, so that subclassing the installer does not
create duplicates of the original group.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Add class hierarchy which allows inherting knob definitions between the
various client and server install scripts.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Migrate ipa-server-install and ipa-replica-install from the old installer
classes to the new installer class hierarchy classes.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
Migrate ipa-client-install from the custom script to the new installer
class hierarchy classes.

https://fedorahosted.org/freeipa/ticket/6392

Reviewed-By: Martin Basti <mbasti@redhat.com>
@HonzaCholasta HonzaCholasta added the ack Pull Request approved, can be merged label Nov 11, 2016
@MartinBasti
Copy link
Contributor

ACK, we may ignore those minor PEP8 issues, it is mainly caused by copying code to other parts.

@HonzaCholasta HonzaCholasta added the pushed Pull Request has already been pushed label Nov 11, 2016
@HonzaCholasta
Copy link
Contributor Author

Fixed upstream
master:
https://fedorahosted.org/freeipa/changeset/eac6f52957c361c219ad6048b515ddb62da31154
https://fedorahosted.org/freeipa/changeset/83e72d704630b9cc5a1f713dfee30601950eb5e9
https://fedorahosted.org/freeipa/changeset/7279ef1d0f28dae9f3203362ca9e2245e56e111f
https://fedorahosted.org/freeipa/changeset/2fdc2d0cb7fa98992fe6c2070cb5dc34c500ac09
https://fedorahosted.org/freeipa/changeset/b1283c1e56976a3019c81c3be88fa821431ac6a6
https://fedorahosted.org/freeipa/changeset/8a7e79a7a6fad8dc87c8f148cb5098434f988ea3
https://fedorahosted.org/freeipa/changeset/0e232b5f526168af6bb0b52244f79dfacb43a9b7
https://fedorahosted.org/freeipa/changeset/dc38d53de1eff71570ec5ef55db6de2c6f9b5bbd
https://fedorahosted.org/freeipa/changeset/0933e080aa9635bba12efc53d904d524b309027f
https://fedorahosted.org/freeipa/changeset/f98faec47847022879b8bceb63839fcfd6e45402
https://fedorahosted.org/freeipa/changeset/5c16608a0d5d4abe98319a077917f5424b72d031
https://fedorahosted.org/freeipa/changeset/49f201e2b2523c83fa2b20fe91c91733e2ee947f
https://fedorahosted.org/freeipa/changeset/cc6efb97985bb93e3cdb2a6c2943d45e1132e122
https://fedorahosted.org/freeipa/changeset/c30b45ab157f611312c0cd0f4f7c3a12d7a02c11
https://fedorahosted.org/freeipa/changeset/1c9267803c6f41cc7d7485024f8864fbd62c9128
https://fedorahosted.org/freeipa/changeset/31a9ef4f8b8e2d6bb11f68ce34a7575ced9816aa
https://fedorahosted.org/freeipa/changeset/2dedfe5d33062fc7121bf36be12d7b423b62120a
https://fedorahosted.org/freeipa/changeset/cf1c4e84e74ea15fe5cf7219872cf131bd53281e
https://fedorahosted.org/freeipa/changeset/bddd4fac462c07458297d1cea5272bde97fb3707
https://fedorahosted.org/freeipa/changeset/822e1bc82af3a6c1556546c4fbe96eeafad45762
https://fedorahosted.org/freeipa/changeset/89bb5ed1ebc0b5952a1d5eae34e0f39c5ba540d7
https://fedorahosted.org/freeipa/changeset/8e36e030910a4a6ec5ddb37cc19824f37b25ab51
https://fedorahosted.org/freeipa/changeset/3d5161d7e943fc6d4d092d18fc980fd40d21a59f
https://fedorahosted.org/freeipa/changeset/a6ec37255441294285fac58c9bf08129db110fac
https://fedorahosted.org/freeipa/changeset/19912796edf5d6427920ff67c33e6288223e0466
https://fedorahosted.org/freeipa/changeset/33537f555636db935dd809b62498e2415d765e8e
https://fedorahosted.org/freeipa/changeset/2c226ebc27e2a4e2677549003c4c70a777794296
https://fedorahosted.org/freeipa/changeset/3f690a0a3a7e039183eca1578a3cb13f2c0632ef
https://fedorahosted.org/freeipa/changeset/fcea3b3fb88ede0e9414f83ac2372e000e728587
https://fedorahosted.org/freeipa/changeset/83fe6b626fd2fb7f43ddf3568aaffca1ce569079
https://fedorahosted.org/freeipa/changeset/1f65c07524c8cf80996de9f6250a4e19c3a043c9
https://fedorahosted.org/freeipa/changeset/8cbbb5359155446be22a5efb1e2372e527d2d745
https://fedorahosted.org/freeipa/changeset/bbad08900bbe8f76e59b159cd2af800f5c089ca1
https://fedorahosted.org/freeipa/changeset/b3786730e50080fa4dadeffa86388592c10b3a62
https://fedorahosted.org/freeipa/changeset/c38ce49e8d280e52c61f722b0e5ad7aa9f53cc1a
https://fedorahosted.org/freeipa/changeset/5249eb817efbb5708d097173a8d5f1e322fb201e
https://fedorahosted.org/freeipa/changeset/847b6eddab00973740413b4c46f86940cb73d25a
https://fedorahosted.org/freeipa/changeset/0914a3aeb778986dea4020ddf8ca550ebef02bad
https://fedorahosted.org/freeipa/changeset/990e1acb1a667b90619e7799bb96e2cd81e97e61
https://fedorahosted.org/freeipa/changeset/b068d3336ad65748881d0dc74505f41dac9f0f13
https://fedorahosted.org/freeipa/changeset/a3c9def4e982bcc90e9ece0900993ace53777906
https://fedorahosted.org/freeipa/changeset/8cb315af627d712dd21396164cfa2b5d03ccb466
https://fedorahosted.org/freeipa/changeset/87c3c1abecdfb8b5eb227239eeacfbee386a7ed7
https://fedorahosted.org/freeipa/changeset/bde1d82ebe32be339c30c85048fd18e1ce99867d
https://fedorahosted.org/freeipa/changeset/ba4df6449aaa0843ab43a1a2b3cb1df8bb022c24
https://fedorahosted.org/freeipa/changeset/1fc128b05fd13a3f400346cc6d2e7fb5f66875ac
https://fedorahosted.org/freeipa/changeset/500327b7754e032738ab88ae19fad287f2d8cdab
https://fedorahosted.org/freeipa/changeset/2de43e7aca7d4d4873ad3e5053ad75311e81dc68
https://fedorahosted.org/freeipa/changeset/e40d6a2a53a931b4d2be3e45c84da99950e60a84
https://fedorahosted.org/freeipa/changeset/0b68899779e4500d231e974f11e428f8a3577538
https://fedorahosted.org/freeipa/changeset/928a4aa6f281df55e0f655d5cbf5a327794507b6
https://fedorahosted.org/freeipa/changeset/606cac1c9e85633f54b1cc1c9fc1351e6d1a545f
https://fedorahosted.org/freeipa/changeset/835923750bff4f26d9b90df9870a961d16728488
https://fedorahosted.org/freeipa/changeset/bc2e3386e7fa30211a46c0c2284d901cc2509147
https://fedorahosted.org/freeipa/changeset/37578cfc2bbec99d75b19c94c337c406bf6a6ef7
https://fedorahosted.org/freeipa/changeset/1e6366bc9f10de66de84b9506341f021fb3650d9
https://fedorahosted.org/freeipa/changeset/15f282cf2c4a5315aa3e259bd923718685d88245
https://fedorahosted.org/freeipa/changeset/81bf72dc350b9c7daab669aaa796e96aee6ecbb8
https://fedorahosted.org/freeipa/changeset/32599987fdc998e104846e8a176f70399cca2af2
https://fedorahosted.org/freeipa/changeset/4286f3885b173da9ceeb2d13d66f90336b9ef094
https://fedorahosted.org/freeipa/changeset/6181844c0ce62b8d7d35554032346396b20ad3c0
https://fedorahosted.org/freeipa/changeset/3129b874a2c222ff207f1302e5d85ae12df2eac9
https://fedorahosted.org/freeipa/changeset/4e97a0171a862e20089863e4bf0ec88d0ba98a53
https://fedorahosted.org/freeipa/changeset/73fc15556d28706b0b9a10480fee8d56b2be9ab7
https://fedorahosted.org/freeipa/changeset/7cd3b1bfa76c846b7ffec18e380b71a6617d97ec
https://fedorahosted.org/freeipa/changeset/8c742b1539591b49474fe8ec871e1b523e9898bd
https://fedorahosted.org/freeipa/changeset/a641e279ff76e09f59c4d5fef1dc1f9355dbacf7
https://fedorahosted.org/freeipa/changeset/be0c1afa74cdf9a6e7640cd4110519e61250ae93
https://fedorahosted.org/freeipa/changeset/9fd1981ae8abf720f5234b6049c9beabbb1f2211
https://fedorahosted.org/freeipa/changeset/a929ac333833a5cbf503d1fcbdee150658d933a4
https://fedorahosted.org/freeipa/changeset/043c262ce48a0d667e914c315e21e6e1b3862202
https://fedorahosted.org/freeipa/changeset/269ca6c4547fc017bb3a88e994ca770047122b3e
https://fedorahosted.org/freeipa/changeset/08a446a6bc516936497c1e0f278a699148f6330c
https://fedorahosted.org/freeipa/changeset/a8fdb8de8248fe24f382e44b05293405b0b309ac
https://fedorahosted.org/freeipa/changeset/225fae841882832668c0842479ab11c89dfcd1a5
https://fedorahosted.org/freeipa/changeset/714699a81fa377e6033cbc7564f0f0fd10cd9f1a
https://fedorahosted.org/freeipa/changeset/09423acb6574a3773d7783f9ddec022bed3539c8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ack Pull Request approved, can be merged pushed Pull Request has already been pushed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants