Adding spec file, cleanup

Signed-off-by: Eric Jackson <ejackson@suse.com>

.gitignore

@@ -0,0 +1,5 @@
+*.py[co]
+*.sw[pon]
+*~
+*#
+

Makefile

@@ -0,0 +1,14 @@
+
+
+all: rpm
+
+rpm: tarball
+	rpmbuild -bb deepsea.spec
+
+tarball: tests
+	VERSION=`awk '/^Version/ {print $$2}' deepsea.spec`; \
+	git archive --prefix deepsea/ -o ~/rpmbuild/SOURCES/deepsea-$$VERSION.tar.gz HEAD
+
+tests:
+	@echo "Need to write tests :)"
+

README.md

@@ -1,38 +1,32 @@
-# Pillar Prototype
-These salt files are intended to allow the creation and management of multiple ceph clusters with a single salt master.
+# DeepSea
+A collection of Salt files for deploying, managing and automating Ceph.
 
-The [diagram](pillar-proposal.png) should explain the intended flow for the orchestration runners and related salt states.
+These goal is to manage multiple ceph clusters with a single salt master.  At this time, only one Ceph cluster is managed.
+
+The [diagram](deepsea.png) should explain the intended flow for the orchestration runners and related salt states.
 
 ## Status
-This is alpha at best.  It works, but runners need refactoring, commenting, correct returns and unit tests.  Several decisions still remain and a few files have notes alluding to missing functionality.  Only a single cluster named `ceph` has been tested.
+The stages 0-4 which covers optional provisioning to creating gateways for Ceph services are working.  Role removal is in progress.
 
-## Usage
 
+## Usage
+Prepare Salt
 - Install salt-master on one host
-- Install salt-minion on all the other hosts you want to be ceph nodes, and *also* on the master host
-- Clone this repository and copy the contents i.e. /etc and /srv in their respective directories
-- Be sure to have python-pip and git-core installed
-- Install https://github.com/oms4suse/python-ceph-cfg on all hosts. If salt-minion is installed on all hosts & keys 
-accepted, this can be done by doing salt "*" pip.install git+https://github.com/oms4suse/python-ceph-cfg 
-- Copy etc/salt/master.d/reactor.conf-example to /etc/salt/master.d/reactor.conf on the master, or set "startup_states: 'highstate'" in all the minion configs.
-- Accept all the salt keys on the master, make sure salt works (`salt '*' test.ping`)
-- Create an SLS file in /srv/pillar/ceph/cluster/ for each host, indicating what cluster the host belongs to.
-  - The easy way to do this is to just run `salt-run bootstrap.all`, which will make all salt minions ceph cluster nodes. Note that `/srv/modules` needs to be added in `extension_modules` in the salt-master configuration. 
-  - If you don't want to use all your minions for the cluster, use the
-    bootstrap.selection runner. The runner expects a compound target selector as
-    parameter (regarding compound targets refer to
-    https://docs.saltstack.com/en/latest/topics/targeting/compound.html).
-  - Alternately, you can create the files manually.  For each one, the file should contain just the line "cluster: ceph".  For the admin node (or salt master), use "cluster: unassigned".
-- If necessary edit the sntp invocation in /srv/salt/ceph/sync/init.sls to point to a real NTP server (it tries to use the host named 'salt' by default).
-- Run `salt-run state.orchestrate ceph.stage.0`
-- Run `salt-run state.orchestrate ceph.stage.1`
-- Verify /srv/pillar/ceph/stack/ files are populated as mentioned in the [diagram](pillar-proposal.png).
-- Edit /srv/pillar/ceph/stack/{ceph.cfg,core.yml,defaults/ceph-custom.yml} to pick up the desired layout and storage files (you need to uncomment whatever sections make sense in ceph.cfg), and set network interfaces if necessary.
-- Be sure to add you master_node in startup_states: pillar/ceph/stack/default/global.yml
-- Copy either /srv/pillar/ceph/proposals/generic-hostnames.example or /srv/pillar/ceph/proposals/rolebased-hostnames.example to /srv/pillar/ceph/proposals/policy.cfg
-- Adapt your policy.cfg to match your cluster topology.
-- Just in case, run `salt '*' saltutil.refresh_pillar`.
-- Run `salt-run state.orchestrate ceph.stage.2`
-- Verify /srv/pillar/ceph/stack/cluster/ceph.conf.yml looks sane (it should havecluster_network, public_network, mon_host and mon_initial_members filled in).
-- Run `salt-run state.orchestrate ceph.stage.3`
-- You should now have a running cluster.
+- Install salt-minion on all hosts including the master.
+- Accept keys (e.g. salt-key -A -y)
+
+Install DeepSea
+- Install rpm (link TBD)
+
+Configure
+- Edit /etc/salt/master.d/logging.conf
+- Edit /srv/pillar/ceph/master_minion.sls
+
+Steps
+- Run `salt-run state.orch ceph.stage.0 or salt-run state.orch ceph.stage.prep`
+- Run `salt-run state.orch ceph.stage.1 or salt-run state.orch ceph.stage.discovery`
+- Create /srv/pillar/ceph/proposals/policy.cfg.  Examples are in ...
+- Run `salt-run state.orch ceph.stage.2 or salt-run state.orch ceph.stage.configure`
+- Run `salt-run state.orch ceph.stage.3 or salt-run state.orch ceph.stage.deploy`
+- Run `salt-run state.orch ceph.stage.4 or salt-run state.orch ceph.stage.services`
+

deepsea.spec

@@ -0,0 +1,298 @@
+#
+# spec file for package deepsea
+#
+# Copyright (c) 2016 SUSE LINUX GmbH, Nuernberg, Germany.
+#
+# All modifications and additions to the file contributed by third parties
+# remain the property of their copyright owners, unless otherwise agreed
+# upon. The license for this file, and modifications and additions to the
+# file, is the same license as for the pristine package itself (unless the
+# license for the pristine package is not an Open Source License, in which
+# case the license is the MIT License). An "Open Source License" is a
+# license that conforms to the Open Source Definition (Version 1.9)
+# published by the Open Source Initiative.
+
+# Please submit bugfixes or comments via http://bugs.opensuse.org/
+#
+
+
+# See also http://en.opensuse.org/openSUSE:Shared_library_packaging_policy
+
+Name:           deepsea
+Version:        0.1
+Release:        0
+Summary:        Salt solution for deploying and managing Ceph
+
+License:        GPL-3.0
+Group:          System/Libraries
+Url:            http://bugs.opensuse.org
+Source0:        deepsea-%{version}.tar.gz
+
+#BuildRequires:  
+#Requires:       
+BuildRoot:     %{_tmppath}/%{name}-%{version}-build
+
+%description
+
+
+#%package        devel
+#Summary:        Development files for %{name}
+#Group:          Development/Libraries
+#Requires:       %{name} = %{version}
+#
+#%description    devel
+#The %{name}-devel package contains libraries and header files for
+#developing applications that use %{name}.
+
+%prep
+
+%build
+%__tar xvzf %{SOURCE0}
+
+%install
+install -d 755 %{buildroot}%{_mandir}/man8
+
+cd %{name}
+
+install -d 755 %{buildroot}/etc/salt/master.d
+install -m 644 etc/salt/master.d/modules.conf %{buildroot}/etc/salt/master.d/
+install -m 644 etc/salt/master.d/reactor.conf %{buildroot}/etc/salt/master.d/
+
+install -d 755 %{buildroot}%{_docdir}/%{name}
+install -m 644 LICENSE %{buildroot}%{_docdir}/%{name}
+install -m 644 README.md %{buildroot}%{_docdir}/%{name}
+
+install -d 755 %{buildroot}%{_docdir}/%{name}/examples
+install -m 644 doc/examples/policy.cfg-rolebased %{buildroot}%{_docdir}/%{name}/examples
+install -m 644 doc/examples/policy.cfg-generic %{buildroot}%{_docdir}/%{name}/examples
+install -m 644 doc/examples/policy.cfg-regex %{buildroot}%{_docdir}/%{name}/examples
+
+# Included in Salt 2016.3
+install -d 755 %{buildroot}/srv/modules/pillar
+install -m 644 srv/modules/pillar/stack.py %{buildroot}/srv/modules/pillar
+
+%define _runners srv/modules/runners
+install -d 755 %{buildroot}/%{_runners}
+install -m 644 %{_runners}/__init__.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/bootstrap.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/check.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/configure.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/filequeue.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/minions.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/populate.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/push.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/ready.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/select.py %{buildroot}/%{_runners}
+install -m 644 %{_runners}/validate.py %{buildroot}/%{_runners}
+
+%define _pillar srv/pillar
+install -d 755 %{buildroot}/%{_pillar}/ceph
+install -m 644 %{_pillar}/ceph/cluster/README %{buildroot}/%{_pillar}/ceph
+install -m 644 %{_pillar}/ceph/init.sls %{buildroot}/%{_pillar}/ceph
+install -m 644 %{_pillar}/ceph/master_minion.sls %{buildroot}/%{_pillar}/ceph
+install -m 644 %{_pillar}/ceph/stack/stack.cfg %{buildroot}/%{_pillar}/ceph
+
+install -m 644 %{_pillar}/top.sls %{buildroot}/%{_pillar}
+
+install -d 755 %{buildroot}/srv/salt/_modules
+install -m 644 srv/salt/_modules/cephdisks.py %{buildroot}/srv/salt/_modules
+install -m 644 srv/salt/_modules/freedisks.py %{buildroot}/srv/salt/_modules
+install -m 644 srv/salt/_modules/retry.py %{buildroot}/srv/salt/_modules
+install -m 644 srv/salt/_modules/wait.py %{buildroot}/srv/salt/_modules
+install -m 644 srv/salt/_modules/zypper_locks.py %{buildroot}/srv/salt/_modules
+
+%define _saltceph srv/salt/ceph
+install -d 755 %{buildroot}/%{_saltceph}/admin
+install -m 644 %{_saltceph}/admin/authtool.sls %{buildroot}/%{_saltceph}/admin
+install -m 644 %{_saltceph}/admin/default.sls %{buildroot}/%{_saltceph}/admin
+install -m 644 %{_saltceph}/admin/files/keyring.j2 %{buildroot}/%{_saltceph}/admin
+install -m 644 %{_saltceph}/admin/init.sls %{buildroot}/%{_saltceph}/admin
+install -m 644 %{_saltceph}/admin/pcc.sls %{buildroot}/%{_saltceph}/admin
+
+install -d 755 %{buildroot}/%{_saltceph}/configuration
+install -m 644 %{_saltceph}/configuration/default.sls %{buildroot}/%{_saltceph}/configuration
+install -m 644 %{_saltceph}/configuration/files/ceph.conf.j2 %{buildroot}/%{_saltceph}/configuration
+install -m 644 %{_saltceph}/configuration/init.sls %{buildroot}/%{_saltceph}/configuration
+
+
+install -d 755 %{buildroot}/%{_saltceph}/events
+install -m 644 %{_saltceph}/events/begin_prep.sls %{buildroot}/%{_saltceph}/events
+install -m 644 %{_saltceph}/events/complete_prep.sls %{buildroot}/%{_saltceph}/events
+
+#install -m 644 %{_saltceph}/files/multipath.conf
+#install -m 644 %{_saltceph}/initiator/iscsiadm-salt.sls
+#install -m 644 %{_saltceph}/initiator/iscsiadm.sls
+#install -m 644 %{_saltceph}/initiator/multipathd.sls
+
+install -d 755 %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/authtool.sls %{buildroot}/%{_saltceph}/iscsi
+
+# Hardcoded secret... need to fix
+install -m 644 %{_saltceph}/iscsi/files/ceph.client.iscsi.keyring %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/files/sysconfig.lrbd %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/import-salt.sls %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/import.sls %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/keyring.sls %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/lrbd-salt.sls %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/lrbd.sls %{buildroot}/%{_saltceph}/iscsi
+install -m 644 %{_saltceph}/iscsi/sysconfig.sls %{buildroot}/%{_saltceph}/iscsi
+
+
+install -d 755 %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/auth.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/bootstrap-auth.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/default.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/files/bootstrap.j2 %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/files/keyring.j2 %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/init.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/keyring.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/pcc.sls %{buildroot}/%{_saltceph}/mds
+install -m 644 %{_saltceph}/mds/pools.sls %{buildroot}/%{_saltceph}/mds
+
+
+install -d 755 %{buildroot}/%{_saltceph}/mine_functions
+install -m 644 %{_saltceph}/mine_functions/files/mine_functions.conf %{buildroot}/%{_saltceph}/mine_functions
+install -m 644 %{_saltceph}/mine_functions/init.sls %{buildroot}/%{_saltceph}/mine_functions
+
+install -d 755 %{buildroot}/%{_saltceph}/mon
+install -m 644 %{_saltceph}/mon/default.sls %{buildroot}/%{_saltceph}/mon
+install -m 644 %{_saltceph}/mon/files/keyring.j2 %{buildroot}/%{_saltceph}/mon
+install -m 644 %{_saltceph}/mon/init.sls %{buildroot}/%{_saltceph}/mon
+install -m 644 %{_saltceph}/mon/pcc.sls %{buildroot}/%{_saltceph}/mon
+install -m 644 %{_saltceph}/mon/start.sls %{buildroot}/%{_saltceph}/mon
+
+install -d 755 %{buildroot}/%{_saltceph}/openattic
+install -m 644 %{_saltceph}/openattic/authtool.sls %{buildroot}/%{_saltceph}/openattic
+install -m 644 %{_saltceph}/openattic/files/ceph.client.openattic.keyring %{buildroot}/%{_saltceph}/openattic
+install -m 644 %{_saltceph}/openattic/init.sls %{buildroot}/%{_saltceph}/openattic
+install -m 644 %{_saltceph}/openattic/keyring.sls %{buildroot}/%{_saltceph}/openattic
+install -m 644 %{_saltceph}/openattic/openattic.sls %{buildroot}/%{_saltceph}/openattic
+
+install -d 755 %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/auth.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/custom.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/default.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/files/keyring.j2 %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/init.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/keyring.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/partition.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/pcc-auth.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/pcc-custom.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/pcc-keyring.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/pcc.sls %{buildroot}/%{_saltceph}/osd
+install -m 644 %{_saltceph}/osd/scheduler.sls %{buildroot}/%{_saltceph}/osd
+
+install -d 755 %{buildroot}/%{_saltceph}/packages
+install -m 644 %{_saltceph}/packages/custom-salt.sls %{buildroot}/%{_saltceph}/packages
+install -m 644 %{_saltceph}/packages/default.sls %{buildroot}/%{_saltceph}/packages
+install -m 644 %{_saltceph}/packages/init.sls %{buildroot}/%{_saltceph}/packages
+
+install -d 755 %{buildroot}/%{_saltceph}/pool
+install -m 644 %{_saltceph}/pool/custom.sls %{buildroot}/%{_saltceph}/pool
+install -m 644 %{_saltceph}/pool/default.sls %{buildroot}/%{_saltceph}/pool
+install -m 644 %{_saltceph}/pool/init.sls %{buildroot}/%{_saltceph}/pool
+
+install -d 755 %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/all_stages.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/discovery.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/highstate.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/initialize.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/master.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/prep_begin.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/prep_complete.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/prep_minion.sls %{buildroot}/%{_saltceph}/reactor
+install -m 644 %{_saltceph}/reactor/readycheck %{buildroot}/%{_saltceph}/reactor
+
+install -d 755 %{buildroot}/%{_saltceph}/refresh
+install -m 644 %{_saltceph}/refresh/init.sls %{buildroot}/%{_saltceph}/refresh
+
+install -d 755 %{buildroot}/%{_saltceph}/repo
+install -m 644 %{_saltceph}/repo/custom.sls %{buildroot}/%{_saltceph}/repo
+install -m 644 %{_saltceph}/repo/default.sls %{buildroot}/%{_saltceph}/repo
+install -m 644 %{_saltceph}/repo/init.sls %{buildroot}/%{_saltceph}/repo
+
+install -d 755 %{buildroot}/%{_saltceph}/rgw
+install -m 644 %{_saltceph}/rgw/default.sls %{buildroot}/%{_saltceph}/rgw
+install -m 644 %{_saltceph}/rgw/init.sls %{buildroot}/%{_saltceph}/rgw
+install -m 644 %{_saltceph}/rgw/pools.sls %{buildroot}/%{_saltceph}/rgw
+
+install -d 755 %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/all.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/benchmark.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/cephfs.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/configure.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/configure/default.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/deploy.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/discovery.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/discovery/custom.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/discovery/default.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/iscsi.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/prep.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/prep/default.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/prep_minions.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/removal.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/rgw.sls %{buildroot}/%{_saltceph}/stage
+install -m 644 %{_saltceph}/stage/services.sls %{buildroot}/%{_saltceph}/stage
+
+install -d 755 %{buildroot}/%{_saltceph}/sync
+install -m 644 %{_saltceph}/sync/init.sls %{buildroot}/%{_saltceph}/sync
+
+install -d 755 %{buildroot}/%{_saltceph}/time
+install -m 644 %{_saltceph}/time/custom-salt.sls %{buildroot}/%{_saltceph}/time
+install -m 644 %{_saltceph}/time/custom.sls %{buildroot}/%{_saltceph}/time
+install -m 644 %{_saltceph}/time/init.sls %{buildroot}/%{_saltceph}/time
+install -m 644 %{_saltceph}/time/ntp.sls %{buildroot}/%{_saltceph}/time
+
+install -d 755 %{buildroot}/%{_saltceph}/updates
+install -m 644 %{_saltceph}/updates/default.sls %{buildroot}/%{_saltceph}/updates
+install -m 644 %{_saltceph}/updates/init.sls %{buildroot}/%{_saltceph}/updates
+install -m 644 %{_saltceph}/updates/restart.sls %{buildroot}/%{_saltceph}/updates
+
+install -m 644 srv/salt/top.sls %{buildroot}/srv/salt
+
+%post 
+
+%postun 
+
+%files
+%defattr(-,salt,salt,-)
+/etc/salt/master.d/*
+/srv/modules/pillar/*
+/%{_runners}/*
+/%{_pillar}/top.sls
+/%{_pillar}/ceph/*
+/srv/salt/top.sls
+/srv/salt/_modules/*
+/%{_saltceph}/admin/*
+/%{_saltceph}/configuration/*
+/%{_saltceph}/events/*
+/%{_saltceph}/iscsi/*
+/%{_saltceph}/mds/*
+/%{_saltceph}/mine_functions/*
+/%{_saltceph}/mon/*
+/%{_saltceph}/openattic/*
+/%{_saltceph}/osd/*
+/%{_saltceph}/packages/*
+/%{_saltceph}/pool/*
+/%{_saltceph}/reactor/*
+/%{_saltceph}/refresh/*
+/%{_saltceph}/repo/*
+/%{_saltceph}/rgw/*
+/%{_saltceph}/stage/*
+/%{_saltceph}/sync/*
+/%{_saltceph}/time/*
+/%{_saltceph}/updates/*
+%doc
+%dir %attr(-, root, root) %{_docdir}/%{name}
+%{_docdir}/%{name}/*
+%{_docdir}/%{name}/examples/*
+
+#%files devel
+#%defattr(-,root,root,-)
+#%doc
+#%{_includedir}/*
+#%{_libdir}/*.so
+
+%changelog
+* Thu Sep  8 2016 Eric Jackson
+-