Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: do not encourage readers to run multiple clusters on same HW #34188

Merged
merged 1 commit into from Mar 26, 2020
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
74 changes: 9 additions & 65 deletions doc/rados/configuration/common.rst
Expand Up @@ -183,76 +183,20 @@ Example ceph.conf



Running Multiple Clusters
=========================
Running Multiple Clusters (DEPRECATED)
======================================

With Ceph, you can run multiple Ceph Storage Clusters on the same hardware.
Running multiple clusters provides a higher level of isolation compared to
using different pools on the same cluster with different CRUSH rules. A
separate cluster will have separate monitor, OSD and metadata server processes.
When running Ceph with default settings, the default cluster name is ``ceph``,
which means you would save your Ceph configuration file with the file name
``ceph.conf`` in the ``/etc/ceph`` default directory.
Some Ceph CLI commands take a ``-c`` (cluster name) option. This option is
present purely for backward compatibility. You should not attempt to deploy
or run multiple clusters on the same hardware, and it is recommended to always
leave the cluster name as the default ("ceph").
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you need to allow multiple clusters to exist on the same host, please use :ref:cephadm, which uses containers to fully isolate each cluster.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, @liewegas - done!


See `Create a Cluster`_ for details.

.. _Create a Cluster: ../../deployment/ceph-deploy-new

When you run multiple clusters, you must name your cluster and save the Ceph
configuration file with the name of the cluster. For example, a cluster named
``openstack`` will have a Ceph configuration file with the file name
``openstack.conf`` in the ``/etc/ceph`` default directory.

.. important:: Cluster names must consist of letters a-z and digits 0-9 only.

Separate clusters imply separate data disks and journals, which are not shared
between clusters. Referring to `Metavariables`_, the ``$cluster`` metavariable
evaluates to the cluster name (i.e., ``openstack`` in the foregoing example).
Various settings use the ``$cluster`` metavariable, including:

.. _Metavariables: ../ceph-conf#Metavariables

- ``keyring``
- ``admin socket``
- ``log file``
- ``pid file``
- ``mon data``
- ``mon cluster log file``
- ``osd data``
- ``osd journal``
- ``mds data``
- ``rgw data``

See `General Settings`_, `OSD Settings`_, `Monitor Settings`_, `MDS Settings`_,
`RGW Settings`_ and `Log Settings`_ for relevant path defaults that use the
``$cluster`` metavariable.

.. _General Settings: ../general-config-ref
.. _OSD Settings: ../osd-config-ref
.. _Monitor Settings: ../mon-config-ref
.. _MDS Settings: ../../../cephfs/mds-config-ref
.. _RGW Settings: ../../../radosgw/config-ref/
.. _Log Settings: ../../troubleshooting/log-and-debug


When creating default directories or files, you should use the cluster
name at the appropriate places in the path. For example::

sudo mkdir /var/lib/ceph/osd/openstack-0
sudo mkdir /var/lib/ceph/mon/openstack-a

.. important:: When running monitors on the same host, you should use
different ports. By default, monitors use port 6789. If you already
have monitors using port 6789, use a different port for your other cluster(s).

To invoke a cluster other than the default ``ceph`` cluster, use the
``-c {filename}.conf`` option with the ``ceph`` command. For example::

ceph -c {cluster-name}.conf health
ceph -c openstack.conf health
If you need to allow multiple clusters to exist on the same host, please use
`cephadm`_, which uses containers to fully isolate each cluster.


.. _Hardware Recommendations: ../../../start/hardware-recommendations
.. _Network Configuration Reference: ../network-config-ref
.. _OSD Config Reference: ../osd-config-ref
.. _Configuring Monitor/OSD Interaction: ../mon-osd-interaction
.. _cephadm: ../../cephadm