ceph · liewegas · Jun 7, 2021 · May 20, 2021 · May 19, 2021 · May 25, 2021
diff --git a/PendingReleaseNotes b/PendingReleaseNotes
@@ -19,6 +19,20 @@
   support an NFS export of both ``rgw`` and ``cephfs`` from a single
   NFS cluster instance.
 
+* The ``nfs cluster update`` command has been removed.  You can modify
+  the placement of an existing NFS service (and/or its associated
+  ingress service) using ``orch ls --export`` and ``orch apply -i
+  ...``.
+
+* The ``orch apply nfs`` command no longer requires a pool or
+  namespace argument. We strongly encourage users to use the defaults
+  so that the ``nfs cluster ls`` and related commands will work
+  properly.
+
+* The ``nfs cluster delete`` and ``nfs export delete`` commands are
+  deprecated and will be removed in a future release.  Please use
+  ``nfs cluster rm`` and ``nfs export rm`` instead.
+
 * mgr-pg_autoscaler: Autoscaler will now start out by scaling each
   pool to have a full complements of pgs from the start and will only
   decrease it when other pools need more pgs due to increased usage.

@@ -126,8 +126,8 @@ Adoption process
 
    This will perform a ``cephadm check-host`` on each host before adding it;
    this check ensures that the host is functioning properly. The IP address
-   argument is required only if DNS does not allow you to connect to each host
-   by its short name.
+   argument is recommended; if not provided, then the host name will be resolved
+   via DNS.
 
 #. Verify that the adopted monitor and manager daemons are visible:
 

@@ -37,14 +37,18 @@ To add each new host to the cluster, perform two steps:
 
    .. prompt:: bash #
 
-     ceph orch host add *newhost* [*<label1> ...*]
+     ceph orch host add *<newhost>* [*<ip>*] [*<label1> ...*]
 
    For example:
 
    .. prompt:: bash #
 
-     ceph orch host add host2
-     ceph orch host add host3
+     ceph orch host add host2 10.10.0.102
+     ceph orch host add host3 10.10.0.103
+
+   It is best to explicitly provide the host IP address.  If an IP is
+   not provided, then the host name will be immediately resolved via
+   DNS and that IP will be used.
 
    One or more labels can also be included to immediately label the
    new host.  For example, by default the ``_admin`` label will make
@@ -53,7 +57,7 @@ To add each new host to the cluster, perform two steps:
 
    .. prompt:: bash #
 
-       ceph orch host add host4 _admin
+       ceph orch host add host4 10.10.0.104 --labels _admin
 
 .. _cephadm-removing-hosts:
 
@@ -174,21 +178,21 @@ Many hosts can be added at once using
 
     ---
     service_type: host
-    addr: node-00
     hostname: node-00
+    addr: 192.168.0.10
     labels:
     - example1
     - example2
     ---
     service_type: host
-    addr: node-01
     hostname: node-01
+    addr: 192.168.0.11
     labels:
     - grafana
     ---
     service_type: host
-    addr: node-02
     hostname: node-02
+    addr: 192.168.0.12
 
 This can be combined with service specifications (below) to create a cluster spec
 file to deploy a whole cluster in one command.  see ``cephadm bootstrap --apply-spec``
@@ -286,23 +290,12 @@ There are two ways to customize this configuration for your environment:
 Fully qualified domain names vs bare host names
 ===============================================
 
-cephadm has very minimal requirements when it comes to resolving host
-names etc. When cephadm initiates an ssh connection to a remote host,
-the host name  can be resolved in four different ways:
-
--  a custom ssh config resolving the name to an IP
--  via explicitly providing an IP address to cephadm: ``ceph orch host add <hostname> <IP>``
--  automatic name resolution via DNS.
-
-Ceph itself uses the command ``hostname`` to determine the name of the
-current host.
-
 .. note::
 
   cephadm demands that the name of the host given via ``ceph orch host add`` 
   equals the output of ``hostname`` on remote hosts.
 
-Otherwise cephadm can't be sure, the host names returned by
+Otherwise cephadm can't be sure that names returned by
 ``ceph * metadata`` match the hosts known to cephadm. This might result
 in a :ref:`cephadm-stray-host` warning.
 

@@ -153,6 +153,17 @@ For example
 
      ceph config set mgr mgr/cephadm/container_image_prometheus prom/prometheus:v1.4.1
 
+If there were already running monitoring stack daemon(s) of the type whose
+image you've changed, you must redeploy the daemon(s) in order to have them
+actually use the new image.
+
+For example, if you had changed the prometheus image
+
+.. prompt:: bash #
+
+     ceph orch redeploy prometheus
+
+
 .. note::
 
      By setting a custom image, the default value will be overridden (but not

@@ -1,32 +1,35 @@
+.. _deploy-cephadm-nfs-ganesha:
+
 ===========
 NFS Service
 ===========
 
 .. note:: Only the NFSv4 protocol is supported.
 
-.. _deploy-cephadm-nfs-ganesha:
+The simplest way to manage NFS is via the ``ceph nfs cluster ...``
+commands; see :ref:`cephfs-nfs`.  This document covers how to manage the
+cephadm services directly, which should only be necessary for unusual NFS
+configurations.
 
 Deploying NFS ganesha
 =====================
 
-Cephadm deploys NFS Ganesha using a pre-defined RADOS *pool*
-and optional *namespace*.
+Cephadm deploys NFS Ganesha daemon (or set of daemons).  The configuration for
+NFS is stored in the ``nfs-ganesha`` pool and exports are managed via the
+``ceph nfs export ...`` commands and via the dashboard.
 
 To deploy a NFS Ganesha gateway, run the following command:
 
 .. prompt:: bash #
 
-    ceph orch apply nfs *<svc_id>* *<pool>* *<namespace>* --placement="*<num-daemons>* [*<host1>* ...]"
+    ceph orch apply nfs *<svc_id>* [--port *<port>*] [--placement ...]
 
-For example, to deploy NFS with a service id of *foo* that will use the RADOS
-pool *nfs-ganesha* and the namespace *nfs-ns*, run this command:
+For example, to deploy NFS with a service id of *foo* on the default
+port 2049 with the default placement of a single daemon:
 
 .. prompt:: bash #
 
-   ceph orch apply nfs foo nfs-ganesha nfs-ns
-
-.. note::
-   If the *nfs-ganesha* pool doesn't exist, create it.
+   ceph orch apply nfs foo
 
 See :ref:`orchestrator-cli-placement-spec` for the details of the placement
 specification.
@@ -36,9 +39,6 @@ Service Specification
 
 Alternatively, an NFS service can be applied using a YAML specification. 
 
-A service of type ``nfs`` requires a pool name and can contain
-an optional namespace:
-
 .. code-block:: yaml
 
     service_type: nfs
@@ -48,15 +48,66 @@ an optional namespace:
         - host1
         - host2
     spec:
-      pool: mypool
-      namespace: mynamespace
+      port: 12345
 
-In this example, ``pool`` is a RADOS pool where NFS client recovery data is
-stored and ``namespace`` is a RADOS namespace where NFS client recovery data
-is stored.
+In this example, we run the server on the non-default ``port`` of
+12345 (instead of the default 2049) on ``host1`` and ``host2``.
 
 The specification can then be applied by running the following command:
 
 .. prompt:: bash #
 
    ceph orch apply -i nfs.yaml
+
+
+High-availability NFS
+=====================
+
+Deploying an *ingress* service for an existing *nfs* service will provide:
+
+* a stable, virtual IP that can be used to access the NFS server
+* fail-over between hosts if there is a host failure
+* load distribution across multiple NFS gateways (although this is rarely necessary)
+
+Ingress for NFS can be deployed for an existing NFS service
+(``nfs.mynfs`` in this example) with the following specification:
+
+.. code-block:: yaml
+
+    service_type: ingress
+    service_id: nfs.mynfs
+    placement:
+      count: 2
+    spec:
+      backend_service: nfs.mynfs
+      frontend_port: 2049
+      monitor_port: 9000
+      virtual_ip: 10.0.0.123/24
+
+A few notes:
+
+  * The *virtual_ip* must include a CIDR prefix length, as in the
+    example above.  The virtual IP will normally be configured on the
+    first identified network interface that has an existing IP in the
+    same subnet.  You can also specify a *virtual_interface_networks*
+    property to match against IPs in other networks; see
+    :ref:`ingress-virtual-ip` for more information.
+  * The *monitor_port* is used to access the haproxy load status
+    page.  The user is ``admin`` by default, but can be modified by
+    via an *admin* property in the spec.  If a password is not
+    specified via a *password* property in the spec, the auto-generated password
+    can be found with:
+
+    .. prompt:: bash #
+
+	ceph config-key get mgr/cephadm/ingress.*{svc_id}*/monitor_password
+
+    For example:
+
+    .. prompt:: bash #
+
+	ceph config-key get mgr/cephadm/ingress.nfs.myfoo/monitor_password
+
+  * The backend service (``nfs.mynfs`` in this example) should include
+    a *port* property that is not 2049 to avoid conflicting with the
+    ingress service, which could be placed on the same host(s).
@@ -112,18 +112,21 @@ elected as master, and the virtual IP will be moved to that node.
 The active haproxy acts like a load balancer, distributing all RGW requests
 between all the RGW daemons available.
 
-**Prerequisites:**
+Prerequisites
+-------------
 
 * An existing RGW service, without SSL.  (If you want SSL service, the certificate
   should be configured on the ingress service, not the RGW service.)
 
-**Deploy of the high availability service for RGW**
+Deploying
+---------
 
 Use the command::
 
     ceph orch apply -i <ingress_spec_file>
 
-**Service specification file:**
+Service specification
+---------------------
 
 It is a yaml format file with the following properties:
 
@@ -171,7 +174,10 @@ where the properties of this service specification are:
     SSL certificate, if SSL is to be enabled. This must contain the both the certificate and
     private key blocks in .pem format.
 
-**Selecting ethernet interfaces for the virtual IP:**
+.. _ingress-virtual-ip:
+
+Selecting ethernet interfaces for the virtual IP
+------------------------------------------------
 
 You cannot simply provide the name of the network interface on which
 to configure the virtual IP because interface names tend to vary
@@ -204,7 +210,8 @@ configuring a "dummy" IP address is an unroutable network on the correct interfa
 and reference that dummy network in the networks list (see above).
 
 
-**Useful hints for ingress:**
+Useful hints for ingress
+------------------------
 
-* Good to have at least 3 RGW daemons
-* Use at least 3 hosts for the ingress
+* It is good to have at least 3 RGW daemons.
+* We recommend at least 3 hosts for the ingress service.