pacific: mgr/nfs: allow dynamic update of cephfs nfs export #45543
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, the NFSRados object accepted the "Module" as the first argument but only used the rados attribute (type rados.Rados). It's better to limit the scope of types when reasonably possible so we can see what the true dependencies are. So we restrict NFSRados to accepting a rados.Rados as the argument. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit d94b638)
This was previously a staticmethod. This static method was only used by NFSRados object. Staticmethods are nearly always better implemented as functions, which is done so here. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit c51a675)
The naming of rados objects used to store the nfs config was spread all over the code, including inline f-strings, not-static methods, etc. This change unifies the naming by putting constant string prefixes and name generating functions into the utils.py file. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit 8826614)
This function was not using self and thus has no need to be a method. While we're at it, rename it to normalize_path because that's what it is doing. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit f91dd1b)
Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit ffa95fb)
Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit 56323a2)
This change allows the `ceph nfs export ...` commands to function without the entire mgr/nfs subsystem requiring orchestration to be enabled. When there's no orchestration available, the code falls back to examining the namespaces in the ".nfs" rados pool to determine what cluster_id values are valid. This change does not add support for creating the rados objects and namespace needed to manage a nfs cluster. As discussed with the orchestration group on 2022-01-22, rook does not need the mgr module to establish the namespace. So, for now, we'll defer the work needed to create the namespace/objects when orchestration is disabled. Fixes: https://tracker.ceph.com/issues/54043 Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit 8c711af)
A recent change in the mgr/nfs module should enable the functioning of export management commands/API calls as long as the rados namespaces and objects have been already established. Document this fact, noting that now only the `ceph nfs cluster ...` calls *require* an orchestration module. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit b5b3e0b)
The changes to the nfs module in 8c711af are working but when I began writing more test automation I found a few more places in the export-configuration code path relying on the orchestration module only. This change generalizes the logic to source nfs clusters from orchestration when it's enabled but from the .nfs pool when orchestration is disabled. It then uses that call when loading the exports cache on the ExportMgr object. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit 4d09660)
The `exports` property of the ExportMgr exists to cache the exports configuration found in the .nfs namespace. Using that property within the property method is probably not intentional and is probably only working due to the lucky construction of the _exports dict immediately after the None check so that the _exports dict is returned (and is a mutable type). Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit daa455c)
Remove the extra check of the cluster id from _apply method. As _apply is a "private" method that should be only called from other private methods that have already validated the cluster_id. It also removes a dependency on the orch-requiring func available_clusters. Signed-off-by: John Mulligan <jmulligan@redhat.com> (cherry picked from commit dd5a47f)
mgr/nfs module's apply_export() method is used to update an existing CephFS NFS export. The method always restarted the ganesha service ( ganesha server cluster) after updating the export object and notifying the ganesha servers to reload their exports. The restart temporarily affected the clients connections of all the exports served by the ganesha servers. It is not always necessary to restart the ganesha servers. Only updating the export ID, path, or FSAL block of a CephFS NFS export requires a restart. So modify apply_export() to only restart the ganesha servers for such export updates. The mgr/nfs module creates a FSAL ceph user with read-only or read-write path restricted MDS caps for each export. To change the access type of the CephFS NFS export, the MDS caps of the export's FSAL ceph user must also be changed. Ganesha can dynamically enforce an export's access type changes, but Ceph server daemons can't dynamically enforce changes in caps of the Ceph clients. To allow dynamic updates of CephFS NFS exports, always create a FSAL Ceph user with read-write path restricted MDS caps per export. Rely on the ganesha servers to enforce the export access type changes for the NFS clients. Fixes: https://tracker.ceph.com/issues/54025 Signed-off-by: Ramana Raja <rraja@redhat.com> (cherry picked from commit 2415e03)
... to update export if the ganesha server is going to be restarted. The ganesha server restart will reload the exports. Signed-off-by: Ramana Raja <rraja@redhat.com> (cherry picked from commit 4df294b)
... to fail the test if write to read only export succeeded. Signed-off-by: Ramana Raja <rraja@redhat.com> (cherry picked from commit 3c22f6e)
Add test to verify that the NFS servers don't restart when the access type of a CephFS NFS export is updated. And check the NFS servers are restarted when the pseudo path of a CephFS NFS export is updated. Signed-off-by: Ramana Raja <rraja@redhat.com> (cherry picked from commit d7904e7)
github-actions
bot
added
cephfs
|
This PR depends on #45508 hence the extra commits. |
adk3798
approved these changes
Mar 24, 2022
|
jenkins retest this please |
vshankar
approved these changes
Mar 25, 2022
|
jenkins test make check |
|
jenkins test docs |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #44795
Tracker https://tracker.ceph.com/issues/54025
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windows