nautilus: mgr/volumes: misc fix and feature enhancements #33122
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajarr
force-pushed
the
wip-ajarr-mgr-volumes-nautilus
branch
from
573c309
to
9e80caf
Compare
|
Note to reviewer: I've included commits of Patrick's backport PR #33116 as the commits of this PR depend on Patrick's PR. |
|
The minor changes had to do to resolve conflicts were, a8ffb0d#diff-29724c965f03adba0d31395b03ac724bR40 |
ajarr
force-pushed
the
wip-ajarr-mgr-volumes-nautilus
branch
2 times, most recently
from
51554ba
to
b513b58
Compare
ajarr
force-pushed
the
wip-ajarr-mgr-volumes-nautilus
branch
from
b513b58
to
ac86d66
Compare
Add a new Python binding equivalent to lstat so that information about the symlink itself can be also obtained, along with other type of files. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 8cce7da)
Fixes: http://tracker.ceph.com/issues/42646 Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit b9cff8a)
* Fixed raises that doesn't re-raise * Dropped some commands with --force remove commands, as it is unnecessary. Signed-off-by: Jos Collin <jcollin@redhat.com> (cherry picked from commit 992d8b6)
* Raised RuntimeException when the commands, which were expected to fail succeed. * Dropped some commands with --force remove commands, as it is unnecessary. Signed-off-by: Jos Collin <jcollin@redhat.com> (cherry picked from commit c8f6a25)
subvolume base class implements common routines/helpers and initializes a metadata manager. later, when v2 subvolume version is implemented, the metadata manager would be used to persist subvolume metadata in ceph filesystem. this would allow flexible metadata management when complex subvolume features are added. typically, a subvolume would be implemented by subclassing the subvolume base class and the subvolume template -- instantiating this would be called a "subvolume object". with this commit, current subvolume topology is maintained. but we introduce the concept of subvolume versions. a loader stub loads available "versions" of subvolumes. right now, the only available version is v1. since backward compatibility needs to be maintained for existing subvolumes, the loader API allows version discovery w/ auto upgradation to the most recent version. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 97170d7)
create_subvolume() creates a subvolume with the max version known to the plugin. open_subvolume() performs version discovery by using loader stub and returns a subvoule object. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 8a64914)
apart from the new way of provisioning subvolumes, this makes heavy
use of context manager for volumes, groups and subvolumes.
this change classifies volumes, groups and subvolumes to be treated
as filesystem dentries and inodes. a "volume" can be thought as a
dentry with "groups" as it's entries (inodes). likewise, a "group"
is a dentry again with "subvolumes" as entries (inodes). this is
built into the access mechanism as follows:
with open_volume(...) as fs_handle:
with open_gorup(fs_handle, ...) as group:
with open_subvolume(group, ...) as subvolume:
# call subvolume object API
path = subvolume.getpath()
this way, lot of redundant checks such as verifying if a volume or
group exist before accessing a subvolume is built right into the
access mechanism, plus, an added bonus of simple error handling.
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 9b87bd7)
Fixes: https://tracker.ceph.com/issues/43349 Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 03ee966)
this was lying around post versioning changes. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 5595476)
Fixes: http://tracker.ceph.com/issues/43645 Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit c158a13)
... and fetch creation state from state machine table. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 46f29bf)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 5318779)
This will be required when creating a clone as the clone would inherit source subvolumes creation mode and uid/gid. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit f02b1e7)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 7089808)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 461909b)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 8d68f1a)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit fa3c56f)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit b2145b7)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 7ad14cf)
This also makes `_cancel_jobs()` thread safe, which was not the case earlier (with `_cancel_purge_job()`) -- this also makes the code simpler by sharing the lock betweent two condition variables. Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit f16cc1e)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 4f09568)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit 451be11)
Signed-off-by: Venky Shankar <vshankar@redhat.com> (cherry picked from commit b5970ff)
This fix is only needed in nautilus, and the issue
was observed during upstream teuthology testing.
File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 114, in cptree
copy_file(fs_handle, d_full_src, d_full_dst, mo, st.st_uid, st.st_gid)
File "/usr/share/ceph/mgr/volumes/fs/fs_util.py", line 97, in copy_file
fs.chown(dst, uid, gid)
File "cephfs.pyx", line 855, in cephfs.LibCephFS.chown
TypeError: uid must be an int
The issue wasn't observed in master/octopus teuthology
testing.
Signed-off-by: Ramana Raja <rraja@redhat.com>
Fix the following issue seen while upstream teuthology testing,
File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 98, in load_config
self.metadata_mgr = MetadataManager(self.fs, self.legacy_config_path, 0o640)
File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 73, in legacy_config_path
meta_config = "{0}.meta".format(m.digest().hex())
AttributeError: 'str' object has no attribute 'hex'
This issue is not observed in master/octopus, as it only supports
py3.
Signed-off-by: Ramana Raja <rraja@redhat.com>
ajarr
force-pushed
the
wip-ajarr-mgr-volumes-nautilus
branch
from
ac86d66
to
3b514df
Compare
OK -- will take a look! |
smithfarm
changed the title
|
NOTE: Added https://tracker.ceph.com/issues/42738 to PR description. |
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: https://tracker.ceph.com/issues/44020
Fixes: https://tracker.ceph.com/issues/43724
Fixes: https://tracker.ceph.com/issues/43629
Fixes: https://tracker.ceph.com/issues/43338
Fixes: https://tracker.ceph.com/issues/43271
Fixes: https://tracker.ceph.com/issues/42951
Fixes: https://tracker.ceph.com/issues/42738