The master version of this PR deals with the unified asok/tell
interface. Nautilus- are separated.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9dc07d8)

Rationale can be found in [1]. Point is that EC pools incur a
significant performance penalty when dealing with small files and xattr
updates. This is because _every_ inode has a corresponding data pool
object with backtrace information stored in its xattr.

[1] doc/cephfs/createfs.rst

Fixes: https://tracker.ceph.com/issues/42450
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 3e0aee5)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit bf0cf8e)

Conflicts:
	qa/tasks/cephfs/filesystem.py

In the future, we should add the EC data pool as a supplementary data
pool but that requires a mount to setup which is awkward in the code
here. When cephfs-shell is more widely available, this will be easier.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6e448f9)

Connection pointer is not helpful.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 9b71bbe)

Conflicts:
	src/mgr/DaemonServer.cc: actually do print the Connection*, Connection& cannot be dumped in Nautilus.

If the mgr is waiting on daemon metadata from the mons, it has no
DaemonState associated with the daemon yet. If we try to process this
MgrOpen, the metadata sent by the daemon (like its config) will not be
recorded.

Fixes: https://tracker.ceph.com/issues/43037
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 16a1deb)

Added 'telemetry show-device' command to print a preview of telemetry device report.
Added a message at the bottom of 'telemetry show' about 'telemetry show-device' new command.

Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit cae87cc)

Conflicts:
- path: src/pybind/mgr/telemetry/module.py
  comment: nautilus version of json.dumps() don't have sort_keys arg.

smartctl JSON output contains the device's serial number in two
different keys ('serial_number' & 'output'). Serial is now obfuscated in
both.

Fixes: https://tracker.ceph.com/issues/43939
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit be1257f)

get_metada() returns 'None' when requesting a missing service, hence
trying to access its content fails. Added a check for osd and mgr
get_metadata() calls.

Fixes: https://tracker.ceph.com/issues/43642
Signed-off-by: Yaarit Hatuka <yaarit@redhat.com>
(cherry picked from commit 9e7a0cb)

nautilus: mgr/dashboard: Using wrong identifiers in RGW user/bucket datatables

Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

nautilus: mgr/dashboard: iSCSI targets not available if any gateway is down (and more...)

Reviewed-by: Jason Dillaman <dillaman@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Tiago Melo <tmelo@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

nautilus: mgr/dashboard: add debug mode, and accept expected exception when SSL handshaking

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Stephan Müller <smueller@suse.com>
Reviewed-by: Tatjana Dehler <tdehler@suse.com>

nautilus: mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>
Reviewed-by: Volker Theile <vtheile@suse.com>

nautilus: mgr/dashboard: check embedded Grafana dashboard references

Reviewed-by: Laura Paduano <lpaduano@suse.com>

nautilus: mgr/dashboard: check if user has config-opt permissions

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
Reviewed-by: Laura Paduano <lpaduano@suse.com>

nautilus: mgr/dashboard: disable 'Add Capability' button in rgw user edit

Reviewed-by: Laura Paduano <lpaduano@suse.com>

Nothing inherits from PQ.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 761cc0e)

Conflicts:
	src/mds/PurgeQueue.h

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 096a5ca)

This makes the corresponding test not racy.

Fixes: https://tracker.ceph.com/issues/16881
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Conflicts:
	src/mds/PurgeQueue.cc
	src/mds/PurgeQueue.h

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 98e3b7e)
Note: removed mgr blacklist test which applies to Octopus.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

The squelched error prevented us from knowing connection cleanup doesn't
work on py3.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit b45c08b)

Otherwise this raises an exception.

Fixes: https://tracker.ceph.com/issues/43113
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 03f8080)

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
(cherry picked from commit 6714364)

nautilus: mgr/MgrClient: fix open condition

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

nautilus: selinux: Allow ceph to read udev db

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Boris Ranto <branto@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: kv: fix shutdown vs async compaction

Reviewed-by: Neha Ojha <nojha@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

nautilus: osd: Diagnostic logging for upmap cleaning

Reviewed-by: David Zafman <dzafman@redhat.com>

nautilus: osd: Use physical ratio for nearfull (doesn't include backfill resserve)

Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: osd/OSD: enhance osd numa affinity compatibility

Reviewed-by: Kefu Chai <kchai@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

nautilus: osd/PeeringState.cc: skip peer_purged when discovering all missing

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: qa/suites/rados/thrash: force normal pg log length with cache tiering

Reviewed-by: Sage Weil <sage@redhat.com>

nautilus: osd/PeeringState.cc: don't let num_objects become negative

Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: core: osd/OSDMap: health alert for non-power-of-two pg_num

Reviewed-by: Neha Ojha <nojha@redhat.com>

Fixes: https://tracker.ceph.com/issues/36728

Fallback to predefined paths for backward compatibility.
Alter test involved for partial match in warning

Signed-off-by: Shyukri Shyukriev <shshyukriev@suse.com>
(cherry picked from commit a857708)

Users complained[1] the error message isn't clear, and they thought
it referred to the cluster fsid instead of the osd_fsid.
Made it clearer.

[1] rook/rook#4547

Fixes: https://tracker.ceph.com/issues/43442
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
(cherry picked from commit ff3ba92)

Fixes: bb4de1a
Fixes: https://tracker.ceph.com/issues/42970

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c1bd09f)

Conflicts:
	src/ceph-volume/ceph_volume/tests/api/test_lvm.py

…int from string.

Fixes: https://tracker.ceph.com/issues/43186

Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit 81ff4be)

…r_device

Actual data size depending on osds_per_device needs to be calculated here. Otherwise, if osds_per_device is greater than 1, ceph-volume will allocate 100% of the device to the first osd and then fail to create the LV for the second because the volume group is already full.

Fixes: https://tracker.ceph.com/issues/39442
Signed-off-by: Fabian Niepelt <f.niepelt@mittwald.de>
(cherry picked from commit ecde6cd)

 Conflicts:
	src/ceph-volume/ceph_volume/devices/lvm/strategies/bluestore.py

I've removed `data_uuid` since it's not in nautilus already

This allows for a symlink to be passed to

```
ceph-volume lvm list <path>
```

which makes it possible to use `/dev/disk/by-path/*` devices, for
instance.

Fixes: https://tracker.ceph.com/issues/43497
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 09fa3df)

A lot of our functionality depends on the mgr now. If there is a cluster
set up with osds but no mgr, issue a warning.

Fixes: https://tracker.ceph.com/issues/38942
Signed-off-by: Neha Ojha <nojha@redhat.com>
(cherry picked from commit c20e51c)

On Linux system it is possible to set 64 character length hostname when
HOST_NAME_MAX is set to 64. It means that if we execute gethostname
function we should expect HOST_NAME_MAX characters + 1 for null
character ending hostname string as described here:
http://man7.org/linux/man-pages/man2/sethostname.2.html

With the current code on host with 64 long hostname osd during start
updates crush map with host=unknown_host.

Signed-off-by: Michal Skalski <mskalski@juniper.net>
(cherry picked from commit 5201048)

Fixes: https://tracker.ceph.com/issues/42119

Signed-off-by: Richard Bai(白学余) <baixueyu@inspur.com>
(cherry picked from commit 78125a8)

…ils but no error messages

Fixes: None
Signed-off-by: Snow Si <silonghu@inspur.com>

(cherry picked from commit ff2f4af)

The session gets put as result of the set_session call in the next
block.

Fixes: https://tracker.ceph.com/issues/38345

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
(cherry picked from commit 8f23f2c)

Introduced by 08fcf01, which activated
this (broken) code path.

Fixes: https://tracker.ceph.com/issues/43892
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ab64a22)

This commit adds per-pool pg states metrics
with unique 'pool_id' label.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
(cherry picked from commit 8fb16e4)

If we have all other stats by pool, it's better to have total
count by pool too. We always can sum() all of total, but it's
hard to count by-pool total.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
(cherry picked from commit 41e7d20)

Update pg metrics descriptions to show that we have per
pool stats now.

Signed-off-by: Aleksei Zakharov <zakharov.a.g@yandex.ru>
(cherry picked from commit 96bd77d)

Signed-off-by: Aleksei Zakharov <zaharov@selectel.ru>
(cherry picked from commit 4eb58f7)

Also, revert table formatting.

Signed-off-by: Aleksei Zakharov <zaharov@selectel.ru>
(cherry picked from commit a37cf38)

If an entity name (id.type) has more than one dot (i.e. 'type' has
dots), split only on the first one

Fixes: https://tracker.ceph.com/issues/43313
Signed-off-by: Dan Mick <dan.mick@redhat.com>
(cherry picked from commit 626c7df)

Using raw_used_rate to calculate the pool_pg_target results in too
many PGs for erasure coded pools (e.g. EC 4+2 has raw_used_rate=1.5
but size is 6, so there will be 4x too many PGs). Calculate using
p['size'] instead.

Signed-off-by: Dan van der Ster <daniel.vanderster@cern.ch>
Fixes: https://tracker.ceph.com/issues/43546
(cherry picked from commit b766d5f)

Fixes: https://tracker.ceph.com/issues/43583

Signed-off-by: dongdong tao <dongdong.tao@canonical.com>
(cherry picked from commit fb6f78a)

Conflicts:
	src/rgw/rgw_reshard.cc
- nautilus does not have 7e613fd
- in nautilus, RGWMPObj is defined in rgw_rados.h

nautilus: test: Fix wait_for_state() to wait for a PG to get into a state

Reviewed-by: Neha Ojha <nojha@redhat.com>

nautilus: os/bluestore/BlueStore.cc: set priorities for compression stats

Reviewed-by: Igor Fedotov <ifedotov@suse.com>

nautilus: common: fix deadlocky inflight op visiting in OpTracker.

Reviewed-by: Kefu Chai <kchai@redhat.com>

nautilus: common/util: use ifstream to read from /proc files

Reviewed-by: Kefu Chai <kchai@redhat.com>

nautilus: mon/MgrMonitor.cc: add always_on_modules to the output of "ceph mgr module ls"

Reviewed-by: David Zafman <dzafman@redhat.com>

nautilus: common/config: update values when they are removed via mon

Reviewed-by: Sage Weil <sage@redhat.com>

nautilus: common/options: bluestore 4k min_alloc_size for SSD

Reviewed-by: Sage Weil <sage@redhat.com>
Reviewed-by: Igor Fedotov <ifedotov@suse.com>

nautilus: ceph-volume: minor clean-up of "simple scan" subcommand help

nautilus: ceph-volume/test: patch VolumeGroups

Fixes: https://tracker.ceph.com/issues/42777

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit b35e8c4)

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4749f4c)

Conflicts:
	src/ceph-volume/ceph_volume/tests/conftest.py
        resolved by importing PropertyMock

When batch is called non-interactively and a user explicitly specifies,
say a db-device, this will be filtered when unavailable. This can cause
the resulting OSD to be very different from the users intention
(standalone vs external db when the db-device was filtered). If devices
get filtered in non-interactive mode, ceph-volume should fail.

Fixes: https://tracker.ceph.com/issues/43105

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2e98505)

nautilus: ceph-volume: assume msgrV1 for all branches containing mimic

nautilus: ceph-volume: util: look for executable in $PATH

nautilus: ceph-volume/lvm/activate.py: clarify error message: fsid refers to osd_fsid

nautilus: ceph-volume: use correct extents if using db-devices and >1 osds_per_device

nautilus: ceph-volume: fix the integer overflow

nautilus: ceph-volume: Dereference symlink in lvm list

nautilus: ceph-volume: import mock.mock instead of unittest.mock (py2)

nautilus: ceph-volume: make get_devices fs location independent

nautilus: ceph-volume/batch: fail on filtered devices when non-interactive

This changes create_lv so one can pass the desired device and either a
VG with a name starting with ceph is re-used or a new one is created.
This commit also adds two new lvm primitives, making use of lvm's select
feature. The goal is to eventually avoid keeping a full list of lv's (or
vg's) around and query the lvm system as needed.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit bb4de1a)

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 01a603f)

Add option to pass raw physical devices everywhere, restructure a little
(bluestore section before filestore) and reword a few things.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit f2018d7)

nautilus: ceph-volume: allow raw block devices everywhere

Simply calls lvchange -an to deactivate a logical volume.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8087600)

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 2558b55)

Thsi unmounts a path if and only if it's a tmpfs mount.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 705ed11)

This new subcommand unmounts and OSDs tmpfs mount and closes crypt
devices if there are any.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 9797f6b)

nautilus: mds: reject sessionless messages

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: MDSMonitor: warn if a new file system is being created with an EC default data pool

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: mds: reject forward scrubs when cluster has multiple active MDS (more than one rank)

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: mds: fix revoking caps after after stale->resume circle

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: cephfs-journal-tool: fix crash and usage

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: mds: note client features when rejecting client

Reviewed-by: Ramana Raja <rraja@redhat.com>

nautilus: client: disallow changing fuse_default_permissions option at runtime

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Jeff Layton <jlayton@redhat.com>

nautilus: ceph-volume: lvm deactivate command

Remove the --all flag until its actually implemented.

Fixes: https://tracker.ceph.com/issues/43330

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit c13901f)

Conflicts:
	src/ceph-volume/ceph_volume/devices/lvm/deactivate.py

nautilus: mon: print FSMap regardless of file system count

Reviewed-by: Ramana Raja <rraja@redhat.com>

Filters can be passed to these commands by using option '-S'.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit a4f2fce)

nautilus: ceph-volume: add methods to pass filters to pvs, vgs and lvs commands

nautilus: ceph-volume: lvm/deactivate: add unit tests, remove --all

…elease

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

…autilus

qa/tests: added client-upgrade-nautilus suite to be used on octopus …

Reviewed-by: Neha Ojha <nojha@redhat.com>

The Size class can now parse strings and has support for arithmetic
operations and comparisons with numbers.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit dd89f46)

This adds options to size to-be-created LVs in the prepare and create
subcommands. Sizing can be done explicitly by passing a sizes or
implicitly by specifying the number of slots per [data|journal|wal|db]
device. The former will try to create a LV of the specified size and use
that to create OSDs if it succeeds. The latter will carve up the device
size into $n slots and use one of those slots for the to-be-created OSD.
If partitions or LVs are passed these options are ignored.
This also creates the foundation to move to byte-based sizing, by moving
VolumeGroup lvm querying and size calculation to bytes as the base unit.

Fixes: https://tracker.ceph.com/issues/43299

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 8b8913a)

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 9e61b80)

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 4051129)

This was introduced in ceph#32242
erroneously.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit 6f107f7)

Fixes: https://tracker.ceph.com/issues/43844

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit df18497)

nautilus: pybind/mgr/volumes: idle connection drop is not working

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Sage Weil <sage@redhat.com>

Add a new Python binding equivalent to lstat so that information about
the symlink itself can be also obtained, along with other type of files.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 8cce7da)

Fixes: http://tracker.ceph.com/issues/42646
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b9cff8a)

* Fixed raises that doesn't re-raise
* Dropped some commands with --force remove commands, as it is unnecessary.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 992d8b6)

* Raised RuntimeException when the commands, which were expected to fail succeed.
* Dropped some commands with --force remove commands, as it is unnecessary.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit c8f6a25)

…-force

* tests 'fs subvolume rm --force'
* tests 'fs subvolume snapshot rm --force'
* tests 'fs subvolumegroup rm --force'
* tests 'fs subvolumegroup snapshot rm --force'

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 42c135d)

Fixes: https://tracker.ceph.com/issues/42872
Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 5e998bd)

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 6913524)

There are only 2 cases which needs cleanup:
1. The volume is successfully created
2. The volume is successfully created but create_mds fails

In either case, we could do a 'volume rm'.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 67e43f4)

... which was not fully implemented anyway, so just remove the
boilerplates.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 45c22bd)

introduce with statement in rmtree. This change
simplifies the code's handling of directory cleanup.

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 9e27cd1)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 2eb0c50)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 968f675)

Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit 2f705ee)

Instead of checking if the --yes-i-really-mean-it
flag was set _after_ removing the MDS daemon, we
need to check if before starting any removal operation.

Fixes: https://tracker.ceph.com/issues/42931
Signed-off-by: Joshua Schmid <jschmid@suse.de>
(cherry picked from commit c1db7f8)

* clean up on fs create error
* drop unnecessary check in create_pool

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit 171c375)

This is fixed already.
Now the pool names are:
cephfs.<volume name>.meta
cephfs.<volume name>.data

Signed-off-by: Jos Collin <jcollin@redhat.com>
(cherry picked from commit ffda5f6)

helpers for various filesystem querying routines, utils
for creating/removing filesystem, pool and MDSs.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 6682c77)

unlike existing subvolume specification, this is just a
minimal set of globally available configurations. bulk
of other configurations will be moved to the respective
entity modules (subsequent commits).

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 0039b5d)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bc89d05)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f9ae6e3)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 74f349f)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b30f0cb)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 0e3c48e)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3eccd61)

subvolume base class implements common routines/helpers and
initializes a metadata manager. later, when v2 subvolume version
is implemented, the metadata manager would be used to persist
subvolume metadata in ceph filesystem. this would allow flexible
metadata management when complex subvolume features are added.

typically, a subvolume would be implemented by subclassing the
subvolume base class and the subvolume template -- instantiating
this would be called a "subvolume object".

with this commit, current subvolume topology is maintained. but
we introduce the concept of subvolume versions. a loader stub
loads available "versions" of subvolumes. right now, the only
available version is v1. since backward compatibility needs to
be maintained for existing subvolumes, the loader API allows
version discovery w/ auto upgradation to the most recent version.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 97170d7)

create_subvolume() creates a subvolume with the max version known
to the plugin. open_subvolume() performs version discovery by
using loader stub and returns a subvoule object.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8a64914)

apart from the new way of provisioning subvolumes, this makes heavy
use of context manager for volumes, groups and subvolumes.

this change classifies volumes, groups and subvolumes to be treated
as filesystem dentries and inodes. a "volume" can be thought as a
dentry with "groups" as it's entries (inodes). likewise, a "group"
is a dentry again with "subvolumes" as entries (inodes). this is
built into the access mechanism as follows:

      with open_volume(...) as fs_handle:
          with open_gorup(fs_handle, ...) as group:
              with open_subvolume(group, ...) as subvolume:
                  # call subvolume object API
                  path = subvolume.getpath()

this way, lot of redundant checks such as verifying if a volume or
group exist before accessing a subvolume is built right into the
access mechanism, plus, an added bonus of simple error handling.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 9b87bd7)

Fixes: https://tracker.ceph.com/issues/43349
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 03ee966)

this was lying around post versioning changes.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5595476)

Fixes: http://tracker.ceph.com/issues/43645
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit c158a13)

... and fetch creation state from state machine table.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 46f29bf)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 5318779)

This will be required when creating a clone as the clone would
inherit source subvolumes creation mode and uid/gid.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f02b1e7)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7089808)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 461909b)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8d68f1a)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit fa3c56f)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b2145b7)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7ad14cf)

This also makes `_cancel_jobs()` thread safe, which was not the
case earlier (with `_cancel_purge_job()`) -- this also makes the
code simpler by sharing the lock betweent two condition variables.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f16cc1e)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 4f09568)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 451be11)

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b5970ff)

This fix is only needed in nautilus, and the issue
was observed during upstream teuthology testing.

   File "/usr/share/ceph/mgr/volumes/fs/async_cloner.py", line 114, in cptree
     copy_file(fs_handle, d_full_src, d_full_dst, mo, st.st_uid, st.st_gid)
   File "/usr/share/ceph/mgr/volumes/fs/fs_util.py", line 97, in copy_file
     fs.chown(dst, uid, gid)
   File "cephfs.pyx", line 855, in cephfs.LibCephFS.chown
      TypeError: uid must be an int

The issue wasn't observed in master/octopus teuthology
testing.

Signed-off-by: Ramana Raja <rraja@redhat.com>

Fix the following issue seen while upstream teuthology testing,
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 98, in load_config
   self.metadata_mgr = MetadataManager(self.fs, self.legacy_config_path, 0o640)
 File "/usr/share/ceph/mgr/volumes/fs/operations/versions/subvolume_base.py", line 73, in legacy_config_path
   meta_config = "{0}.meta".format(m.digest().hex())
 AttributeError: 'str' object has no attribute 'hex'

This issue is not observed in master/octopus, as it only supports
py3.

Signed-off-by: Ramana Raja <rraja@redhat.com>

nautilus: ceph-volume: add sizing arguments to prepare

nautilus: ceph-volume: batch bluestore fix create_lvs call

get_pvs, get_vgs and get_lvs must accept tags and filter volumes based
on tags.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit fb13909)

These convenience methods shortens following phrase to
"lv = get_first_lv()" -

lvs = get_lvs()
if len(lvs) >= 1:
	lvs = lv[0]

These methods do the same things as above phrase internall. Rewrite
listing.py to use these new helper methods.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 17957d9)

The method determines whether given LV is managed by Ceph or not.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 876244b)

Get rid of duplicate and redundant code and use get_lvs, get_vgs and
get_pvs to simplify the module as much as possible.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d02bd7d)

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit d1ae6d1)

lisitng.py doesn't call api.Volumes anymore. Therefore, this test is
redundant.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
(cherry picked from commit 665ed24)

17957d9 introduced a regression in `lvm
list`.

When passing a vg/lv path for generating a single report, it fails
because the filter used in the `lvs` command isn't right. It uses the lv
name instead of the vg name because `os.path.basename(device)` is used
while it should be `os.path.dirname(device)`

Fixes: https://tracker.ceph.com/issues/43969

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 0179fed)

Also drop the sep argument from get_lvs and siblings, unused.
Introduce LV_CMD_OPTIONS to unify options to lvs.

Signed-off-by: Jan Fajerski <jfajerski@suse.com>
(cherry picked from commit ffe5b57)