Merge pull request #6830 from vumrao/wip-12112-vumrao

rbd: add --object-size option, deprecate --order

Reviewed-by: Josh Durgin <jdurgin@redhat.com>

doc/man/8/rbd.rst

@@ -10,7 +10,7 @@ Synopsis
 ========
 
 | **rbd** [ -c *ceph.conf* ] [ -m *monaddr* ] [--cluster *cluster name*]
-  [ -p | --pool *pool* ] [--size *size* ] [ --order *bits* ] [ *command* ... ] 
+  [ -p | --pool *pool* ] [--size *size* ] [ --object-size *B/K/M* ] [ *command* ... ] 
 
 
 Description
@@ -69,10 +69,11 @@ Parameters
 
    Specifies the size (in M/G/T) of the new rbd image.
 
-.. option:: --order bits
+.. option:: --object-size B/K/M
+
+   Specifies the object size in B/K/M, it will be rounded up the nearest power of two.
+   The default object size is 4 MB, smallest is 4K and maximum is 32M.
 
-   Specifies the object size expressed as a number of bits, such that
-   the object size is ``1 << order``. The default is 22 (4 MB).
 
 .. option:: --stripe-unit size-in-B/K/M
 
@@ -177,17 +178,17 @@ Commands
   require querying the OSDs for every potential object within the image.
 
 :command:`info` *image-spec* | *snap-spec*
-  Will dump information (such as size and order) about a specific rbd image.
+  Will dump information (such as size and object size) about a specific rbd image.
   If image is a clone, information about its parent is also displayed.
   If a snapshot is specified, whether it is protected is shown as well.
 
-:command:`create` (-s | --size *size-in-M/G/T*) [--image-format *format-id*] [--order *bits*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *image-spec*
+:command:`create` (-s | --size *size-in-M/G/T*) [--image-format *format-id*] [--object-size *B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *image-spec*
   Will create a new rbd image. You must also specify the size via --size.  The
   --stripe-unit and --stripe-count arguments are optional, but must be used together.
 
-:command:`clone` [--order *bits*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*] [--image-shared] *parent-snap-spec* *child-image-spec*
+:command:`clone` [--object-size *B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*] [--image-shared] *parent-snap-spec* *child-image-spec*
   Will create a clone (copy-on-write child) of the parent snapshot.
-  Object order will be identical to that of the parent image unless
+  Object size will be identical to that of the parent image unless
   specified. Size will be the same as the parent snapshot. The --stripe-unit
   and --stripe-count arguments are optional, but must be used together.
 
@@ -219,11 +220,11 @@ Commands
 :command:`export` (*image-spec* | *snap-spec*) [*dest-path*]
   Exports image to dest path (use - for stdout).
 
-:command:`import` [--image-format *format-id*] [--order *bits*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *src-path* [*image-spec*]
+:command:`import` [--image-format *format-id*] [--object-size *B/K/M*] [--stripe-unit *size-in-B/K/M* --stripe-count *num*] [--image-feature *feature-name*]... [--image-shared] *src-path* [*image-spec*]
   Creates a new image and imports its data from path (use - for
   stdin).  The import operation will try to create sparse rbd images 
   if possible.  For import from stdin, the sparsification unit is
-  the data block size of the destination image (1 << order).
+  the data block size of the destination image (object size).
 
   The --stripe-unit and --stripe-count arguments are optional, but must be
   used together.
@@ -258,7 +259,7 @@ Commands
 
 :command:`cp` (*src-image-spec* | *src-snap-spec*) *dest-image-spec*
   Copies the content of a src-image into the newly created dest-image.
-  dest-image will have the same size, order, and image format as src-image.
+  dest-image will have the same size, object size, and image format as src-image.
 
 :command:`mv` *src-image-spec* *dest-image-spec*
   Renames an image.  Note: rename across pools is not supported.
@@ -385,10 +386,10 @@ bottleneck when individual images get large or busy.
 
 The striping is controlled by three parameters:
 
-.. option:: order
+.. option:: object-size
 
-  The size of objects we stripe over is a power of two, specifically 2^[*order*] bytes.  The default
-  is 22, or 4 MB.
+  The size of objects we stripe over is a power of two. It will be rounded up the nearest power of two.
+  The default object size is 4 MB, smallest is 4K and maximum is 32M.
 
 .. option:: stripe_unit
 
@@ -398,8 +399,8 @@ The striping is controlled by three parameters:
 .. option:: stripe_count
 
   After we write [*stripe_unit*] bytes to [*stripe_count*] objects, we loop back to the initial object
-  and write another stripe, until the object reaches its maximum size (as specified by [*order*].  At that
-  point, we move on to the next [*stripe_count*] objects.
+  and write another stripe, until the object reaches its maximum size.  At that point,
+  we move on to the next [*stripe_count*] objects.
 
 By default, [*stripe_unit*] is the same as the object size and [*stripe_count*] is 1.  Specifying a different
 [*stripe_unit*] requires that the STRIPINGV2 feature be supported (added in Ceph v0.53) and format 2 images be
@@ -470,7 +471,7 @@ To create a new rbd image that is 100 GB::
 
 To use a non-default object size (8 MB)::
 
-       rbd create mypool/myimage --size 102400 --order 23
+       rbd create mypool/myimage --size 102400 --object-size 8M
 
 To delete an rbd image (be careful!)::
 

src/test/cli-integration/rbd/defaults.t

@@ -12,7 +12,7 @@ Plain create with various options specified via usual cli arguments
       "size": 1048576
   }
   $ rbd rm test --no-progress
-  $ rbd create -s 1 --order 20 test
+  $ rbd create -s 1 --object-size 1M test
   $ rbd info test --format json | python -mjson.tool | sed 's/,$/, /'
   {
       "block_name_prefix": "rb.0.*",  (glob)
@@ -24,6 +24,18 @@ Plain create with various options specified via usual cli arguments
       "size": 1048576
   }
   $ rbd rm test --no-progress
+  $ rbd create -s 1G --object-size 4K test
+  $ rbd info test --format json | python -mjson.tool | sed 's/,$/, /'
+  {
+      "block_name_prefix": "rb.0.*",  (glob)
+      "format": 1, 
+      "name": "test", 
+      "object_size": 4096, 
+      "objects": 262144, 
+      "order": 12, 
+      "size": 1073741824
+  }
+  $ rbd rm test --no-progress
   $ rbd create -s 1 test --image-format 2
   $ rbd info test --format json | python -mjson.tool | sed 's/,$/, /'
   {
@@ -58,7 +70,7 @@ Plain create with various options specified via usual cli arguments
       "size": 1073741824
   }
   $ rbd rm test --no-progress
-  $ rbd create -s 1 test --image-format 2 --order 20
+  $ rbd create -s 1 test --image-format 2 --object-size 1M
   $ rbd info test --format json | python -mjson.tool | sed 's/,$/, /'
   {
       "block_name_prefix": "rbd_data.*",  (glob)
@@ -173,7 +185,7 @@ Format 2 Usual arguments with custom rbd_default_* params
       "stripe_unit": 1048576
   }
   $ rbd rm test --no-progress
-  $ rbd create -s 1 test --image-format 2 --stripe-unit 1048576 --stripe-count 8 --order 23 --rbd-default-order 20
+  $ rbd create -s 1 test --image-format 2 --stripe-unit 1048576 --stripe-count 8 --object-size 8M --rbd-default-order 20
   $ rbd info test --format json | python -mjson.tool | sed 's/,$/, /'
   {
       "block_name_prefix": "rbd_data.*",  (glob)

src/test/cli/rbd/help.t

@@ -123,6 +123,7 @@
   rbd help clone
   usage: rbd clone [--pool <pool>] [--image <image>] [--snap <snap>] 
                    [--dest-pool <dest-pool>] [--dest <dest>] [--order <order>] 
+                   [--object-size <object-size>] 
                    [--image-feature <image-feature>] [--image-shared] 
                    [--stripe-unit <stripe-unit>] [--stripe-count <stripe-count>] 
                    [--journal-splay-width <journal-splay-width>] 
@@ -146,6 +147,7 @@
     --dest-pool arg           destination pool name
     --dest arg                destination image name
     --order arg               object order [12 <= order <= 25]
+    --object-size arg         object size in B/K/M [4K <= object size <= 32M]
     --image-feature arg       image features
                               [layering(+), striping(+), exclusive-lock(*),
                               object-map(*), fast-diff(*), deep-flatten,
@@ -164,6 +166,7 @@
   rbd help copy
   usage: rbd copy [--pool <pool>] [--image <image>] [--snap <snap>] 
                   [--dest-pool <dest-pool>] [--dest <dest>] [--order <order>] 
+                  [--object-size <object-size>] 
                   [--image-feature <image-feature>] [--image-shared] 
                   [--stripe-unit <stripe-unit>] [--stripe-count <stripe-count>] 
                   [--journal-splay-width <journal-splay-width>] 
@@ -187,6 +190,7 @@
     --dest-pool arg              destination pool name
     --dest arg                   destination image name
     --order arg                  object order [12 <= order <= 25]
+    --object-size arg            object size in B/K/M [4K <= object size <= 32M]
     --image-feature arg          image features
                                  [layering(+), striping(+), exclusive-lock(*),
                                  object-map(*), fast-diff(*), deep-flatten,
@@ -206,8 +210,9 @@
   rbd help create
   usage: rbd create [--pool <pool>] [--image <image>] 
                     [--image-format <image-format>] [--new-format] 
-                    [--order <order>] [--image-feature <image-feature>] 
-                    [--image-shared] [--stripe-unit <stripe-unit>] 
+                    [--order <order>] [--object-size <object-size>] 
+                    [--image-feature <image-feature>] [--image-shared] 
+                    [--stripe-unit <stripe-unit>] 
                     [--stripe-count <stripe-count>] 
                     [--journal-splay-width <journal-splay-width>] 
                     [--journal-object-size <journal-object-size>] 
@@ -227,6 +232,7 @@
     --new-format              use image format 2
                               (deprecated)
     --order arg               object order [12 <= order <= 25]
+    --object-size arg         object size in B/K/M [4K <= object size <= 32M]
     --image-feature arg       image features
                               [layering(+), striping(+), exclusive-lock(*),
                               object-map(*), fast-diff(*), deep-flatten,
@@ -446,8 +452,9 @@
   rbd help import
   usage: rbd import [--path <path>] [--dest-pool <dest-pool>] [--dest <dest>] 
                     [--image-format <image-format>] [--new-format] 
-                    [--order <order>] [--image-feature <image-feature>] 
-                    [--image-shared] [--stripe-unit <stripe-unit>] 
+                    [--order <order>] [--object-size <object-size>] 
+                    [--image-feature <image-feature>] [--image-shared] 
+                    [--stripe-unit <stripe-unit>] 
                     [--stripe-count <stripe-count>] 
                     [--journal-splay-width <journal-splay-width>] 
                     [--journal-object-size <journal-object-size>] 
@@ -470,6 +477,7 @@
     --new-format              use image format 2
                               (deprecated)
     --order arg               object order [12 <= order <= 25]
+    --object-size arg         object size in B/K/M [4K <= object size <= 32M]
     --image-feature arg       image features
                               [layering(+), striping(+), exclusive-lock(*),
                               object-map(*), fast-diff(*), deep-flatten,

src/tools/rbd/ArgumentTypes.cc

@@ -218,6 +218,8 @@ void add_create_image_options(po::options_description *opt,
   opt->add_options()
     (IMAGE_ORDER.c_str(), po::value<ImageOrder>(),
      "object order [12 <= order <= 25]")
+    (IMAGE_OBJECT_SIZE.c_str(), po::value<ImageObjectSize>(),
+     "object size in B/K/M [4K <= object size <= 32M]")
     (IMAGE_FEATURES.c_str(), po::value<ImageFeatures>()->composing(),
      ("image features\n" + get_short_features_help(true)).c_str())
     (IMAGE_SHARED.c_str(), po::bool_switch(), "shared image")
@@ -335,7 +337,7 @@ void validate(boost::any& v, const std::vector<std::string>& values,
   po::validators::check_first_occurrence(v);
   const std::string &s = po::validators::get_single_string(values);
   try {
-    uint32_t order = boost::lexical_cast<uint32_t>(s);
+    uint64_t order = boost::lexical_cast<uint64_t>(s);
     if (order >= 12 && order <= 25) {
       v = boost::any(order);
       return;
@@ -345,6 +347,19 @@ void validate(boost::any& v, const std::vector<std::string>& values,
   throw po::validation_error(po::validation_error::invalid_option_value);
 }
 
+void validate(boost::any& v, const std::vector<std::string>& values,
+              ImageObjectSize *target_type, int dummy) {
+  po::validators::check_first_occurrence(v);
+  const std::string &s = po::validators::get_single_string(values);
+
+  std::string parse_error;
+  uint64_t objectsize = strict_sistrtoll(s.c_str(), &parse_error);
+  if (!parse_error.empty()) {
+    throw po::validation_error(po::validation_error::invalid_option_value);
+  }
+  v = boost::any(objectsize);
+}
+
 void validate(boost::any& v, const std::vector<std::string>& values,
               ImageFormat *target_type, int dummy) {
   po::validators::check_first_occurrence(v);

src/tools/rbd/ArgumentTypes.h

@@ -59,6 +59,7 @@ static const std::string WHOLE_OBJECT("whole-object");
 static const std::string IMAGE_FORMAT("image-format");
 static const std::string IMAGE_NEW_FORMAT("new-format");
 static const std::string IMAGE_ORDER("order");
+static const std::string IMAGE_OBJECT_SIZE("object-size");
 static const std::string IMAGE_FEATURES("image-feature");
 static const std::string IMAGE_SHARED("image-shared");
 static const std::string IMAGE_SIZE("size");
@@ -80,6 +81,7 @@ static const std::set<std::string> SWITCH_ARGUMENTS = {
 
 struct ImageSize {};
 struct ImageOrder {};
+struct ImageObjectSize {};
 struct ImageFormat {};
 struct ImageNewFormat {};
 
@@ -171,6 +173,8 @@ void validate(boost::any& v, const std::vector<std::string>& values,
               ImageSize *target_type, int);
 void validate(boost::any& v, const std::vector<std::string>& values,
               ImageOrder *target_type, int);
+void validate(boost::any& v, const std::vector<std::string>& values,
+              ImageObjectSize *target_type, int);
 void validate(boost::any& v, const std::vector<std::string>& values,
               ImageFormat *target_type, int);
 void validate(boost::any& v, const std::vector<std::string>& values,

src/tools/rbd/Utils.cc

@@ -317,11 +317,16 @@ int validate_snapshot_name(at::ArgumentModifier mod,
 
 int get_image_options(const boost::program_options::variables_map &vm,
 		      bool get_format, librbd::ImageOptions *opts) {
-  uint64_t order, features, stripe_unit, stripe_count;
+  uint64_t order, features, stripe_unit, stripe_count, object_size;
   bool features_specified = false;
 
   if (vm.count(at::IMAGE_ORDER)) {
     order = vm[at::IMAGE_ORDER].as<uint64_t>();
+    std::cerr << "rbd: --order is deprecated, use --object-size"
+	      << std::endl;
+  } else if (vm.count(at::IMAGE_OBJECT_SIZE)) {
+    object_size = vm[at::IMAGE_OBJECT_SIZE].as<uint64_t>();
+    order = std::round(std::log2(object_size)); 
   } else {
     order = 22;
   }