osd: objclass sdk #14723

neha-ojha · 2017-04-21T22:16:57Z

This pull request aims at creating an object class SDK interface for building Ceph object classes outside the tree. The cls_sdk object class makes use of this interface. It has unittests that can be run to verify correctness.

tchaikov · 2017-04-24T15:10:34Z

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osd-scrub-repair.sh:2214: corrupt_scrub_erasure:  test no = yes
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osd-scrub-repair.sh:2214: corrupt_scrub_erasure:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/osd/osd-scrub-repair.sh:45: run:  return 1

https://jenkins.ceph.com/job/ceph-pull-requests/22697/

@neha-ojha i don't think the build failure is relevant to your change.

tchaikov · 2017-04-24T15:10:51Z

retest this please.

liewegas · 2017-04-25T15:27:20Z

Two questions:

Eventually we should get everything out of objclass/objclass.h and into the rados/objclass.h, right? Is there any reason to hold off on that?
I wonder if we should include zlog or not. Is it better to have a version in the tree?

neha-ojha · 2017-04-25T17:06:20Z

Most parts of objclass/objclass.h can move into rados/objclass.h, except for the PGLSFilter class(requires changes in the osd). Moving some bits requires changes in objclass/class_api.cc.
This initial version was created with the aim to keep the patch smaller, but I can incorporate the required changes and create another pull request.
Not sure of about that. I added it here to demonstrate the use of the sdk.

liewegas · 2017-04-25T17:15:57Z

1. that's fine; we don't have to do it all at once. 2. ok!

jdurgin · 2017-04-25T19:44:17Z

src/include/rados/objclass.h

+
+#include <map>
+#include <set>
+#include "../include/buffer.h"


one small thing: change this to #include "buffer.h" so it references the symlink in include/rados/buffer.h from the source tree, and the version installed by librados-dev when installed from packages (you've already got the dependency on that package so buffer.h will be installed)

Sure, I will change it.

jdurgin · 2017-04-25T19:50:27Z

great to see good docs and packaging included from the start! You can take DNM out of the commit message. Looks good to me beyond the small include change.

neha-ojha · 2017-04-25T20:59:15Z

@liewegas @jdurgin @tchaikov: Thanks for your feedback!

dotnwat · 2017-04-26T14:13:24Z

@neha-ojha cls_zlog in this PR is a little out-of-date. How do you want me to send you updates?

liewegas · 2017-04-26T15:17:06Z

Would it be simpler to just leave zlog out, or do you want to get the latest upstream into luminous?

liewegas · 2017-04-26T18:16:29Z

Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.1-1548-g6b64459/rpm/el7/BUILDROOT/ceph-12.0.1-1548.g6b64459.el7.x86_64
error: Installed (but unpackaged) file(s) found:
   /usr/include/rados/objclass.h

during centos build

liewegas · 2017-04-26T18:17:02Z

looks like the %files section for the new package is missing from ceph.spec.in

yuriw · 2017-04-26T20:21:32Z

Per @liewegas removed as suspect breaking centos builds https://shaman.ceph.com/builds/ceph/wip-sage-testing_2017_4_27/

dotnwat · 2017-04-26T22:29:16Z

leaving cls_zlog out makes sense to me. I can provide feedback / patches as an SDK user. Some of the protocol buffer stuff seems to be a little slow, and that's an example of something that will be easier to handle maintaining it as an external sdk user, on a slower or faster pace than ceph releases.

liewegas · 2017-04-26T22:32:47Z

ok. in that case, let's strip zlog out of this pr, including the protobufs dependency. minimal works for me. thanks!

neha-ojha · 2017-04-26T22:34:06Z

Sure.

tchaikov · 2017-04-27T05:57:30Z

src/include/rados/objclass.h

+#ifdef __cplusplus
+
+#include <map>
+#include <set>


why we need to include set and map here?

Yes, they are not required now. I had included them for cls_cxx_getxattrs() and cls_cxx_map_get_keys(), which are no more a part of the api. I will remove them. Thanks for pointing that.

tchaikov · 2017-04-27T06:05:02Z

src/test/cls_sdk/CMakeLists.txt

+  radostest
+  ${UNITTEST_LIBS}
+  )
+install(TARGETS


why would we install this test? i believe it's but a unittest, and is not exercised by any qa suite, right?

instead i'd recommend add it using add_ceph_unittest().

I followed the CMakeLists.txt for unittests of existing object classes.

@neha-ojha, ceph_test_cls_sdk is not a "cls", it's an executable. could you point me to a unittest of existing object class unittest which installs the executable?

I mostly followed this https://github.com/ceph/ceph/blob/master/src/test/cls_hello/CMakeLists.txt#L15. But I can always incorporate your suggestion.

@neha-ojha oic. it's up to you to run this test as part of unit test ("make check"), or exercise it using qa/workunits/cls/test_cls_hello.sh. i'd recommend the latter.

@tchaikov I made the following changes and it works. Is there anything else that I would need to change?

Removed the install section and added
add_ceph_unittest(ceph_test_cls_sdk ${CMAKE_RUNTIME_OUTPUT_DIRECTORY}/ceph_test_cls_sdk) in CMakeLists.txt.

Added qa/workunits/cls/test_cls_sdk.sh

@neha-ojha you cannot do both. if you choose to run this test as part of "make check", then "add_ceph_unittest()", and there is no necessary to run it in qa/workunits/cls/test_cls_sdk.sh, but if you choose otherwise, you need to package this test by install(TARGET ...), so qa/workunits/cls/test_cls_sdk.sh can exercise it in a qa suite.

Got it. I'll add qa/workunits/cls/test_cls_sdk.sh. Thanks!

tchaikov

might want to use add_ceph_unittest() to add unittest.

liewegas · 2017-04-27T14:18:01Z

Can you squash all of this down into one commit now that the zlog pieces are removed?

neha-ojha · 2017-04-27T14:22:16Z

yes.

runsisi · 2017-04-27T15:44:00Z

ceph.spec.in

@@ -718,6 +718,15 @@ This package contains the Java libraries for the Ceph File System.

 %endif

+%package -n rados-objclass-dev


rados-objclass-devel ?

liewegas · 2017-04-27T16:18:58Z

right, the suffix is -devel for rpms

Creates an installable version of "src/include/rados/objclass.h" that allows object classes to be built outside of the Ceph tree. cls_sdk is an example of such an object class. Signed-off-by: Neha Ojha <nojha@redhat.com>

liewegas · 2017-04-28T23:29:47Z

2017-04-28T22:01:36.530 INFO:tasks.workunit.client.0.smithi138.stdout:[==========] Running 2 tests from 1 test case.
2017-04-28T22:01:36.530 INFO:tasks.workunit.client.0.smithi138.stdout:[----------] Global test environment set-up.
2017-04-28T22:01:36.530 INFO:tasks.workunit.client.0.smithi138.stdout:[----------] 2 tests from ClsSDK
2017-04-28T22:01:36.530 INFO:tasks.workunit.client.0.smithi138.stdout:[ RUN      ] ClsSDK.TestSDKCoverageWrite
2017-04-28T22:01:38.682 INFO:tasks.workunit.client.0.smithi138.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.1-1708-gfa34740/rpm/el7/BUILD/ceph-12.0.1-1708-gfa34740/src/test/cls_sdk/test_cls_sdk.cc:1
7: Failure
2017-04-28T22:01:38.683 INFO:tasks.workunit.client.0.smithi138.stdout:      Expected: 0
2017-04-28T22:01:38.683 INFO:tasks.workunit.client.0.smithi138.stdout:To be equal to: ioctx.exec("myobject", "sdk", "test_coverage_write", in, out)
2017-04-28T22:01:38.683 INFO:tasks.workunit.client.0.smithi138.stdout:      Which is: -1
2017-04-28T22:01:38.683 INFO:tasks.ceph.osd.1.smithi138.stderr:2017-04-28 22:01:38.686067 7f9c65131700 -1 osd.1 98 class sdk open got (1) Operation not permitted
2017-04-28T22:01:38.686 INFO:tasks.workunit.client.0.smithi138.stdout:[  FAILED  ] ClsSDK.TestSDKCoverageWrite (2156 ms)
2017-04-28T22:01:38.686 INFO:tasks.workunit.client.0.smithi138.stdout:[ RUN      ] ClsSDK.TestSDKCoverageReplay
2017-04-28T22:01:40.920 INFO:tasks.workunit.client.0.smithi138.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.0.1-1708-gfa34740/rpm/el7/BUILD/ceph-12.0.1-1708-gfa34740/src/test/cls_sdk/test_cls_sdk.cc:3
0: Failure
2017-04-28T22:01:40.921 INFO:tasks.workunit.client.0.smithi138.stdout:      Expected: 0
2017-04-28T22:01:40.921 INFO:tasks.workunit.client.0.smithi138.stdout:To be equal to: ioctx.exec("myobject", "sdk", "test_coverage_write", in, out)
2017-04-28T22:01:40.921 INFO:tasks.workunit.client.0.smithi138.stdout:      Which is: -1
2017-04-28T22:01:40.921 INFO:tasks.ceph.osd.1.smithi138.stderr:2017-04-28 22:01:40.924241 7f9c65131700 -1 osd.1 100 class sdk open got (1) Operation not permitted
2017-04-28T22:01:40.923 INFO:tasks.workunit.client.0.smithi138.stdout:[  FAILED  ] ClsSDK.TestSDKCoverageReplay (2237 ms)
2017-04-28T22:01:40.923 INFO:tasks.workunit.client.0.smithi138.stdout:[----------] 2 tests from ClsSDK (4393 ms total)

/a/sage-2017-04-28_19:45:54-rados-wip-sage-testing---basic-smithi/1077690

I think we need to add sdk to the whitelist

liewegas · 2017-04-29T00:45:34Z

teuthology:1077690 12:44 AM $ cat summary.yaml
description: rados/basic/{clusters/{fixed-2.yaml openstack.yaml} mon_kv_backend/rocksdb.yaml
msgr-failures/few.yaml msgr/simple.yaml objectstore/bluestore-comp.yaml rados.yaml
tasks/rados_cls_all.yaml z-require-luminous/at-end.yaml}

so i would add an override to qa/suites/rados/basic/tasks/rados_cls_all.yaml with something like

override:
  ceph:
    conf:
      osd:
        ...

adding sdk to these options

OPTION(osd_class_load_list, OPT_STR, "cephfs hello journal lock log numops "
    "rbd refcount replica_log rgw statelog timeindex user version") // list of object classes allowed to be loaded (allow all: *)
OPTION(osd_class_default_list, OPT_STR, "cephfs hello journal lock log numops "
    "rbd refcount replica_log rgw statelog timeindex user version") // list of object classes with default execute perm (allow all: *)

Signed-off-by: Neha Ojha <nojha@redhat.com>

jdurgin

conf overrides look good

yuriw · 2017-05-03T16:29:07Z

http://pulpito.ceph.com/yuriw-2017-05-03_05:32:43-rados-wip-yuri-testing2_2017_5_3_2-distro-basic-smithi/

http://pulpito.ceph.com/yuriw-2017-05-03_14:43:05-rados-wip-yuri-testing2_2017_5_3_2---basic-smithi/

liewegas added the core label Apr 24, 2017

liewegas self-requested a review April 25, 2017 14:59

liewegas changed the title ~~DNM: Wip Objclass-sdk~~ DNM: objclass sdk Apr 25, 2017

liewegas added the needs-qa label Apr 25, 2017

jdurgin reviewed Apr 25, 2017

View reviewed changes

neha-ojha force-pushed the objclass-sdk branch from e844fcf to f5059e6 Compare April 26, 2017 00:07

liewegas approved these changes Apr 26, 2017

View reviewed changes

liewegas changed the title ~~DNM: objclass sdk~~ librados: objclass sdk Apr 26, 2017

liewegas changed the title ~~librados: objclass sdk~~ osd: objclass sdk Apr 26, 2017

liewegas added the wip-sage-testing label Apr 26, 2017

yuriw removed the wip-sage-testing label Apr 26, 2017

tchaikov reviewed Apr 27, 2017

View reviewed changes

tchaikov requested changes Apr 27, 2017

View reviewed changes

runsisi reviewed Apr 27, 2017

View reviewed changes

neha-ojha force-pushed the objclass-sdk branch from 8caded4 to 49f9cac Compare April 27, 2017 18:36

objclass-sdk: create SDK for Ceph object classes

b7215b0

Creates an installable version of "src/include/rados/objclass.h" that allows object classes to be built outside of the Ceph tree. cls_sdk is an example of such an object class. Signed-off-by: Neha Ojha <nojha@redhat.com>

neha-ojha force-pushed the objclass-sdk branch from 49f9cac to b7215b0 Compare April 27, 2017 20:08

liewegas added the wip-sage-testing label Apr 27, 2017

liewegas removed needs-qa wip-sage-testing labels Apr 29, 2017

objclass-sdk: add override to whitelist the sdk class

522bd84

Signed-off-by: Neha Ojha <nojha@redhat.com>

neha-ojha force-pushed the objclass-sdk branch from afb60e9 to 522bd84 Compare May 1, 2017 23:31

jdurgin approved these changes May 2, 2017

View reviewed changes

jdurgin added the needs-qa label May 2, 2017

tchaikov approved these changes May 2, 2017

View reviewed changes

yuriw added the wip-yuri2-testing label May 2, 2017

yuriw merged commit 1cce4c7 into ceph:master May 3, 2017

tchaikov mentioned this pull request Feb 12, 2016

Makefile: workaround an automake bug for "make check" #7626

Merged

neha-ojha deleted the objclass-sdk branch May 6, 2022 00:36

		@@ -718,6 +718,15 @@ This package contains the Java libraries for the Ceph File System.

		%endif

		%package -n rados-objclass-dev

osd: objclass sdk #14723

osd: objclass sdk #14723

Conversation

neha-ojha commented Apr 21, 2017 • edited

tchaikov commented Apr 24, 2017

tchaikov commented Apr 24, 2017

liewegas commented Apr 25, 2017

neha-ojha commented Apr 25, 2017

liewegas commented Apr 25, 2017 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jdurgin commented Apr 25, 2017

neha-ojha commented Apr 25, 2017

dotnwat commented Apr 26, 2017

liewegas commented Apr 26, 2017 via email

liewegas commented Apr 26, 2017

liewegas commented Apr 26, 2017

yuriw commented Apr 26, 2017

dotnwat commented Apr 26, 2017

liewegas commented Apr 26, 2017 via email

neha-ojha commented Apr 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tchaikov Apr 27, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tchaikov left a comment

Choose a reason for hiding this comment

liewegas commented Apr 27, 2017 via email

neha-ojha commented Apr 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liewegas commented Apr 27, 2017 via email

liewegas commented Apr 28, 2017

liewegas commented Apr 29, 2017

jdurgin left a comment

Choose a reason for hiding this comment

yuriw commented May 3, 2017

neha-ojha commented Apr 21, 2017 •

edited

tchaikov Apr 27, 2017 •

edited