Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

librados: support copy-less writes in librados C API #24941

Closed
wants to merge 1,469 commits into from
Closed

librados: support copy-less writes in librados C API #24941

wants to merge 1,469 commits into from

Conversation

liyichao
Copy link
Contributor

@liyichao liyichao commented Nov 6, 2018

Based on work #12216 , and fix the problem as mentioned in yuyuyu101@794b49b#r15707924 .

  • Cancel ops to the old osd when ops are redirected, so buffer will not be used when it is freed.
  • out_seq is only updated for new messages, and messages from sent queue are sent using their original sequence. So even when we remove ops from sent queue, the out_seq is correct.

John Spray and others added 30 commits November 1, 2017 23:03
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 3048e85)
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 4f7007d)
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7b629ae)
These would throw an exception when passed
a status code.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit e2442c1)
...by using PyModuleRunner class from ActivePyModule too.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit df87973)
Was passing a reference to a local stringstream into
Thread::create, not realising that it was taking a char*
reference instead of a copy.  Result was garbage (or usually,
all threads having the name of the last one created)

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit bb4e71e)
Was calling way too early, which did a
Py_Finalize before the modules had been
joined.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 0d5b1d7)
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit eacc902)
Implement real python classes from the C side,
rather than exposing only module methods.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 7e61f79)
A bunch of the previous commits were done
before this class existed, so updating in
one go instead of trying to edit history
in fine detail.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 29193a4)
This was doing a Py_DECREF outside of the Gil.

Fixes: http://tracker.ceph.com/issues/21593
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 58dfa97)
These didn't need to keep the GIL to go and do their
pure C++ parts, and by keeping it they could deadlock
while trying to take ActiveMgrModules::lock.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 27ee148)
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit a382c3f)
Some extra coverage of the dashboard, including its standby
redirect mode and the publishing of URIs.

Also invoking the command_spam mode of the selftest module.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 05e648b)
This was still using Kraken era settings

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 9988ebe)
Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit ceb2a91)
...including the new "mgr services" command.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit d220e1a)
Otherwise, when someone wants to see what's possible
to do with `mgr module enable` they have to trawl
through the whole mgr map dump.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 5861c4c)
This was converted to options.cc only, but we cannot
backport another commit that removes this legacy health
reporting code, so this commit updates the legacy health
reporting code for the options.cc style config opt.

Signed-off-by: John Spray <john.spray@redhat.com>
This was getting the definition some other way in master,
but in luminous we need to include the backport14 header.

Signed-off-by: John Spray <john.spray@redhat.com>
I inadvertently broke this with the latest change
to the module ls output.

Signed-off-by: John Spray <john.spray@redhat.com>
(cherry picked from commit 4fb3025)
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 45e4b0c)
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit 3837289)
This also updates the ansible and testinfra version to match the current
ceph-ansible master branch.

Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit b49a19a)
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
(cherry picked from commit f8b114b)
luminous: Ubuntu amd64 client can not discover the ubuntu arm64 ceph cluster

Reviewed-by: Kefu Chai <kchai@redhat.com>
luminous: "ceph-volume: adds functional testing for bluestore"

Reviewed-by: Alfredo Deza <adeza@redhat.com>
luminous: bulk backport of ceph-mgr improvements
luminous: selinux: Allow getattr on lnk sysfs files

Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
…d-partitions-luminous

luminous: ceph-disk: unlocks dmcrypted partitions when activating them

Reviewed-by: Nathan Cutler <ncutler@suse.com>
smithfarm and others added 29 commits November 26, 2017 14:47
Very old, and very new, versions of flake8 treat E722 as an error:

flake8 runtests: commands[0] | flake8 --ignore=H105,H405,E127 ceph_disk tests
ceph_disk/main.py:1575:9: E722 do not use bare except'
ceph_disk/main.py:1582:9: E722 do not use bare except'
ceph_disk/main.py:3252:5: E722 do not use bare except'
ceph_disk/main.py:3288:21: E722 do not use bare except'
ceph_disk/main.py:3296:17: E722 do not use bare except'
ceph_disk/main.py:4358:5: E722 do not use bare except'
tests/test_main.py:26:1: E722 do not use bare except'
ERROR: InvocationError: '/opt/j/ws/mkck/src/ceph-disk/.tox/flake8/bin/flake8 --ignore=H105,H405,E127 ceph_disk tests'

Fixes: http://tracker.ceph.com/issues/22207
References: https://gitlab.com/pycqa/flake8/issues/361

Signed-off-by: Nathan Cutler <ncutler@suse.com>
(cherry picked from commit 3600cd7)
rpm expands all macros in a .spec file, even those in comments. Drop the
percent signs so rpm will not expand these.

This change silences rpmlint's warning about macros in comments.

Signed-off-by: Ken Dreyer <kdreyer@redhat.com>
(cherry picked from commit 5c1e9f5)
… be old

Early point release mons don't handle legacy ruleset-* ec profiles, new
ones do.  Skip the ec tests that may trigger this when we are doing a
workload that races with mon upgrades.

Signed-off-by: Sage Weil <sage@redhat.com>
qa/suites/upgrade/jewel-x/point-to-point: skip ec tests when mons may be old

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
...and add more health whitelists.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit ddb2746)
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
qa/suites/rados/rest: move rest_test from qa/suites/rest/

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
luminous: build/ops: macros expanding in spec file comment

Reviewed-by: David Disseldorp <ddiss@suse.com>
Reviewed-by: Ken Dreyer <kdreyer@redhat.com>
100MB will be allocated for journal, and the remaining 100MB is for data
device. taking the inode into consideration, there will be approximately
87988 kB available for the activated OSD. and it will complain with a
"nearfull" state.

Fixes: http://tracker.ceph.com/issues/22136
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b3c159e)
luminous: ceph-disk flake8 test fails on very old, and very new, versions of flake8

Reviewed-by: Kefu Chai <kchai@redhat.com>
luminous: qa/ceph-disk: enlarge the simulated SCSI disk

Reviewed-by: Alfredo Deza <adeza@redhat.com>
… warning

It's generated on the mgr.

Signed-off-by: Sage Weil <sage@redhat.com>
qa/suites/upgrade/jewel-x/point-to-point: fix suppression of pool app warning
Signed-off-by: Sage Weil <sage@redhat.com>
Otherwise we do 4 mounts for no good reason.

Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
Previous code forgot to multiple raw_used_ratio to calculate
used byte.

Fixes: http://tracker.ceph.com/issues/22247
Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
(cherry picked from commit d10c6c2)
Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit 8d22856)
…e flags

Signed-off-by: Alfredo Deza <adeza@redhat.com>
(cherry picked from commit f0ac2df)
Fixes: http://tracker.ceph.com/issues/22266
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit f63d1da)
luminous: ceph-volume correctly fallback to bluestore when no objectstore is specified

Reviewed-by: Andrew Schoen <aschoen@redhat.com>
luminous: mon/PGMap: Fix %USED calculation

Reviewed-by: Greg Farnum <gfarnum@redhat.com>
luminous: tools/ceph_monstore_tool: rebuild initial mgrmap also

Reviewed-by: Abhishek Lekshmanan <abhishek@suse.com>
Signed-off-by: wanglilaurence <wanglilaurence@didichuxing.com>
Signed-off-by: liyichao <liyichao.good@gmail.com>
Output latency perfcounters.

See merge request !3
@liyichao liyichao closed this Nov 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet