New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test/osd/osd-scrub-repair.sh: Adjust for FreeBSD and ec_overwrite #15445

Merged
merged 1 commit into from Jun 26, 2017

Conversation

Projects
None yet
2 participants
@wjwithagen
Contributor

wjwithagen commented Jun 3, 2017

  • Do not test Bluestore OSDs on FreeBSD, since that does not work
    And all erasure code overwrite tests are executed on BlueStore OSDs
    Erasure code overwrites are unsafe on Filestore, see:
    docs

  • the JQ expression errors out with

    (version 1.5-1-g940132e-dirty)
    ====
    jq: error (at :232): Cannot iterate over null (null)
    Traceback (most recent call last):
    File "", line 1, in
    File "/usr/lib64/python2.7/json/init.py", line 338, in loads
    return _default_decoder.decode(s)
    File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib64/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
    ====
    Adding a ? to the jq expression allows to proceed on null blocks.

Signed-off-by: Willem Jan Withagen wjw@digiware.nl

@wjwithagen wjwithagen requested review from tchaikov and dzafman Jun 3, 2017

@wjwithagen wjwithagen added the tests label Jun 3, 2017

@@ -17,6 +17,12 @@
source $(dirname $0)/../detect-build-env-vars.sh
source $CEPH_ROOT/qa/workunits/ceph-helpers.sh
if [ `uname` = FreeBSD ]; then
use_bluestore=false

This comment has been minimized.

@tchaikov

tchaikov Jun 6, 2017

Contributor

could you add a comment explaining the connection between bluestore and overwrites on erasure-coded pool? and s/use_bluestore/have_ec_overwrite/.

This comment has been minimized.

@wjwithagen

wjwithagen Jun 6, 2017

Contributor

@tchaikov
This the left-over of a rather old PR.
So I'll have to check it. I saw more or less the same when redoing this PR.
But I think I remember that blue_store was actually the trigger for this all.
Got to check what happens when I do run these tests...

This comment has been minimized.

@wjwithagen

wjwithagen Jun 6, 2017

Contributor

@tchaikov
I do not know enough about erasure and/or the tests, but the overwrite tests actually use bluestore OSDs. So there is a relation between bluestore and overwrite.
But the problem is bluestore, so I'd still like to call the setting use_bluestore.
However I'll augment the commit and PR text.

This comment has been minimized.

@wjwithagen

This comment has been minimized.

@tchaikov

tchaikov Jun 13, 2017

Contributor

sorry, i missed this PR. will review it tomorrow.

This comment has been minimized.

@tchaikov

tchaikov Jun 20, 2017

Contributor

please see http://docs.ceph.com/docs/master/rados/operations/erasure-code/#erasure-coding-with-overwrites for more details. and you are disabling all ec_overwrite tests, so i think use_ec_overwrite accompanied with comment would be better.

This comment has been minimized.

@wjwithagen

wjwithagen Jun 20, 2017

Contributor

@tchaikov
Good pointer, thanx.

Will update the PR and commit with this info.
And I'll change the nameing.

@tchaikov tchaikov self-assigned this Jun 13, 2017

@wjwithagen wjwithagen changed the title from test/osd/osd-scrub-repair.sh: Adjust for FreeBSD to test/osd/osd-scrub-repair.sh: Adjust for FreeBSD and ec_overwrite Jun 20, 2017

@tchaikov tchaikov removed their assignment Jun 20, 2017

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jun 20, 2017

lgtm once jenkins is happy.

@wjwithagen

This comment has been minimized.

Contributor

wjwithagen commented Jun 21, 2017

Jenkins retest please

test/osd/osd-scrub-repair.sh: Adjust for FreeBSD
Fixes 2 problems:
 -  Do not test Bluestore on FreeBSD, since that does not work (yet)
    And all erasure code overwrite tests are executed on BlueStore OSDs
    Erasure code overwrites are unsafe on Filestore, see:
    http://docs.ceph.com/docs/master/rados/operations/erasure-code/#erasure-coding-with-overwrites

 -  the JQ expression errors out with:
    (version 1.5-1-g940132e-dirty)
    ====
    jq: error (at :232): Cannot iterate over null (null)
    Traceback (most recent call last):
    File "", line 1, in
    File "/usr/lib64/python2.7/json/init.py", line 338, in loads
    return _default_decoder.decode(s)
    File "/usr/lib64/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib64/python2.7/json/decoder.py", line 383, in raw_decode
    raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
    ====
    Adding a ? to the jq expression allows to proceed on null blocks.

Signed-off-by: Willem Jan Withagen <wjw@digiware.nl>
@wjwithagen

This comment has been minimized.

Contributor

wjwithagen commented Jun 23, 2017

@tchaikov
Tests seem oke, but only got bitten bij the fact that ceph_admin and restfull are not doing what they are suppossed to do. So I would suggest merge or QA

@tchaikov

This comment has been minimized.

Contributor

tchaikov commented Jun 23, 2017

retest this please.

@tchaikov tchaikov merged commit 9638fdb into ceph:master Jun 26, 2017

4 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
arm64 make check arm64 make check succeeded
Details
make check make check succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment