Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hammer: osd: ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) #5825

Merged
10 commits merged into from Nov 10, 2015

Conversation

tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Sep 6, 2015

@tchaikov tchaikov assigned ghost Sep 6, 2015
@tchaikov tchaikov added this to the hammer milestone Sep 6, 2015
@ghost
Copy link

ghost commented Sep 6, 2015

@tchaikov is there an upgrade test that verifies the gmt feature ? I guess what we want is to check that hammer-x with this patch in hammer will upgrade well to infernalis.

@ghost ghost changed the title Wip 12848 hammer ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Sep 6, 2015
@tchaikov
Copy link
Contributor Author

tchaikov commented Sep 6, 2015

@dachary i am testing the upgrade test at http://pulpito.ceph.com/kchai-2015-09-06_03:41:25-upgrade:hammer-x-master---basic-vps/ . will update this pr with the test result.

@tchaikov
Copy link
Contributor Author

tchaikov commented Sep 6, 2015

@dachary the test passed. but the test was designed so that hammer without bitwise sort and gmt fix can be upgraded to specified branch (with sortbitwise and gmt fix). i will remove following code and post a pull request (ceph/ceph-qa-suite#554) to ceph-qa-suite/master

          ceph set pool set cache-pool use_gmt_hitset true 2>&1 | grep "not all OSDs support GMT hit set"

after this pr is merged.

if hammer branch supports gmt hitset, the hitset archives will be upgraded to infernalis naturally.

i will also prepare a test for firefly => hammer upgrade as suggested by @athanatos at http://tracker.ceph.com/issues/9732#note-11 .

@ghost
Copy link

ghost commented Sep 6, 2015

@tchaikov that makes sense, thanks for the detailed explanation :-)

@tchaikov
Copy link
Contributor Author

tchaikov commented Sep 6, 2015

the tests is posted at ceph/ceph-qa-suite#554

@tchaikov tchaikov changed the title ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) [DNM] ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Sep 8, 2015
@tchaikov
Copy link
Contributor Author

tchaikov commented Sep 8, 2015

please hold until #5837 is merged. and then we will backport that pr, and put it into this one.

@tchaikov tchaikov assigned tchaikov and ghost and unassigned ghost and tchaikov Sep 8, 2015
@tchaikov tchaikov changed the title [DNM] ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Sep 9, 2015
@tchaikov
Copy link
Contributor Author

tchaikov commented Sep 9, 2015

added the backport from #5837

@tchaikov
Copy link
Contributor Author

rebased against ceph.git/hammer, and pushed to ceph.git/wip-12848-hammer to test using the upgrade test suite

@ghost ghost changed the title ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) DNM: ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Oct 2, 2015
@ghost
Copy link

ghost commented Oct 2, 2015

@tchaikov when merged with #6147 it fails because the prototype changed. I think it would make sense to merge #6147 into this series. If I'm not mistaken, it happens before and will reduce the number of conflicts you had to resolve. What do you think ?

@tchaikov
Copy link
Contributor Author

tchaikov commented Oct 9, 2015

@dachary let me give it a try: merge #6147 into this pr.

@ghost ghost removed their assignment Oct 9, 2015
@ghost ghost assigned tchaikov Oct 9, 2015
@ghost
Copy link

ghost commented Oct 9, 2015

@tchaikov ack :-)

tchaikov and others added 10 commits October 9, 2015 03:20
* bump the encoding version of pg_hit_set_info_t to 2, so we can
  tell if the corresponding hit_set is named using localtime or
  GMT
* bump the encoding version of pg_pool_t to 20, so we can know
  if a pool is using GMT to name the hit_set archive or not. and
  we can tell if current cluster allows OSDs not support GMT
  mode or not.
* add an option named `osd_pool_use_gmt_hitset`. if enabled,
  the cluster will try to use GMT mode when creating a new pool
  if all the the up OSDs support GMT mode. if any of the
  pools in the cluster is using GMT mode, then only OSDs
  supporting GMT mode are allowed to join the cluster.

Fixes: ceph#9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 42f8c5d)

 Conflicts:
	src/include/ceph_features.h
	src/osd/ReplicatedPG.cc
	src/osd/osd_types.cc
	src/osd/osd_types.h
		fill pg_pool_t with default settings in master branch.
allow "ceph osd pool set $pool use_gmt_hitset <true|1>" as long as
the cluster supports gmt hitset.

Fixes: ceph#9732
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 03a1a3c)
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cc2bcf7)

Conflicts:
	src/mon/OSDMonitor.cc
		use the old way to dump pool info
Signed-off-by: David Zafman <dzafman@redhat.com>
(cherry picked from commit 2bc5a48)

Conflicts:
	src/osd/osd_types.cc
		minor changes to resolve conflicts
* to ease the backport to hammer

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 5a4f6a8)

Conflicts:
	src/include/ceph_features.h
		minor changes to resolve conflicts
the gmt_hitset is enabled by default in the ctor of pg_pool_t, this
is intentional. because we want to remove this setting and make
gmt_hitset=true as a default in future. but this forces us to
disable it explicitly when preparing a new pool if any OSD does
not support gmt hitset.

Fixes: ceph#12968
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit e48cec3)

Conflicts:
	src/mon/OSDMonitor.cc
		remove unrelated hunk
When hit set is not configured on startup or on a change, remove all
previous hitsets.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit be28319)
Conflicts:
        src/osd/ReplicatedPG.cc
          pass the new argument of `p->using_gmt` to get_hit_set_archive_object()
Only the primary PG is allowed to remove all the hit set objects. And
the PG should be in the active or peered states.

Signed-off-by: Zhiqiang Wang <zhiqiang.wang@intel.com>
(cherry picked from commit fd38902)
Otherwise impl is unpopulated and we risk segfaults in several
methods.  It also seems like it would always indicate a bug.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 8e5a801)
We only want to do it if the pool config changed AND we are
primary && active.

fd38902 partially fixed a related
bug.  This should be backported along with it.

Fixes: 13192
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 36e4a80)
@tchaikov tchaikov changed the title DNM: ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Oct 9, 2015
@tchaikov tchaikov assigned ghost and unassigned tchaikov Oct 9, 2015
ghost pushed a commit that referenced this pull request Nov 10, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
@ghost ghost merged commit 3954471 into ceph:hammer Nov 10, 2015
@tchaikov tchaikov deleted the wip-12848-hammer branch November 17, 2015 09:05
ghost pushed a commit that referenced this pull request Nov 17, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost pushed a commit that referenced this pull request Nov 17, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost pushed a commit that referenced this pull request Nov 17, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost pushed a commit that referenced this pull request Nov 17, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost pushed a commit that referenced this pull request Nov 17, 2015
ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc)

Reviewed-by: Loic Dachary <ldachary@redhat.com>
ghost pushed a commit that referenced this pull request Nov 18, 2015
This reverts commit 3954471, reversing
changes made to 4ad9716.
@ghost ghost changed the title ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) hammer: osd: ReplicatedPG::hit_set_trim osd/ReplicatedPG.cc: 11006: FAILED assert(obc) Feb 18, 2016
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants