Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hammer: tools: add a tool to rebuild mon store from OSD #11125

Merged
merged 11 commits into from
Nov 23, 2016
Merged

hammer: tools: add a tool to rebuild mon store from OSD #11125

merged 11 commits into from
Nov 23, 2016

Conversation

tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Sep 19, 2016 •
edited by smithfarm
Loading

@tchaikov tchaikov added this to the hammer milestone Sep 19, 2016
@tchaikov tchaikov changed the title Wip 17179 hammer hammer: add a tool to rebuild mon store from OSD Sep 19, 2016
@tchaikov tchaikov changed the title hammer: add a tool to rebuild mon store from OSD [DNM] hammer: add a tool to rebuild mon store from OSD Sep 30, 2016
@tchaikov tchaikov force-pushed the wip-17179-hammer branch 2 times, most recently from 1cf474f to e09a10d Compare September 30, 2016 11:46
@tchaikov tchaikov changed the title [DNM] hammer: add a tool to rebuild mon store from OSD hammer: add a tool to rebuild mon store from OSD Sep 30, 2016
@tchaikov tchaikov changed the title hammer: add a tool to rebuild mon store from OSD [DNM] hammer: add a tool to rebuild mon store from OSD Oct 10, 2016
@tchaikov
Copy link
Contributor Author

tchaikov commented Oct 10, 2016 •
edited
Loading

will update the x-refs of commits not merged in master yet.

@tchaikov tchaikov changed the title [DNM] hammer: add a tool to rebuild mon store from OSD hammer: add a tool to rebuild mon store from OSD Oct 18, 2016
@tchaikov
Copy link
Contributor Author

changelog

  • fix the xrefs so they point to the corresponding commits in master

@smithfarm smithfarm self-assigned this Oct 23, 2016
@smithfarm
Copy link
Contributor

@tchaikov In all likelihood, 0.94.10 will be the last hammer release. If we include this new tool, and it has bugs, there won't be another release in which to fix those (admittedly hypothetical) bugs. If that's OK with you, then I'm in favor of inclusion since I'm sure the new tool is useful.

@smithfarm smithfarm changed the title hammer: add a tool to rebuild mon store from OSD hammer: tools: add a tool to rebuild mon store from OSD Oct 23, 2016
@tchaikov
Copy link
Contributor Author

@smithfarm, this tool has been tested on jewel and master, so it should work well. i am okay to include it in this hammer release.

@smithfarm
Copy link
Contributor

@tchaikov Can we ignore the new warning "This branch is out-of-date with the base branch" ?

@tchaikov tchaikov mentioned this pull request Nov 12, 2016
Merged
@smithfarm
Copy link
Contributor

@tchaikov Thanks, updated tracker and PR description.

smithfarm added a commit that referenced this pull request Nov 18, 2016
@smithfarm
Merge pull request #11125: hammer: tools: add a tool to rebuild mon s…
caea8cb 
…tore from OSD

Reviewed-by: Nathan Cutler <ncutler@suse.com>
smithfarm added a commit that referenced this pull request Nov 20, 2016
@smithfarm
Merge pull request #11125: hammer: tools: add a tool to rebuild mon s…
07b90b1 
…tore from OSD

Reviewed-by: Nathan Cutler <ncutler@suse.com>
@smithfarm
Copy link
Contributor

@tchaikov This backport passed a rados suite at http://tracker.ceph.com/issues/17151#note-7 with failures that I believe have been addressed (except for http://tracker.ceph.com/issues/15139 which is caused by the build system no longer providing dumpling-era packages).

I have rebuilt the integration branch to include the two fixes and scheduled a new run at http://tracker.ceph.com/issues/17151#note-14

Do you think it's OK to merge provided the second run succeeds?

@smithfarm
Copy link
Contributor

@tchaikov Also, could you provide the text of a hammer 0.94.10 release notes entry describing this new tool?

@tchaikov tchaikov self-assigned this Nov 20, 2016
smithfarm added a commit that referenced this pull request Nov 21, 2016
@smithfarm
Merge pull request #11125: hammer: tools: add a tool to rebuild mon s…
d43e74f 
…tore from OSD

Reviewed-by: Nathan Cutler <ncutler@suse.com>
smithfarm added a commit that referenced this pull request Nov 21, 2016
@smithfarm
Merge pull request #11125: hammer: tools: add a tool to rebuild mon s…
918f597 
…tore from OSD

Reviewed-by: Nathan Cutler <ncutler@suse.com>
@smithfarm
Copy link
Contributor

Second run succeeded; third run here: http://pulpito.front.sepia.ceph.com/smithfarm-2016-11-21_11:52:41-rados-hammer-backports---basic-smithi/

@tchaikov
Copy link
Contributor Author

Also, could you provide the text of a hammer 0.94.10 release notes entry describing this new tool?

@smithfarm sure thing! will do it tmr.

tchaikov and others added 11 commits November 22, 2016 11:47
@tchaikov
mon/AuthMonitor: make AuthMonitor::IncType public
e7d9c15 
so ceph-objectstore-tool is able to use it when rebuilding monitor
db.

Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 19ef4f1)
@tchaikov
tools/ceph-objectstore-tool: add "update-mon-db" command
6695d17 
Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 24faea7)

Conflicts:
	src/tools/Makefile-server.am: add rebuild_mondb.{cc,h}, as autotools
          support is dropped in master
	ceph_objectstore_tool.cc, rebuild_mondb.cc: remove C++11ish
          adapt to hammer
@tchaikov
tools/ceph_monstore_tool: add "rebuild" command
3ef5dcd 
Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit d909fa0)

Conflicts:
	ceph_monstore_tool.cc: remove C++11ish
@tchaikov
doc: add rados/operations/disaster-recovery.rst
828a4d7 
document the process to recover from leveldb corruption.

Fixes: http://tracker.ceph.com/issues/17179
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit 79a9f29)
@xiexingguo @tchaikov
tools/rebuild_mondb: avoid unnecessary result code cast
eaeb520 
In general we return negative codes for error cases, so there is
no need perform the cast here.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit 6a1c01d)
@xiexingguo @tchaikov
tools/rebuild_mondb: return error if ondisk version of pg_info is inc…
94390a3 
…ompatible

In this case "r" will be >= 0 and caller will be able to catch
this kind of error.

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit ac7c1dc)
@xiexingguo @tchaikov
tools/rebuild_mondb: kill compiling warning
ca67e8e 
As follow:

[ 72%] Building CXX object src/tools/CMakeFiles/ceph-objectstore-tool.dir/RadosDump.cc.o
/home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/rebuild_mondb.cc: In function ‘int update_mon_db(ObjectStore&, OSDSuperblock&, const string&, const string&)’:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/rebuild_mondb.cc:289:22: warning: ‘crc’ may be used uninitialized in this function [-Wmaybe-uninitialized]
         if (have_crc && osdmap.get_crc() != crc) {
                      ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/tools/rebuild_mondb.cc:238:14: note: ‘crc’ was declared here
     uint32_t crc;

Signed-off-by: xie xingguo <xie.xingguo@zte.com.cn>
(cherry picked from commit f16a314)
@tchaikov
tools/ceph_monstore_tool: update pgmap_meta also when rebuilding stor…
48696e9 
…e.db

we should rebuild pgmap_meta table from the collected osdmaps

Fixes: http://tracker.ceph.com/issues/17400
Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit cdfa7a6)
Conflicts:
	src/tools/ceph_monstore_tool.cc: remove C++11-ism
@tchaikov
tools/ceph_monstore_tool: bail out if no caps found for a key
9c00ba1 
we take it as an error if no caps is granted to an entity in the
specified keyring file when rebuilding the monitor db.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit b4bd400)
@tchaikov
doc: fill keyring with caps before passing it to ceph-monstore-tool
9473135 
to make sure the recovered monitor store is ready for use.

Signed-off-by: Kefu Chai <kchai@redhat.com>
(cherry picked from commit af8e211)
@tchaikov
PendingReleaseNotes: document the feature to rebuild monstore
10472f0 
Signed-off-by: Kefu Chai <kchai@redhat.com>
@tchaikov
Copy link
Contributor Author

@smithfarm done, and rebased against master.

@tchaikov tchaikov removed their assignment Nov 22, 2016
@smithfarm smithfarm merged commit 48b6054 into ceph:hammer Nov 23, 2016
@hjwsm1989 hjwsm1989 mentioned this pull request Feb 24, 2017
Closed
@tchaikov tchaikov deleted the wip-17179-hammer branch February 24, 2017 02:09
hjwsm1989 added a commit to hjwsm1989/ceph that referenced this pull request Feb 24, 2017
@hjwsm1989
ceph_monstore_tool: fix the illegal memory access when update_paxos
3cfa68b 
  Conflicts:
      we don't use C++11 in hammer. and this fix addresses the issue introduced by ceph#11125,
      where we are iterating in a plain array instead of a std::vector in master. so this change is not cherry-picked from master.

  Signed-off-by: huangjun <hjwsm1989@gmail.com>
hjwsm1989 added a commit to hjwsm1989/ceph that referenced this pull request Feb 24, 2017
@hjwsm1989
ceph_monstore_tool: fix the illegal memory access when update_paxos
8a56ac5 
  Conflicts:
      we don't use C++11 in hammer. and this fix addresses the issue introduced by ceph#11125,
      where we are iterating in a plain array instead of a std::vector in master. so this change is not cherry-picked from master.

  Fixes: http://tracker.ceph.com/issues/19070
  Signed-off-by: huangjun <hjwsm1989@gmail.com>
tchaikov pushed a commit to tchaikov/ceph that referenced this pull request Mar 28, 2018
@hjwsm1989 @tchaikov
ceph_monstore_tool: fix the illegal memory access when update_paxos
db5bda5 
  Conflicts:
      we don't use C++11 in hammer. and this fix addresses the issue introduced by ceph#11125,
      where we are iterating in a plain array instead of a std::vector in master. so this change is not cherry-picked from master.

  Fixes: http://tracker.ceph.com/issues/19070
  Signed-off-by: huangjun <hjwsm1989@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@smithfarm smithfarm

Labels
feature tools
Projects
None yet
Milestone
hammer
Development

Successfully merging this pull request may close these issues.
3 participants
@tchaikov @smithfarm @xiexingguo