Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mimic: core: mon,osd: limit MOSDMap messages by size as well as map count #29242

Merged
merged 3 commits into from Oct 3, 2019

Conversation

@smithfarm
Copy link
Contributor

smithfarm commented Jul 24, 2019

@smithfarm smithfarm self-assigned this Jul 24, 2019
@smithfarm smithfarm added this to the mimic milestone Jul 24, 2019
@smithfarm smithfarm requested review from tchaikov and liewegas Jul 24, 2019
@smithfarm smithfarm changed the title mimic: osd_map_message_max default is too high? mimic: core: osd_map_message_max default is too high Jul 24, 2019
@smithfarm smithfarm changed the title mimic: core: osd_map_message_max default is too high mimic: core: mon,osd: limit MOSDMap messages by size as well as map count Jul 24, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 24, 2019

In file included from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/dout.h:22:0,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/common/debug.h:18,
                 from /home/jenkins-build/build/workspace/ceph-pull-requests/src/compressor/Compressor.cc:24:
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/legacy_config_opts.h:607:8: error: 'osd_map_message_max_bytes' has not been declared
 OPTION(osd_map_message_max_bytes, OPT_SIZE)  // max maps per MOSDMap message
        ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config.h:362:17: note: in definition of macro 'OPTION'
     OPTION_##ty(name)
                 ^~~~
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config.h:362:21: error: ISO C++ forbids declaration of 'OPTION_OPT_SIZE' with no type [-fpermissive]
     OPTION_##ty(name)
                     ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/legacy_config_opts.h:607:1: note: in expansion of macro 'OPTION'
 OPTION(osd_map_message_max_bytes, OPT_SIZE)  // max maps per MOSDMap message
 ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/config.h:362:21: error: expected ';' at end of member declaration
     OPTION_##ty(name)
                     ^
/home/jenkins-build/build/workspace/ceph-pull-requests/src/common/legacy_config_opts.h:607:1: note: in expansion of macro 'OPTION'
 OPTION(osd_map_message_max_bytes, OPT_SIZE)  // max maps per MOSDMap message
 ^
src/compressor/CMakeFiles/compressor_objs.dir/build.make:62: recipe for target 'src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o' failed
make[3]: *** [src/compressor/CMakeFiles/compressor_objs.dir/Compressor.cc.o] Error 1
Limit MOSDMap message size by bytes as well as map count.

Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit e15be00)

Conflicts:
	src/common/config_values.h
- this file does not exist in mimic; apply change to src/common/config.h instead
- size_t -> uint64_t
@smithfarm smithfarm force-pushed the smithfarm:wip-38277-mimic branch from c84f6a3 to 5e46f45 Jul 24, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 24, 2019

/home/jenkins-build/build/workspace/ceph-pull-requests/src/mon/OSDMonitor.cc: In member function 'virtual void OSDMonitor::encode_pending(MonitorDBStore::TransactionRef)':
/home/jenkins-build/build/workspace/ceph-pull-requests/src/mon/OSDMonitor.cc:1071:31: warning: comparison of integer expressions of different signedness: 'std::vector<pg_t>::size_type' {aka 'long unsigned int'} and 'int64_t' {aka 'long int'} [-Wsign-compare]
       if (pgs_to_check.size() < g_conf->mon_clean_pg_upmaps_per_chunk * 2) {
           ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/jenkins-build/build/workspace/ceph-pull-requests/src/mon/OSDMonitor.cc: In member function 'bool OSDMonitor::preprocess_get_osdmap(MonOpRequestRef)':
/home/jenkins-build/build/workspace/ceph-pull-requests/src/mon/OSDMonitor.cc:2166:30: error: 'g_conf' cannot be used as a function
   ssize_t max_bytes = g_conf()->osd_map_message_max_bytes;
                              ^
src/mon/CMakeFiles/mon.dir/build.make:182: recipe for target 'src/mon/CMakeFiles/mon.dir/OSDMonitor.cc.o' failed
make[3]: *** [src/mon/CMakeFiles/mon.dir/OSDMonitor.cc.o] Error 1
liewegas added 2 commits Feb 8, 2019
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit dcf1cd0)

Conflicts:
	src/mon/OSDMonitor.cc
- g_conf(), ceph_assert
We restructure the function significantly to let us work forwards instead
of backwards through epochs.  We also make the assumption that the OSD
will have the maps it is supposed to have.  If we for some reason fail to
load a map, we fall back to something minimal, but in general there is
little point to sending a more complete message when local maps are missing
since it shouldn't ever happen anyway and the receiver can always go get
maps from the mon.

Fixes: http://tracker.ceph.com/issues/38040
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit 47a3f63)
@smithfarm smithfarm force-pushed the smithfarm:wip-38277-mimic branch from 5e46f45 to 57c4e5b Jul 24, 2019
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 24, 2019

157/157 Test   #3: test_objectstore_memstore.sh ............***Timeout 3600.01 sec
@smithfarm

This comment has been minimized.

Copy link
Contributor Author

smithfarm commented Jul 24, 2019

jenkins test make check

@yuriw

This comment has been minimized.

Copy link
Contributor

yuriw commented Aug 5, 2019

@dzafman
dzafman approved these changes Oct 3, 2019
@yuriw yuriw merged commit 1720691 into ceph:mimic Oct 3, 2019
4 checks passed
4 checks passed
Docs: build check OK - docs built
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details
@smithfarm smithfarm deleted the smithfarm:wip-38277-mimic branch Oct 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.