rgw: break sending data-log list infinitely #16926
Conversation
|
Hi @cbodley I saw a phenomenon that radosgw send data log list op to the opposite end infinitely, which cause the opposite end stuck with high cpu usage. |
fangyuxiangGL
force-pushed
the
infinitely-send-data-log-list
branch
2 times, most recently
from
97c61dd
to
923985b
Compare
|
Jenkins retest this please |
| @@ -1042,6 +1042,8 @@ class RGWDataSyncShardCR : public RGWCoroutine { | |||
| RGWDataChangesLogInfo shard_info; | |||
| string datalog_marker; | |||
|
|
|||
There was a problem hiding this comment.
don't need line spaces, I don't think
There was a problem hiding this comment.
@mattbenjamin thanks, addressed
There was a problem hiding this comment.
cool, this works great
src/rgw/rgw_data_sync.cc
Outdated
| #define INCREMENTAL_MAX_ENTRIES 100 | ||
| ldout(sync_env->cct, 20) << __func__ << ":" << __LINE__ << ": shard_id=" << shard_id << " datalog_marker=" << datalog_marker << " sync_marker.marker=" << sync_marker.marker << dendl; | ||
| if (datalog_marker > sync_marker.marker) { | ||
| spawned_keys.clear(); | ||
| if (sync_marker.marker.empty()) | ||
| remote_trimmed = 2; //remote data log shard might be trimmed; |
There was a problem hiding this comment.
are the 0..1 values of remote_trimmed actually ordinals? if not, I think they should be constants or enumeration values
There was a problem hiding this comment.
Not ordinals, I just used 2 to flag "might be trimmed", and 1 to flag "was trimmed", and 0 to flag "not trimmed"
There was a problem hiding this comment.
in that case, there should probably something like
enum class TrimState : uint8_t { NOT_TRIMMED, TRIMMED, MAYBE_TRIMMED } ;
or your preferred way of expressing this in c++
There was a problem hiding this comment.
(iif you want to keep the test for "remote trimmed" as written, you could do it static constexpr of the values as you wrote them rather than an enum)
There was a problem hiding this comment.
thanks, did you suggest that I use macro?
There was a problem hiding this comment.
@fangyuxiangGL sorry, I'd like to see us use some kind of compile-time constants--I was trying to say, it doesn't have to be an enum or enum class (in particular, enum class won't allow you to use the two non-zero vaues as a boolean without a case -I think-), but if you don't object, you could use that
fangyuxiangGL
force-pushed
the
infinitely-send-data-log-list
branch
from
923985b
to
9945ec9
Compare
src/rgw/rgw_data_sync.cc
Outdated
| @@ -1042,6 +1042,7 @@ class RGWDataSyncShardCR : public RGWCoroutine { | |||
| RGWDataChangesLogInfo shard_info; | |||
| string datalog_marker; | |||
|
|
|||
| int remote_trimmed; | |||
There was a problem hiding this comment.
^^ in my comments about using a symbolic constant, I meant to hold the values assigned to remote_trimmed
There was a problem hiding this comment.
ok , addressed it
fangyuxiangGL
force-pushed
the
infinitely-send-data-log-list
branch
from
9945ec9
to
1e1d5b1
Compare
|
Hi, changed it to enum type. thanks! |
|
ping @mattbenjamin |
radosgw send data-log list infinitely when opposite end trimmed the data-log and in quiescence. Fixes: http://tracker.ceph.com/issues/20951 Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com
fangyuxiangGL
force-pushed
the
infinitely-send-data-log-list
branch
from
1e1d5b1
to
40db1fd
Compare
|
@mattbenjamin if you're satisfied with changes, could you please update your review? |
No description provided.