rgw: fix lc process only schdule the first item of lc objects #17061
Conversation
If more than two bucket hash to the same lc object, only the first bucket will do lifecycle operation. Other buckets lifecycle will never be schduled. Fixes: http://tracker.ceph.com/issues/21022 Signed-off-by: Shasha Lu <lu.shasha@eisoo.com>
| @@ -195,8 +195,10 @@ bool RGWLC::if_already_run_today(time_t& start_date) | |||
| localtime_r(&start_date, &bdt); | |||
|
|
|||
| if (cct->_conf->rgw_lc_debug_interval > 0) { | |||
| /* We're debugging, so say we can run */ | |||
| return false; | |||
| if (now - start_date < cct->_conf->rgw_lc_debug_interval) | |||
There was a problem hiding this comment.
I think if you want to make "lc process" process all items of lc objects multi-times a day, you can modify this configuration item from "rgw_lc_debug_interval" to "rgw_lc_process_interval". How do you think?
There was a problem hiding this comment.
It means this function is not just a debug function
There was a problem hiding this comment.
I think debug is more proper. If you set ExipationDays = 1, it means those objects will expire a day later. By setting rgw_lc_debug_interval, the objects will expire 1*rgw_lc_debug_interval. It just for debug.
Here just to make sure in a debug_interval bucket_lc_prepare only do once.
|
@cbodley Could you review this for me? Thanks |
|
@shashalu i wasn't familiar with this code, but your changes are making sense to me. can you recommend a way to test this? will the lifecycle expiration s3tests hit this bug if you set |
|
@cbodley I haven't run s3tests yet. But if set rgw_lc_max_objs = 1, put lifecycle configuration for more than 2 buckets, only one bucket can do expiration expectantly. |
|
works for me, can be tested with a script like this: #!/bin/bash
# assumes gnu parallel is installed, use a for loop otherwise we have a default
# lc num objects as 32, just create one more so that we have a guaranteed case
# of lc objects with multiple omap entries, assumes 10s as expiry interval
parallel s3cmd mb s3://bucket{} ::: {1..33}
touch foo
parallel s3cmd put foo s3://bucket{} ::: {1..33}
parallel s3cmd expire s3://bucket{} --expiry-days 1 --expiry-prefix foo ::: {1..33}
sleep 30
for i in {1..33}; do s3cmd ls s3://bucket$i; done |
|
@theanalyst thanks for the help testing! |
|
will need a luminous backport as well |
If more than two bucket hash to the same lc object, only the first bucket will do lifecycle operation. Other buckets lifecycle will never be schduled.
Fixes: http://tracker.ceph.com/issues/21022
Signed-off-by: Shasha Lu lu.shasha@eisoo.com