Skip to content
Permalink
Browse files

osd: make do_cache_redirect can work.

If cache tier call do_cache_redirect, the clint don't add
CEPH_OSD_FLAG_IGNORE_OVERLAY. it make op endless.

Signed-off-by: Jianpeng Ma <jianpeng.ma@intel.com>
  • Loading branch information
majianpeng committed Mar 4, 2015
1 parent 01522e1 commit 2f0518d6da9d7f9d9fd109c0848eb422ad05264d
Showing with 1 addition and 1 deletion.
  1. +1 −1 src/osdc/Objecter.cc
@@ -2917,7 +2917,7 @@ void Objecter::handle_osd_op_reply(MOSDOpReply *m)
op->tid = 0;
m->get_redirect().combine_with_locator(op->target.target_oloc,
op->target.target_oid.name);
op->target.flags |= CEPH_OSD_FLAG_REDIRECTED;
op->target.flags |= CEPH_OSD_FLAG_REDIRECTED|CEPH_OSD_FLAG_IGNORE_OVERLAY;
_op_submit(op, lc);
m->put();
return;

8 comments on commit 2f0518d

@liewegas

This comment has been minimized.

Copy link

liewegas replied Mar 5, 2015

The IGNORE_OVERLAY flag is only used inside Objecter.cc in _calc_target, but only if target_oid isn't filled in.. which the previous line here does. Is this something you saw actually happen?

@majianpeng

This comment has been minimized.

Copy link
Owner Author

majianpeng replied Mar 5, 2015

For forward mode, the op will never end(in this loop: client->send cache->cache redirect to client-->client, i don't know why this don't find)
At firstly, i want use IGNORE_CACHE. But in _calc_target don't take into account this.
BTY, how to choose IGNORE_CACHE or IGNORE_OVERLAY? Thanks!

@liewegas

This comment has been minimized.

Copy link

liewegas replied Mar 5, 2015

@majianpeng

This comment has been minimized.

Copy link
Owner Author

majianpeng replied Mar 5, 2015

No, in clean branch, my steps are following:
ceph osd tier add ecpool rbd
ceph osd tier cache-mode rbd forward
ceph osd tier set-overlay ecpool rbd

rados -p ecpool put intel /tmp/foo

this cli don't end up.
I saw the code about do_cache_redirect. It don't change anything and so the next send MOSDOp still send to cache tier. Or am i miss something?

@liewegas

This comment has been minimized.

Copy link

liewegas replied Mar 5, 2015

oooh the bug is that this

  if (osdmap->get_epoch() == pi->last_force_op_resend) {
    force_resend = true;
  }

it always true until the next osdmap change. if that doesn't happen (as in your simple test cluster) it will loop. i fyou ceph osd set nodown ; ceph osd unset nodown or something similar it will stop looping.

have to make it so each request will only resend if the previous send attempt was on a prior epoch, i think!

@liewegas

This comment has been minimized.

Copy link

liewegas replied Mar 5, 2015

@majianpeng

This comment has been minimized.

Copy link
Owner Author

majianpeng replied Mar 5, 2015

I see this commit dd700bd which add force_resend. why remove this case for force_resend. like:
if (osdmap->get_epoch() == pi->last_force_op_resend &&
!(t->flags & CEPHOSD_FLAG_REDIRECTED)) {
force_resend = true;
}

@liewegas

This comment has been minimized.

Copy link

liewegas replied Mar 5, 2015

i think that's not quite right because if the op has been redirected and then we receive the next osdmap we still need to resend. i think what we need is a epoch_t last_force_resend; in Op and make that

if (osdmap->get_epoch() == pi->last_force_op_resend && op->last_force_op_resend < pi->last_force_op_resend) {
op->last_force_op_resend = ...

Please sign in to comment.
You can’t perform that action at this time.