Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crimson/osd/pg: reset the snap mapper's backend when pg interval changes #57125

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

xxhdx1985126
Copy link
Contributor

Fixes: https://tracker.ceph.com/issues/65679
Signed-off-by: Xuehan Xu xuxuehan@qianxin.com

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@Matan-B
Copy link
Contributor

Matan-B commented Apr 30, 2024

DEBUG active+clean+snaptrim  ObjectContextLoader::get_or_load_obc: cache hit on 3:6bb7f7ea:::scephqa03.cpp.bjat.qianxin-inc.cn204052-10:head
/src/crimson/osd/object_context_loader.cc:108:  Assertion `cit != std::end(ss.clones)' failed.

Can you please explain how this assert relates to the stale cache?

Copy link
Contributor

@athanatos athanatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than removing the ifdef's

@@ -85,6 +85,12 @@ class MapCacher {
public:
MapCacher(StoreDriver<K, V> *driver) : driver(driver) {}

#ifdef WITH_SEASTAR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behavior is not specific to seastar, remove the ifdef.

@@ -61,6 +61,13 @@ class SharedPtrRegistry {
waiting(0)
{}

#ifdef WITH_SEASTAR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behavior is not specific to seastar, remove the ifdef.

@@ -356,6 +356,13 @@ class SnapMapper : public Scrub::SnapMapReaderI {
return prefix_itr;
}

#ifdef WITH_SEASTAR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This behavior is not specific to seastar, remove the ifdef.

@xxhdx1985126
Copy link
Contributor Author

xxhdx1985126 commented May 8, 2024

DEBUG active+clean+snaptrim  ObjectContextLoader::get_or_load_obc: cache hit on 3:6bb7f7ea:::scephqa03.cpp.bjat.qianxin-inc.cn204052-10:head
/src/crimson/osd/object_context_loader.cc:108:  Assertion `cit != std::end(ss.clones)' failed.

Can you please explain how this assert relates to the stale cache?

This is a little bit tricky.

  1. Snaps snap1 and snap2 on object obj1 maps to the same clone c
  2. Snap snap1 is being trimmed on PG P;
  3. Before snap1 of obj1 is trimmed and after the obj1 <-> snap1 mapping is removed from the snap mapper, P's interval changed;
  4. Snap snap2 is trimmed on obj1 and caused the c to be removed
  5. The osd gets restarted and find that snap1 of obj1 hasn't been trimmed
  6. The osd tries to trim snap1 on obj1 and meet the assertion cit != std::end(ss.clones)

Copy link
Contributor

@Matan-B Matan-B left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little bit tricky.

  1. Snaps snap1 and snap2 on object obj1 maps to the same clone c
  2. Snap snap1 is being trimmed on PG P;
  3. Before snap1 of obj1 is trimmed and after the obj1 <-> snap1 mapping is removed from the snap mapper, P's interval changed;
  4. Snap snap2 is trimmed on obj1 and caused the c to be removed
  5. The osd gets restarted and find that snap1 of obj1 hasn't been trimmed
  6. The osd tries to trim snap1 on obj1 and meet the assertion cit != std::end(ss.clones)

Ah, thanks! LGTM besides comments left by @athanatos

@xxhdx1985126
Copy link
Contributor Author

All done:-)

@Matan-B
Copy link
Contributor

Matan-B commented May 19, 2024

@Matan-B
Copy link
Contributor

Matan-B commented May 19, 2024

jenkins test windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants