Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

quincy: mds: account for snapshot items when deciding to split or merge a directory #49673

Merged
merged 2 commits into from Feb 16, 2023
Merged

quincy: mds: account for snapshot items when deciding to split or merge a directory #49673

merged 2 commits into from Feb 16, 2023

Conversation

vshankar
Copy link
Contributor

@vshankar vshankar commented Jan 9, 2023

https://tracker.ceph.com/issues/58347

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@vshankar
mds: account for snapshot items when deciding to split or merge a dir…
64c683a 
…ectory

Its easy to "overload" a directory object with large number of omap entries by
doing the following (one shot or over and over again):

      - touch dir/file{0..11000} ; create 11000 files (> mds_bal_split_size)
      - mkdir dir/.snap/snap_a
      - rm -f dir/file{0..11000}

End result - the directory object would have 11000 omap entries since the MDS
does not fragment directory snapshots. If the number of such entries exceed
`osd_deep_scrub_large_omap_object_key_threshold` (default: 200000), a cluster
health warning is generated:

       Large Omap objects found in pool...

CDir::should_merge() does not take into account COW'd inodes and the frags
get merged.

Fixes: http://tracker.ceph.com/issues/55215
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 47cdb3e)
@vshankar
test: verify that snapshot items are accounted when deciding to split…
07a2081 
…/merge a directory

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 7547990)
@vshankar vshankar requested a review from a team January 9, 2023 10:40
@github-actions github-actions bot added the tests label Jan 9, 2023
@github-actions github-actions bot added this to the quincy milestone Jan 9, 2023
vshankar
vshankar commented Feb 15, 2023
View reviewed changes
Copy link
Contributor Author

@vshankar vshankar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://tracker.ceph.com/projects/cephfs/wiki/Quincy#2023-Feb-15

@vshankar
Copy link
Contributor Author

jenkins test api

@vshankar
Copy link
Contributor Author

Reviewed-by: Venky Shankar vshankar@redhat.com

@yuriw yuriw merged commit cb889f5 into ceph:quincy Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lxbsz lxbsz lxbsz approved these changes

Assignees
No one assigned
Labels
cephfs Ceph File System needs-qa quincy-batch-1 tests wip-yuri5-testing
Projects
None yet
Milestone
quincy
3 participants
@vshankar @lxbsz @yuriw