Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pacific: osd/PGLog.cc: Trim duplicates by number of entries #46252

Merged
merged 2 commits into from Jun 7, 2022
Merged

pacific: osd/PGLog.cc: Trim duplicates by number of entries #46252

merged 2 commits into from Jun 7, 2022

Conversation

rzarzynski
Copy link
Contributor

Backport of #45529.

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@NitzanMordhai @rzarzynski
tools/ceph_objectstore_took: Add duplicate entry trimming
1f3fede 
Adding duplicate entries trimming to trim-pg-log opertion, we will use the exist
PGLog trim function to find out the set of entries\dup entries that we suppose to
trim. to use it we need to build the PGLog from disk.

Fixed: https://tracker.ceph.com/issues/53729
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
(cherry picked from commit 9fb7ec6)
@NitzanMordhai @rzarzynski
osd/PGLog.cc: Trim duplicates by number of entries
d49ff13 
PGLog needs to trim duplicates by the number of entries rather than the versions. That way, we prevent unbounded duplicate growth.

Fixed: https://tracker.ceph.com/issues/53729
Signed-off-by: Nitzan Mordechai <nmordec@redhat.com>
(cherry picked from commit 0d253bc)
@rzarzynski rzarzynski added this to the pacific milestone May 12, 2022
@rzarzynski rzarzynski requested review from a team as code owners May 16, 2022 18:49
@rzarzynski rzarzynski requested review from pereman2 and nizamial09 and removed request for a team May 16, 2022 18:49
@rubenk
Copy link
Contributor

rubenk commented May 18, 2022

This PR contains 983 commits, that doesn't seem right.

@k0ste
Copy link
Contributor

k0ste commented May 19, 2022

@rzarzynski, can you rebase please?

@neha-ojha neha-ojha changed the base branch from pacific to pacific-saved May 19, 2022 14:51
@neha-ojha neha-ojha changed the base branch from pacific-saved to pacific May 19, 2022 14:51
@neha-ojha
Copy link
Member

@rubenk @k0ste I have fixed the commits.

@cervigni
Copy link

Hello,

Could someone explain me how long this is going to take to be merged and packaged for pacific stable? I thought it was the case when merged into master but I was mistaken, as 16.2.8/9 does not contain the PR. Thanks !

@gadLinux
Copy link

Yes. Same here. Waiting for this patch to bring up my cluster...

@gadLinux
Copy link

gadLinux commented May 30, 2022 •
edited

@k0ste @rubenk @jdurgin @pereman2 @nizamial09 Can you check please and merge if properly done?

@wvh-github
Copy link

@neha-ojha This PR is approved by 2 people and all checks have passed. What is the next step to merge this?

@neha-ojha
Copy link
Member

@neha-ojha This PR is approved by 2 people and all checks have passed. What is the next step to merge this?

This PR will be picked by @yuriw for teuthology testing. We'll merge the PR after reviewing the test results.

@gadLinux
Copy link

gadLinux commented Jun 5, 2022

@neha-ojha This PR is approved by 2 people and all checks have passed. What is the next step to merge this?

This PR will be picked by @yuriw for teuthology testing. We'll merge the PR after reviewing the test results.

Is there any repository of ubuntu packages or binaries so I can give it a try in my cluster?... I don't know why but I'm unable to build it.

@neha-ojha
Copy link
Member

@neha-ojha This PR is approved by 2 people and all checks have passed. What is the next step to merge this?

This PR will be picked by @yuriw for teuthology testing. We'll merge the PR after reviewing the test results.

Is there any repository of ubuntu packages or binaries so I can give it a try in my cluster?... I don't know why but I'm unable to build it.

@gadLinux This PR is being tested using https://shaman.ceph.com/builds/ceph/wip-yuri4-testing-2022-06-01-1350-pacific/, and the builds will temporarily be available in shaman. Note that the branch wip-yuri4-testing-2022-06-01-1350-pacific has other patches as well https://github.com/ceph/ceph-ci/tree/wip-yuri4-testing-2022-06-01-1350-pacific.
I'd suggest waiting a day or two for this PR to be merged and then use pacific builds from https://shaman.ceph.com/builds/ceph/pacific/.

@ljflores
Copy link
Contributor

ljflores commented Jun 7, 2022 •
edited

http://pulpito.front.sepia.ceph.com/yuriw-2022-06-02_00:50:42-rados-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/
http://pulpito.front.sepia.ceph.com/yuriw-2022-06-02_14:44:32-rados-wip-yuri4-testing-2022-06-01-1350-pacific-distro-default-smithi/

Failures, unrelated:
1. https://tracker.ceph.com/issues/52321
2. https://tracker.ceph.com/issues/54992
3. https://tracker.ceph.com/issues/53939
4. https://tracker.ceph.com/issues/54071
5. https://tracker.ceph.com/issues/53501
6. https://tracker.ceph.com/issues/45318
7. https://tracker.ceph.com/issues/49888
8. https://tracker.ceph.com/issues/48965

Details:
1. qa/tasks/rook times out: 'check osd count' reached maximum tries (90) after waiting for 900 seconds - Ceph - Orchestrator
2. cannot stat '/etc/containers/registries.conf': No such file or directory - Ceph - Orchestrator
3. ceph-nfs-upgrade, pacific: Upgrade Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon osd.0 on host smithi103 failed - Ceph - Orchestrator
4. rados/cephadm/osds: Invalid command: missing required parameter hostname() - Ceph - Orchestrator
5. Exception when running 'rook' task. - Ceph - Orchestrator
6. octopus: Health check failed: 2/6 mons down, quorum b,a,c,e (MON_DOWN)" in cluster log running tasks/mon_clock_no_skews.yaml - Ceph - RADOS
7. rados/singleton: radosbench.py: teuthology.exceptions.MaxWhileTries: reached maximum tries (3650) after waiting for 21900 seconds - Ceph - RADOS
8. qa/standalone/osd/osd-force-create-pg.sh: TEST_reuse_id: return 1 - Ceph - RADOS

@neha-ojha neha-ojha merged commit 5618caf into ceph:pacific Jun 7, 2022
8 checks passed
@cervigni
Copy link

cervigni commented Jun 8, 2022

Due to the bug severity, is this going to be soon released as 16.2.10 or do we have to wait more?
Thanks

@wvh-github
Copy link

wvh-github commented Jun 14, 2022 •
edited

Just to inform everyone following this PR. It is going to be reverted:

#46610

We would also need to reopen this bug report I guess: https://tracker.ceph.com/issues/55631

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@neha-ojha neha-ojha neha-ojha approved these changes

@NitzanMordhai NitzanMordhai NitzanMordhai approved these changes

@jdurgin jdurgin Awaiting requested review from jdurgin

@pereman2 pereman2 Awaiting requested review from pereman2 pereman2 was automatically assigned from ceph/dashboard

@nizamial09 nizamial09 Awaiting requested review from nizamial09 nizamial09 was automatically assigned from ceph/dashboard

Assignees
No one assigned
Labels
core needs-qa tests wip-yuri4-testing
Projects
None yet
Milestone
pacific
10 participants
@rzarzynski @rubenk @k0ste @neha-ojha @cervigni @gadLinux @wvh-github @ljflores @NitzanMordhai @yuriw