Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pacific: mon: osd pool create <pool-name> with --bulk flag #44847

Merged
merged 5 commits into from Feb 25, 2022
Merged

pacific: mon: osd pool create <pool-name> with --bulk flag #44847

merged 5 commits into from Feb 25, 2022

Conversation

kamoltat
Copy link
Member

@kamoltat kamoltat commented Jan 31, 2022 •
edited

Creating the pool with --bulk will allow
the pg_autoscaler to use the scale-down
mode on.

Creating pool:

ceph osd pool create <pool-name> --bulk

Get var:

ceph osd pool get <pool-name> bulk

Set var:

ceph osd pool set <pool-name> bulk=true/false/1/0

To do:

  • incorporate the bulk flag to autoscale behavior
  • hide autoscale profiles from users and use bulk instead
  • modify existing test cases that used autoscale profile
  • add more test cases for bulk flag

Fixes: https://tracker.ceph.com/issues/54082

Backporting the relevant commits from master PR:
#44241

Signed-off-by: Kamoltat ksirivad@redhat.com

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@kamoltat kamoltat requested a review from a team as a code owner January 31, 2022 19:51
@kamoltat kamoltat self-assigned this Jan 31, 2022
@github-actions github-actions bot added this to the pacific milestone Jan 31, 2022
@kamoltat kamoltat force-pushed the wip-ksirivad-pacific-backport-44241 branch 4 times, most recently from d2a2533 to 06ad0a8 Compare February 1, 2022 21:27
@kamoltat
Copy link
Member Author

kamoltat commented Feb 2, 2022 •
edited

1/1 passed against rados:singleton:all:pg-autoscaler.yaml

https://pulpito.ceph.com/ksirivad-2022-02-02_06:19:41-rados:singleton:all:pg-autoscaler.yaml-wip-ksirivad-pacific-backport-44241-distro-basic-smithi/

@kamoltat
Copy link
Member Author

kamoltat commented Feb 2, 2022 •
edited

1/1 passed against rados:singleton-bluestore:all:cephtool.yaml

https://pulpito.ceph.com/ksirivad-2022-02-02_07:31:51-rados:singleton-bluestore:all:cephtool.yaml-wip-ksirivad-pacific-backport-44241-distro-basic-smithi/

@kamoltat
Copy link
Member Author

kamoltat commented Feb 2, 2022

Still need to test against the whole rados suite run, to make sure it doesn't break anything.

@kamoltat
mon: osd pool create <pool-name> with --bulk flag
d41ef37 
Creating the pool with `--bulk` will allow
the pg_autoscaler to use the `scale-down`
mode on.

Creating pool:

`ceph osd pool create <pool-name> --bulk`

Get var:

`ceph osd pool get <pool-name> bulk`

Set var:

`ceph osd pool set <pool-name> bulk=true/false/1/0`

Removed `autoscale_profile` and incorporate bulk flag
into calculating `final_pg_target` for each pool.

bin/ceph osd pool autoscale-status no longer has
`PROFILE` column but has `BULK` instead.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 5f0f967)

Conflicts:
    src/common/options/global.yaml.in
	    -  file doesn’t exists in pacific
	       we edited src/common/options.cc
	       and src/common/legacy_config_opts.h
	       instead.
    src/mon/MonCommands.h
	    - only add `bulk` related stuff not `eio`
    src/mon/OSDMonitor.cc
	    - use `cmd_getval` instead of `cmd_getval_or`
	    - only add `bulk` related stuff not `eio`
    src/osd/osd_types.h
	    - only add `bulk` related stuff not `eio`
    src/pybind/mgr/pg_autoscaler/module.py
	    - remove `profile: ‘ScaleModeT`
	    - keep `threshold: float 3.0`
	    - remove `set_profile_scale_up()`
	    - remove `set_profile_scale_down()`
	    - don’t accept `set_scaling_threshold()`
@kamoltat
pg_autoscaler/test: Modified unit-test for bulk flag
b34a224 
Modified the unit-test cases to account for bulk
flag and remove any `profile` related things.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit d90a374)
@kamoltat
qa/workunits/mon/pg_autoscaler: modified test script
327272d 
Modified test scrtipt to include `bulk` and
remove all `profile` options.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit c194f4a)
@kamoltat
qa/workunits/cephtool: modified test.sh
c4e19c8 
Included bulk flag in the test script
such it tests the `get` and `set` of
the flag

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit b218335)
@kamoltat
Added ReleasesNotes and documentation
778c2b5 
Add Release Notes and remove any `profile`
related stuff in the autoscaler documentation
and replace it with `bulk` flag.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit abaab51)

Conflicts:
	PendingReleaseNotes - trivial fix
@kamoltat kamoltat force-pushed the wip-ksirivad-pacific-backport-44241 branch from 06ad0a8 to 778c2b5 Compare February 3, 2022 04:59
@kamoltat
Copy link
Member Author

kamoltat commented Feb 3, 2022

jenkins test api

@ljflores
Copy link
Contributor

http://pulpito.front.sepia.ceph.com/?branch=wip-yuri7-testing-2022-02-17-0852-pacific

Failures, unrelated:
https://tracker.ceph.com/issues/53501
https://tracker.ceph.com/issues/54337
https://tracker.ceph.com/issues/52124
https://tracker.ceph.com/issues/54360
https://tracker.ceph.com/issues/54406
https://tracker.ceph.com/issues/53827
https://tracker.ceph.com/issues/53939
https://tracker.ceph.com/issues/54411
https://tracker.ceph.com/issues/54071
https://tracker.ceph.com/issues/50042

Details:
1. Exception when running 'rook' task. - Ceph - Orchestrator
2. Selinux denials seen on fs/rados teuthology runs
3. Invalid read of size 8 in handle_recovery_delete() - Ceph - RADOS
4. Dead job at "Finished running handlers" in rados/cephadm/osds/.../rm-zap-wait - Ceph
5. cephadm/mgr-nfs-upgrade: cluster [WRN] overall HEALTH_WARN no active mgr - Ceph - CephFS
6. cephadm exited with error code when creating osd: Input/Output error. Faulty NVME? - Infrastructure - Sepia
7. ceph-nfs-upgrade, pacific: Upgrade Paused due to UPGRADE_REDEPLOY_DAEMON: Upgrading daemon osd.0 on host smithi103 failed - Ceph - Orchestrator
8. mds_upgrade_sequence: "overall HEALTH_WARN 4 failed cephadm daemon(s); 1 filesystem is degraded; insufficient standby MDS daemons available; 33 daemons have recently crashed" during suites/fsstress.sh - Ceph - CephFS
9. rados/cephadm/osds: Invalid command: missing required parameter hostname() - Ceph - Orchestrator
10. sporadic rados/test.sh failures - Ceph - RADOS

@yuriw yuriw merged commit e733abc into ceph:pacific Feb 25, 2022
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anthonyeleven anthonyeleven anthonyeleven left review comments

@neha-ojha neha-ojha neha-ojha approved these changes

Assignees

@kamoltat kamoltat

Labels
common core documentation mon needs-qa pybind tests wip-yuri7-testing
Projects
None yet
Milestone
pacific
5 participants
@kamoltat @ljflores @anthonyeleven @neha-ojha @yuriw