Extend librbdfio.py with a new workloads block. Add minor fixes. by perezjosibm · Pull Request #306 · ceph/cbt

perezjosibm · 2024-05-02T13:05:26Z

This is my first PR, please be gentle in the review. We add the following new features into the CBT for the benchmark librbdfio:

Extend the test plan .yml with a new block section 'workloads' to allow a list of performance tests to be indicated, each with its own range of iodepth and numjobs to iterate. An example of this is provided in example/rbd_fio_test.yml
Extend librbdfio.py with support for such 'workloads' block.

New workloads block

The value of this new workloads block is to allow a finer description of a test job, for example, indicating a range for the iodepth and numjobs FIO parameters, as well as whether to monitor the execution (eg. with perf, top, or collectl). This allow the collection of measurements for computing a response curve.
Here is a snippet that describes:

a precondition phase consisting on a random write workload, that is not being monitored, then
a random read workload over the ranges indicated for the FIO parameters numjobs and iodepth
a random write workload.

 workloads:
    precond1:
      jobname: 'precond1rw'
      mode: 'randwrite'
      numjobs: [ 1 ]
      iodepth: [ 8 ]
      monitor: False
    test1:
      jobname: 'rr'
      mode: 'randread'
      numjobs: [ 1, 4, 8 ]
      iodepth: [ 1, 4, 16]
    test2:
      jobname: 'rw'
      mode: 'randwrite'
      numjobs: [ 4 ]
      iodepth: [ 16 ]

The following snippet shows the execution of the precondition phase:

The following snippet shows the execution of the tests1 and 2:

The following snippet shows the resulting files from the execution of the test:

perezjosibm · 2024-05-15T09:11:34Z

@markhpc : here is my PR for the CBT extensions, would you be able to review please? thanks!

sseshasa · 2024-05-22T14:31:43Z

@perezjosibm I have started taking a look at this. Please allow me some time to understand the changes. I plan to complete the review by the end of this week or in the worst case early next week due to the long weekend here.

sseshasa

I have added my first set of review comments. Will continue the review.

sseshasa · 2024-05-30T08:27:04Z

I have added my first set of review comments. Will continue the review.

Hmm...looks like my initial comments were lost. I will add them again. Sorry about that.

sseshasa

Okay, now my first set of comments are added. Will continue reviewing this.

benchmark/benchmark.py

benchmark/librbdfio.py

sseshasa

I am done with the first pass and have provided my comments.

I mainly focused on the main parts of the code changes related to benchmark.
One important recommendation is to ensure through testing that the original behavior of librbdfio benchmark is preserved

benchmark/librbdfio.py

sseshasa · 2024-05-30T12:24:52Z

cbt.py

+    #logger.debug("Settings.cluster.is_teuthology:%s",settings.cluster.get('is_teuthology', False))
    # Run the benchmarks
    return_code = 0
+


Remove newline.

cbt.py

common.py

monitoring.py

settings.py

sseshasa · 2024-05-30T13:07:16Z

tools/fio-parse-json.py

  sys.exit(NOTOK)

-
+# next_branch: token from the input path


It looks like the changes in this file is formatting related. I'd suggest this change alone be moved to a different commit along with an explanation of why changes were made to this file.

sseshasa · 2024-05-30T13:16:05Z

tools/fio-parse-jsons.py

@@ -0,0 +1,382 @@
+#!/usr/bin/python3


Since this is a standalone tool, I'd suggest the following:

Introduce this tool as part of a separate commit.

Move it to it's own directory under 'tools'.

Include a README.md in the new directory with details around the usage, examples, charts and so on as you have described in this file.

Makes sense, yes I'll do that asap

Hi @sseshasa: I've opened PR #307 for only the new script, thanks!

Hi @perezjosibm I mean introduce the new script in a new commit as a part of this PR(#306) itself. I will take look again once you have pushed all the requested changes. Thanks!

Hi @sseshasa sorry about that, I hope you don't mind reviewing them separately. I have updated this PR with the requested changes to the best of my understanding. I did a git ci -s -a --amend to have them all in a single batch. Many thanks.

perezjosibm · 2024-06-05T15:19:52Z

Tested the remote path by running CBT from my laptop, connecting to the remote nodes:

perezjosibm · 2024-06-05T19:15:49Z

Apologies, I just fixed some subtle bugs I introduced when doing the changes that didn't show up in the preliminary tests, I need to push the fixes again, sorry for the disturbance!

perezjosibm · 2024-06-06T09:27:39Z

Hi @sseshasa changes are ready for review please, many thanks!

sseshasa

Just added a couple of new review comments. But this looks good to me. I would prefer another pair of eyes to have a look at this as well. @markhpc can you please review this?

benchmark/librbdfio.py

cluster/cluster.py

Signed-off-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>

perezjosibm · 2024-07-22T16:22:26Z

Hi @Matan-B wonder if you could please review/approve this PR?, Thanks

perezjosibm force-pushed the wip.fio-parse-jsons branch from c0674af to 5259d8b Compare May 13, 2024 11:21

perezjosibm marked this pull request as ready for review May 13, 2024 11:21

neha-ojha requested review from markhpc and sseshasa May 20, 2024 14:59

perezjosibm force-pushed the wip.fio-parse-jsons branch from 5259d8b to 172ecf4 Compare May 21, 2024 10:47

sseshasa requested changes May 30, 2024

View reviewed changes

perezjosibm mentioned this pull request Jun 4, 2024

Add fio-parse-jsons.py tool to parse a set of json FIO files. #307

Merged

perezjosibm force-pushed the wip.fio-parse-jsons branch from 172ecf4 to ea446a9 Compare June 5, 2024 15:12

perezjosibm requested a review from sseshasa June 5, 2024 15:15

perezjosibm changed the title ~~Add fio-parse-jsons.py to parse a set of json FIO files. Extend librb…~~ Extend librbdfio.py with a new workloads block. Add minor fixes. Jun 5, 2024

perezjosibm force-pushed the wip.fio-parse-jsons branch from ea446a9 to eb0670b Compare June 5, 2024 19:15

sseshasa approved these changes Jun 9, 2024

View reviewed changes

benchmark/librbdfio.py Outdated Show resolved Hide resolved

cluster/cluster.py Outdated Show resolved Hide resolved

Extend librbdfio for a new workloads block. Fix minor bugs.

5823498

Signed-off-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>

perezjosibm force-pushed the wip.fio-parse-jsons branch from eb0670b to 5823498 Compare July 22, 2024 16:17

perezjosibm merged commit 655afe2 into ceph:master Jul 23, 2024

perezjosibm deleted the wip.fio-parse-jsons branch July 23, 2024 15:39

harriscr mentioned this pull request May 19, 2025

New workload classes #333

Merged

harriscr mentioned this pull request Jul 8, 2025

Post-proceesing changes for the new workloads code #336

Closed

Conversation

perezjosibm commented May 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New workloads block

Uh oh!

perezjosibm commented May 15, 2024

Uh oh!

sseshasa commented May 22, 2024

Uh oh!

sseshasa left a comment

Choose a reason for hiding this comment

Uh oh!

sseshasa commented May 30, 2024

Uh oh!

sseshasa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sseshasa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sseshasa May 30, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sseshasa May 30, 2024

Choose a reason for hiding this comment

Uh oh!

sseshasa May 30, 2024

Choose a reason for hiding this comment

Uh oh!

perezjosibm Jun 4, 2024

Choose a reason for hiding this comment

Uh oh!

perezjosibm Jun 4, 2024

Choose a reason for hiding this comment

Uh oh!

sseshasa Jun 5, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

perezjosibm Jun 5, 2024

Choose a reason for hiding this comment

Uh oh!

perezjosibm commented Jun 5, 2024

Uh oh!

perezjosibm commented Jun 5, 2024

Uh oh!

perezjosibm commented Jun 6, 2024

Uh oh!

sseshasa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

perezjosibm commented Jul 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

perezjosibm commented May 2, 2024 •

edited

Loading

sseshasa Jun 5, 2024 •

edited

Loading