osd: erasure code benchmark tool

[ghost]

Implement the ceph_erasure_code_benchmark utility to:

• load an erasure code plugin
• loop over the encode function using the parameters from the command
  line
• print the number of bytes encoded and the time to process

For instance:

$ ceph_erasure_code_benchmark \
   --plugin jerasure \
   --parameter erasure-code-directory=.libs \
   --parameter erasure-code-technique=reed_sol_van \
   --parameter erasure-code-k=2 \
   --parameter erasure-code-m=2 \
   --iterations 1000
0.964759  1048576000

shows 1GB is encoded in 1second.

Signed-off-by: Loic Dachary loic@dachary.org

[ghost]

@apeters1971 I would very much appreciate your review when you get time

[apeters1971]

This looks already very good. I would add also add the option to additionally measure the decoding performance when data stripes are unavailable and evt. beautify the output a little bit - e.g. print key-value pairs with input and output parameters e.g. also compute something human readable like MB/s ?!?!

I would consider the option to have the bufferlist allocated in individual chunks and see the impact of the performance. I don't know if in the final usage in CEPH there will be one big buffer allocated and segemented or each piece will have its own allocation ... in the SNAPRAID sources there is this remark:

• When allocating a sequence of blocks with a size of power of 2,
• there is the risk that the start of each block is mapped into the same cache line,
• resulting in cache collisions if you access all the blocks in parallel,
• from the start to the end.

I don't know your exact plans, but it would be also very useful to wrap the command with some scripting language printing the platform you run, memory and maybe run each command like this and some standard test parameters ?!?!?

perf stat /usr/bin/time -v ceph_erasure_code_benchmark ....
cat /proc/cpuinfo
cat /proc/meminfo

[ghost]

Thanks for the quick feedback. I'll work on it today :-)

[ghost]

@apeters1971 I've added decode + inserting erasure as you suggested.
The plan is to use this within another tool rather than being human readable. It would be a script added to ceph/qa/workunits that would call ceph_erasure_code_benchmark with various relevant parameters and compile the result in a CSV file to be consumed either by a teuthology suite designed to detect regression between versions or by something to show a nice human readable display.
Regarding the impact of chunk allocation, I would rather do it later : as you point out we don't know exactly how it will be done within Ceph.

[ghost]

@apeters1971 be8cfc9 is the implementation of the benchmark workunit based on ceph_erasure_code_benchmark

[ghost]

@apeters1971 I was able to find performance problems in the example ( I know it does not matter really but ... ;-). I was also able to confirm that jerasure performances as reported by ceph_erasure_code_benchmark are the same as the one you can get from the decoder/encoder examples found in Jerasure itself. This is a good sign that the tool works and that the plugin implementation does not introduce any significant performance problem.

Although I could work on a teuthology job using bench.sh, I would like to use it to measure the benefit of your implementation of BPC first.

[ghost]

@apeters1971 I ran bench.sh and interpreted the results as explained in the draft post here : http://dachary.org/loic/ecbench/ . The numbers look good and will be even better after your code is merged :-) It would be great if you could run it independently and confirm the numbers.

[apeters1971]

@loic yes will do,
this looks familiar .. just that you have a 40% faster CPU than me. Which
GCC version did you use?

On Sat, Dec 14, 2013 at 2:41 AM, Loic Dachary notifications@github.comwrote:

@apeters1971 https://github.com/apeters1971 I ran bench.sh and
interpreted the results as explained in the draft post here :
http://dachary.org/loic/ecbench/ . The numbers look good and will be even
better after your code is merged :-) It would be great if you could run it
independently and confirm the numbers.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/933#issuecomment-30556893
.

[ghost]

I use

 gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)

and

 make CFLAGS='-O3' 

so jerasure compiles with -O3 instead of -O2

[ghost]

rebased against master and verified that make check is happy. ceph_erasure_code_benchmark was wrongfully listed as a unit test, it is a debug program. Also added packaging information. Foucault de Bonneval verified that it runs and provides consistent results on Dell R620.

Now checking to see if gitbuilder would be happy about the change.

[ghost]

@apeters1971 implemented -h as you suggested and made it so no argument issues a sensible error message instead of a stack trace