benchmark scaleability of solutions

[rcmlz]

I am interested in learning from solutions to Raku/Perl weekly challenge that are readable and efficient - the second point I can not judge easily by just running the provided minimal testcases.

I was wondering if there is interest out there in automatically comparing different solutions in terms of scalability - meaning, what problem size can be solved in a fixed runtime.

First thoughts and a first shot #8644 and a github-action how an automated process might look like (look into "Run Benchmark" and scroll down a bit).

It looks like this:

O-----------------O---------O--------O-------------O-------------O
| user            | runtime | n      | latency     | total       |
O=================O=========O========O=============O=============O
| two-colours     | 3       | 524288 | 0.94618477  | 1.78539195  |
| mark-anderson   | 3       | 524288 | 0.934987441 | 1.797814585 |
| arne-sommer     | 3       | 524288 | 0.951995122 | 2.034696379 |
| bruce-gray      | 3       | 524288 | 0.966044871 | 2.068688515 |
| Orir            | 3       | 524288 | 1.171071109 | 2.081696549 |
| roger-bell-west | 3       | 524288 | 2.077494412 | 2.574138279 |
| rcmlz           | 3       | 262144 | 0.47831577  | 1.52560537  |
| wambash         | 3       | 262144 | 0.481722904 | 1.922695617 |
| jaldhar-h-vyas  | 3       | 262144 | 2.462856115 | 2.865514525 |
| deadmarshal     | 3       | 65536  | 2.532371318 | 2.635808188 |
| ash             | 3       | 65536  | 2.621710334 | 2.691216938 |
| barroff         | 3       | 32768  | 1.853222122 | 1.883633262 |
| massa           | 3       | 32768  | 2.576829156 | 2.624975424 |
| ulrich-rieke    | 3       | 4096   | 1.51628605  | 1.519228254 |
------------------------------------------------------------------

After checkout try this:

zef install JSON::Tiny Text::Table::Simple
NR=233
raku --optimize=3 -I challenge-nr$NR -- test/benchmark-scalability.raku --max-run-times=3 --test-before-benchmark=True -v --max-problem=20 --out-folder=/tmp nr$NR

I put in the github-action also a small Tokei analysis, looking like below. Maybe this is usefull for some of us to find interesting solutions of other people.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Language                                                               Files        Lines         Code     Comments       Blanks
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 Raku                                                                      37          737          540           80          117
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
 challenge-nr233/packy-anderson/raku/task-one.rakumod                                   57           36           12            9
 challenge-nr233/ulrich-rieke/raku/task-one.rakumod                                     40           34            2            4
 challenge-nr233/athanasius/raku/task-one.rakumod                                       43           29            2           12
 challenge-nr233/ulrich-rieke/raku/task-two.rakumod                                     29           23            2            4
 challenge-nr233/packy-anderson/raku/task-two.rakumod                                   34           23            6            5
 challenge-nr233/roger-bell-west/raku/task-one.rakumod                                  26           23            1            2
 challenge-nr233/roger-bell-west/raku/task-two.rakumod                                  25           22            1            2
 challenge-nr233/arne-sommer/raku/task-one.rakumod                                      29           22            2            5
 challenge-nr233/rcmlz/raku/task-one.rakumod                                            26           20            2            4
 challenge-nr233/rcmlz/raku/task-two.rakumod                                            27           20            2            5
 challenge-nr233/robert-dicicco/raku/task-one.rakumod                                   23           18            2            3
 challenge-nr233/laurent-rosenfeld/raku/task-two.rakumod                                21           16            2            3
 challenge-nr233/wambash/raku/task-one.rakumod                                          20           15            2            3
 challenge-nr233/luca-ferrari/raku/task-two.rakumod                                     17           13            1            3
 challenge-nr233/barroff/raku/task-two.rakumod                                          17           13            2            2
 challenge-nr233/ash/raku/task-one.rakumod                                              17           13            1            3
 challenge-nr233/ash/raku/task-two.rakumod                                              17           13            1            3
 challenge-nr233/Orir/raku/task-two.rakumod                                             16           12            2            2
 challenge-nr233/Orir/raku/task-one.rakumod                                             17           12            3            2
 challenge-nr233/luca-ferrari/raku/task-one.rakumod                                     16           12            1            3
 challenge-nr233/athanasius/raku/task-two.rakumod                                       16           12            1            3
 challenge-nr233/laurent-rosenfeld/raku/task-one.rakumod                                16           12            2            2
 challenge-nr233/barroff/raku/task-one.rakumod                                          18           12            2            4
 challenge-nr233/jaldhar-h-vyas/raku/task-two.rakumod                                   16           11            2            3
 challenge-nr233/deadmarshal/raku/task-one.rakumod                                      14           10            2            2
 challenge-nr233/wambash/raku/task-two.rakumod                                          14           10            2            2
 challenge-nr233/bruce-gray/raku/task-one.rakumod                                       15           10            3            2
 challenge-nr233/deadmarshal/raku/task-two.rakumod                                      13            9            2            2
 challenge-nr233/arne-sommer/raku/task-two.rakumod                                      12            8            2            2
 challenge-nr233/massa/raku/task-two.rakumod                                            11            8            1            2
 challenge-nr233/jaldhar-h-vyas/raku/task-one.rakumod                                   11            7            2            2
 challenge-nr233/mark-anderson/raku/task-two.rakumod                                    10            7            1            2
 challenge-nr233/mark-anderson/raku/task-one.rakumod                                    10            7            1            2
 challenge-nr233/two-colours/raku/task-one.rakumod                                      12            7            3            2
 challenge-nr233/two-colours/raku/task-two.rakumod                                      11            7            2            2
 challenge-nr233/massa/raku/task-one.rakumod                                            10            7            1            2
 challenge-nr233/bruce-gray/raku/task-two.rakumod                                       11            7            2            2

[manwar]

Did you close the issue by mistake?
I thought this was supposed to be merged after few runs?

[rcmlz]

No, closed on purpose. I could not solve the issue with the folder names - merging would only make sense if folders would not need to be renamed.Furthermore no one was interested in benchmarking the different solutions, not whithin a single language nor between languages.For me it is fine to leave it like it is. I am extracting interesting solutions using Tokei, then benchmark myself.Thank you for running the weekly challenge!Am 31.10.2023 um 17:20 schrieb Mohammad Sajid Anwar ***@***.***>: Did you close the issue by mistake? I thought this was supposed to be merged after few runs? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you modified the open/close state.Message ID: ***@***.***>