Show the performance results in graphs

[rm5248]

Show performance results in a graphical way

This shows the performance results in a clearer manner than just the table, and makes it more obvious the relative difference between the different scenarios.

[swebb2066]

The Iteration column is not the data, just an implementation detail.

The actual data is the average (elapsed) time and CPU

[rm5248]

The actual data is the average (elapsed) time and CPU

I changed it to the 'time' column. The CPU column is ~6x as large as the time column for multithreaded tests, which would make sense when there are 6 threads.

[swebb2066]

Are you sure this adds enough clarity to warrant the extra maintenance? Enough to ignore Volkan's advice?

[rm5248]

It's using the same data that you put in the table, just showing it in a graphical manner. If we want to remove the graphs, I would argue that we should also remove the performance table.

[swebb2066]

Can you script the graph generation?

[rm5248]

I am not sure, that should theoretically be possible. It would probably make the repository size grow pretty quickly though. Since they're just generated from libreoffice, you can just copy/paste the new values into the spreadsheet that I added and regenerate the graphs, so at least they are easy to regenerate if needed.

[swebb2066]

Below is a python script that will generate this:

import re
import matplotlib.pyplot as plt

input_file_name  = 'markdown/performance.md'
output_file_name = 'images/Appending.png'
title = 'Appending a log message'
regex_pattern = [ r"Appending (.*) using ([A-Za-z]+), pattern: \\%m\\%n$"
                , r"Async, Sending (.*) using ([A-Za-z <]+)$"
#               , r"Appending (.*) using (MessageBuffer), pattern: \\%m\\%n/threads"
#               , r"Appending (.*) using (FMT), pattern: \\%m\\%n/threads"
                ]
plot_data = { 'MessageBuffer': {}
            , 'FMT': {}
            , 'FMT and AsyncBuffer' : {}
            , 'operator<< and AsyncBuffer' : {}
            }
line_type = { 'MessageBuffer': '--'
            , 'FMT': '-.'
            }
mark_type = { 'FMT and AsyncBuffer' : 'o'
            , 'operator<< and AsyncBuffer' : '+'
            }

# Extract the data
value_regex_pattern = r"([0-9]+) ns"
with open(input_file_name, "r") as file:
    header_found = False
    separator_found = False
    for line in file:
        if header_found and separator_found:
            # A valid table data row (starts and ends with '|')
            if not (line.strip().startswith('|') and line.strip().endswith('|')):
                break
            # Split the line by '|' and strip whitespace from each part
            # The first and last elements will be empty strings due to leading/trailing '|'
            columns = [col.strip() for col in line.split('|')]

            # Ensure there are enough columns to extract data (at least 3 for Benchmark and Time)
            if len(columns) > 2:
                value_match = re.search(value_regex_pattern, columns[2])
                if value_match:
                    nanoseconds = int(value_match.group(1))
                    for r in regex_pattern:
                        benchmark_match = re.search(r, columns[1])
                        if benchmark_match:
                            formatter = benchmark_match.group(2)
                            message_type = benchmark_match.group(1)
                            plot_data[formatter][message_type] = nanoseconds
        elif " Benchmark " in line:
            header_found = True
        elif header_found and "----" in line:
            separator_found = True
        else:
            header_found = False

# Plot the data
fig, ax = plt.subplots()
for formatter, item_time in plot_data.items():
    if len(item_time) < 1:
        raise ValueError(f"{formatter} benchmark data not found in {input_file_name}")
    message_types = item_time.keys()
    time_values = item_time.values()
    if formatter in line_type:
        ax.plot(message_types, time_values, label=formatter, linestyle=line_type[formatter])
    else:
        ax.plot(message_types, time_values, label=formatter, marker=mark_type[formatter])

# Add chart title and axis labels
ax.legend()
ax.set_title(title, fontsize=14)
ax.set_xlabel('Message content', fontsize=12)
ax.set_ylabel('Average Elapsed Time (ns)', fontsize=12)

# Improve layout and remove top/right spines for cleaner look
ax.spines['right'].set_visible(False)
ax.spines['top'].set_visible(False)
plt.tight_layout()

# Save the graph as a file
plt.savefig(output_file_name)

[rm5248]

I like that, it looks pretty good. I'll play around with it in the next few days.

[rm5248]

I updated the python code a bit, but then I realized two things:

1. benchmark can output JSON data
2. Apache echarts can be used to graph data

I messed around with echarts. It's kinda working at the moment, but I need to read the documentation more to understand how to make the graphs properly.

The graphs are a bit prettier, and also interactive so you can hover over for example:

The branch is up to date if you want to mess around with it.

[swebb2066]

Do you know if the web-site can use Apache echarts? I thought we were restricted to non-interactive pages.

[rm5248]

Do you know if the web-site can use Apache echarts? I thought we were restricted to non-interactive pages.

You can add in custom HTML elements with Doxygen by using the \htmlonly block. This allows us to insert the proper javascript code needed.