ISPN-16076 Convert RadarGun test to Hyperfoil

[jabolina]

This PR is the initial setup for the read-write test in RadarGun. We only need a single template file. We can then split the read/write ratio and create the actual benchmark based on those values. By default, it runs 80% reads and 20% writes.

Pending points:

• The use_auth doesn't seem to propagate when running with the all_benchmarks playbook;
• The load_multiplier value is still a placeholder until we find the actual limit in the server (https://issues.redhat.com/browse/ISPN-16077).

I am creating it as a draft until we have everything round.

[jabolina]

To reproduce the issue in AWS, just run the benchmarks as is with this load and the default with auth enabled. The test runs for 4 mins. The generated report should end with a weird result:

And some errors complaining of CPU utilization.

[jabolina]

I've been running the agent with -e agent_java_args="'-Xms128m -Xmx800m -XX:StartFlightRecording=delay=10s,duration=24h,settings=profile,filename=/tmp/hyperfoil/workspace/hfa-hotrod.jfr'".

I also tried disabling C2 and increasing the new generation size. Sometimes, it completes just fine, but sometimes, I still see the 100% CPU spike. There were cases in which just one of the clients showed this behavior. I am looking into the JFR, and I'll try to compare what happened.

[jabolina]

I'll be updating this with our current findings. We'll utilize a max_active with a value of 2 in all Hot Rod tests.

[jabolina]

I'll investigate until we have concrete baseline numbers with this new configuration. I'll keep the base t2.micro instances for this. We have tests for DIST and REPL with 3 server nodes.

[wburns]

I like this, cool reuse!

[wburns]

Can this not be done with a loop invoking set_fact dynamically? Set fact can use dynamic variable arguments such as https://github.com/infinispan/infinispan-ansible-benchmark/blob/main/roles/aws_ec2/tasks/flat-map-instances.yml#L15

[jabolina]

Hrrrmmm, let me see. The set_fact would need to accept a JSON variable as input.

[wburns]

You don't need to. It is already in the ansible variable. You don't need the local_action above either. Just do a loop on the arguments variable which should be a dict at this point.

  ansible.builtin.set_fact:
    "{{ item.key }}" : "{{ item.value }}"
  loop: "{{ arguments | dict2items }}"

Can't remember if you need the dict2items, you might not at all

[jabolina]

It almost works, but the set_fact seems to convert everything to string, which causes issues when parsing the templates.

[jabolina]

Hrrm, it seems there's an option for that: https://stackoverflow.com/a/62268930/9244031

[wburns]

I think the benchmark template can do the cast directly, no?

[wburns]

Btw we need to do something to fix it because I also can't pass -e read_ratio=0.8 -e write_ratio=0.2 to ansible-playbook without getting the same error.

[wburns]

Yeah, I change benchmark to be write_ratio | float and the same with read_ratio in all the places I can pass a command line argument as well, which will also fix the cast issue you had. We should probably do that for any arguments that may be overridden in the benchmark file by the user.

[jabolina]

That's great. I've updated the benchmark adding a cast to all the numeric parameters.

[wburns]

Integrated, thanks @jabolina !