[doc] Benchmark data about casbin-cpp in `casbin.org` maybe wrong.

[sheny1xuan]

Description

• casbin-cpp benchmark data in official website maybe wrong, RBAC time comlexity maybe O(n) rather than O(1), n is policy numbers in memory. So with data increase in RBAC small, RBAC medium and RBAC large, consumed time will increase. We can ref to this action, and other languge benchmark.
  
• And when I run Large benchmark in my local machine, I can't get the benchmark result, even I have run it at least one hour. I only get the some small data case benchmark result.
  

[casbin-bot]

@EmperorYP7 @divy9881 @noob20000405 @sheny1xuan

[hsluoyz]

@sheny1xuan good finding! Can you fix the bencnmark number in the docs?

[sheny1xuan]

@sheny1xuan good finding! Can you fix the bencnmark number in the docs?

Of course, but these are something wrong when I run large rbac model. When I fix it, I will update the newest benchmark number in the docs.

[hsluoyz]

@EmperorYP7 any comments?

[EmperorYP7]

So here are the benchmark results when I fed it in casbin.org

[sheny1xuan]

So here are the benchmark results when I fed it in casbin.org

Maybe in this benchmark reuslt, rbac small should be 50985511 / 78 = 653660.40ns and rbac medium should be 906444271 / 10 = 90644427.1ns. Is my calculation correct？

[EmperorYP7]

Maybe in this benchmark reuslt, rbac small should be 50985511 / 78 = 653660.40ns and rbac medium should be 906444271 / 10 = 90644427.1ns. Is my calculation correct？

1. The CPU and Time already show runtime/operation. You don't need to calculate it.
2. I took the CachedRBAC workflow for the benchmark.

[sheny1xuan]

Oh, it's average time already. May I update the benchmark result in casbin.org using your data?

[hsluoyz]

@sheny1xuan sounds good

[sheny1xuan]

So here are the benchmark results when I fed it in casbin.org

Hi, when I update the result above, I found casbin-cpp' performance maybe not good as we expect. It's too slow compared to golang and rust. I think maybe the result is outdated for us, and I rerun the benchmark in my local mahcine. Here is the runtime result, and it's faster than before, but I think it's not enough compared to golang and rust. And I change somewhere in rbac manager to improve the casbin-cpp performance, you can find changes in this [commit].(sheny1xuan@fabef33)

Maybe we should do some profiling for casbin-cpp to deal with it's performance problem as @hsluoyz say. Then we can update the docs latter. @EmperorYP7

[EmperorYP7]

Maybe we should do some profiling for casbin-cpp to deal with it's performance problem as @hsluoyz say. Then we can update the docs latter. @EmperorYP7

This is exactly what I was up to a few months ago. I tried to set up visual profiling for casbin-cpp. However, I couldn't find a way to include it in the existing CI workflows.

You're right, only after profiling can we improve the benchmark results. But for now, use CachedRBAC benchmarks on casbin.org for the sake of practicality. Do mention that it is for the cached enforcer.