[suggesion] Benchmarking

[puzrin]

Created after internal discussion.

All of us have different experience with code optimization. But sometime it's difficult to explain, and using authority is not enough (and not correct). Having benchmark would be better than thousands of words. It would be nice to have possibility create demo benchmarks. As easy as unit tests.

What is suggested:

• Consider use Gem5 cycle accurate sim. There is chance to run stable benchmarks on PC. With cores "close to our needs" (they are not 100% Cortex M0-M4 but sufficient for benchmarking).
  • While Gem5 itself is a complicated thing, running it via docker in SE mode should not be difficult.
• Since approach is similar to unit-testing, it's better to organize details after testing organized.

[kisvegabor]

Thanks for opening the issue!

I've found that gem5 doesn't support 32 bit ARMs but support 64 bit ARMs. LINK
I think it's still better than x86.

For cycle-accurate ARM simulator I've found CPAK but at first look, it doesn't have CLI.

I agree to postpone it until we establish a testing environment. However, a proof of concept with gem5 would be great before that.

[embeddedt]

I've found that gem5 doesn't support 32 bit ARMs but support 64 bit ARMs.

The link mentions that "The ARM implementation supports 32 or 64-bit kernels and applications." That sounds like 32-bit support to me. Did I miss something?

[kisvegabor]

Sorry, wrong link. I saw it here: http://www.m5sim.org/ARM_Implementation
So it supports ARMv7-A and ARMv8-A. I think the quoted sentence means "you can run 32-bit code as well".

Meanwhile, I found QEMU as well. Still investigating.

[puzrin]

Gem5 has interesting "SE" mode, when it emulates posix system. So, benchmark will be simple CLI application. All you need is to mark code area (with special fake instruction) where it should make snapshots, and count cycles diff.

Meanwhile, I found QEMU as well. Still investigating.

I investigated. It's not cycle accurate. Will not be able to make results reproducible. QEMU is ok to execute code, but can be used for benchmarking only in scope of single desktop. Gem5 benchmarks will give the same result everywhere.

[kisvegabor]

I investigated. It's not cycle accurate. Will not be able to make results reproducible. QEMU is ok to execute code, but can be used for benchmarking only in scope of single desktop. Gem5 benchmarks will give the same result everywhere.

I see, thanks.

---

Is there someone who already used Gem5 and/or interested in playing with it?

[stale]

This issue or pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kisvegabor]

It's added to the ROADMAP.

Now I'm dealing with the features planned for v8 but if anybody has the interest to dive into this now, feel free to comment here.