Monday 16 May 2022
- 08:00-09:00 Pacific Time
- 11:00-12:00 East Coast Time
- 15:00-16:00 UTC
- 16:00-17:00 UK Time
- 17:00-18:00 Central European Time
- 23:00-00:00 Beijing Time
Jeremy Bennett and Paolo Savini to rework the presentation on execution speed measurements to focus just on time and time ratios.
- carried forward to future meeting.
Ray Simar and Will Jones (Embecosm) to investigate TinyMLPerf.
- carried forward to future meeting.
Jeremy Bennett and Paolo Savini to run some tests using both CoreMark and Dhrystone as well as Embench as input data to David Patterson's paper. Results to be shared in a spreadsheet with benchmark results and post to the mailing list.
- in progress, held open
Jeremy Bennett to progress bit manipulation example from Embecosm into a future benchmark.
- carried forward to future meeting.
Apologies were received from David Patterson.
Jeremy Bennett and Paolo Savini to rework the presentation on execution speed measurements to focus just on time and time ratios.
- carried forward to future meeting.
Ray Simar and Will Jones (Embecosm) to investigate TinyMLPerf.
- carried forward to future meeting.
Jeremy Bennett and Paolo Savini to run some tests using both CoreMark and Dhrystone as well as Embench as input data to David Patterson's paper.
- in progress, held open
Jeremy Bennett and Paolo Savini to prepare spreadsheet with benchmark results and post to the mailing list.
- merge with previous action.
Jeremy Bennett to progress bit manipulation example from Embecosm into a future benchmark.
- carried forward to future meeting.
No new pull requests were received since our last meeting.
The following pull requests are still open and under review:
- #153 Fix non-existing executables.
- #152 Remove debug printfs from md5sum.
- #151 Some fixes for build scripts.
- #148 Board config for Freedom E310 Arty FPGA.
- #147 Use calloc_beebs in md5sum.
- #145 Add --benchmark and --exclude options to scripts fore debugging.
- #144 Replace pyelftools with lief.
- #130 Improve JSON output of benchmarking.
There are no new pull requests this month and none remain open and under review.
The following issue has been received
- #160 In nbody some toolchains hoisting bodies_energy out of main loop
- modern compilers are too good for this benchmark as written
- the issue includes a suggestion for fixing the problem
- options are to either apply the fix or drop the benchmark from Embench 2.0
- see discussion below.
- thanks to user Mark Hill for raising this.
A total of 43 issues remain open.
There are no new issues raised this month.
A total of 1 issue remains open.
This paper is being prepared by Dave Patterson to be the definitive reference for the benchmark.
Paolo Savini and Jeremy Bennett gave an update on their data gathering. They are aiming to get Embench, CoreMark and Dhrystone scores for the following architectures/boards:
- Arm Cortex M4 (STM Disovery board @ 16MHz)
- 32-bit RISC-V (SiFive HiFive1 board rev B)
- 32-bit MIPS core (MicroChip Curiosity PIC32 MX470)
- 32-bit RISC-V (CHIP EH1 Verilator model)
At present work is stalled, but working on getting first LLVM results for Arm, then RISC-V results during the coming week.
This is the benchmark suite aimed a larger processor - up to 1MB memory and floating point support.
Ray Simar and Andrew Obler provided an update.
Past semester have taken a FIR filter and changing API to be consistent with Arm CMSIS DSP library. Just the calling conventions. Got this working.
Aim to take this benchmark into the embench-dsp repository over the summer and get it cleaned up. The a template for other benchmarks. Already have IIR and FFT, which could then add quickly. Uses floating point at present. Can also do fixed point versions, but that is a later piece of work.
Will do a write-up and slides to explain what they are doing.
David Harris and his colleagues at Harvey Mudd are looking at better ways to measure Embench performance. This agenda item will be for David to lead a discussion on how to achieve this.
Preparing a textbook on embedded design and want to include chapter on benchmarking using Embench. Noted that Embench IoT is defined not to do I/O and all benchmarks should be self checking, reported via return code.
Propose adding instrumentation to start and stop counters. Have specifics for Arm and RISC-V to read our results.
Erin Olson reported on doing this with the CRT0 to set up and the start_trigger and stop_trigger functions to start and stop timers. She shared https://github.com/riscv-software-src/riscv-tests/blob/master/benchmarks/common/crt.S
Suggest use of linker script to specify locations of global variables.
No questions this month.
The document is open for submissions, for both new benchmarks and benchmarks to cull.
Note that issue #160 identifies a problem with the current nbody benchmark.
Discussion based on this agreed nbody should be removed.
Also noted that the st benchmark discarded results, so could be optimized away. Suggest we need a statistics benchmark, so this should be fixed.
There is an opportunity to speak at the London RISC-V meetup in July.
We currently meet regularly on the third Monday of each month 8-9am California time.
The following are the dates for the remainder of 2022
- Mon 20 Jun 2022
- Mon 18 Jul 2022
- Mon 15 Aug 2022
- Mon 19 Sep 2022
- Mon 17 Oct 2022
- Mon 21 Nov 2022
- Mon 12 Dec 2022 (week 2, to keep clear of Christmas)
Jeremy Bennett
Vice Chair