Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stdcbench results at https://github.com/z88dk/z88dk/wiki/Benchmarks? #1621

Open
spth opened this issue Oct 28, 2020 · 3 comments
Open

Add stdcbench results at https://github.com/z88dk/z88dk/wiki/Benchmarks? #1621

spth opened this issue Oct 28, 2020 · 3 comments
Labels
discussion requests for comment and discussion

Comments

@spth
Copy link

spth commented Oct 28, 2020

At https://github.com/z88dk/z88dk/wiki/Benchmarks, a few benchmarks are used to compare compilers.

Due to the shortcomings of Dhrystone, Whetstone, Coremark and the unsuitability of major benchmarks, such as SPEC for small systems, I wrote stdcbench:

https://dl.acm.org/doi/abs/10.1145/3207719.3207726
https://sourceforge.net/projects/stdcbench/

stdcbench is one of the benchmarks used to track code size and speed over SDCC revisions:

https://sourceforge.net/p/sdcc/code/HEAD/tree/trunk/sdcc-extra/historygraphs/stdcbench-stm8-size.svg
https://sourceforge.net/p/sdcc/code/HEAD/tree/trunk/sdcc-extra/historygraphs/stdcbench-stm8-score.svg

How about adding stdcbench results at https://github.com/z88dk/z88dk/wiki/Benchmarks?

@suborb
Copy link
Member

suborb commented Oct 29, 2020

I've started to take a look, it's showing up a few edge cases that I need to handle better in sccz80.

So far the conclusion is (to no great surprise), that loops in sccz80 aren't that great.

@suborb
Copy link
Member

suborb commented Oct 30, 2020

Because I'm running under ticks the number produced is a measure of fast (well slow!) my laptop is, so I've made a minor tweak to just run through the benchmarks once - I've only got c90base to work with zsdcc (4.0.3 #11877) at present unfortunately. Sadly this means we can't easily compare vanilla sdcc and z88dk:

zcc +test -compiler=sccz80 -O2 -m -> 173_691_034 ticks, 9417 bytes
zcc +test -compiler=sccz80 -O2 -m --opt-code-speed=all -> 145_485_000 ticks, 9599 bytes
zcc +test -compiler=sccz80 -O2 -m --opt-code-speed=all -lfastmath -> 122_275_012 ticks, 11877 bytes
zcc +test -compiler=sccz80 -O2 -m -clib=8080 -> 168_723_490 ticks, 9612 bytes
zcc +test -compiler=sccz80 -O2 -m -clib=rabbit -> 56_163_266 ticks, 8632 bytes
zcc +test -compiler=sccz80 -O2 -m -clib=gbz80 -> 170_860_224 ticks, 9461 bytes

zcc +test -compiler=sdcc -O2 --max-allocs-per-node100000 --opt-code-speed -m -> 94_211_835 ticks, 8749 bytes
zcc +test -compiler=sdcc -O2 --max-allocs-per-node100000 --opt-code-speed -m -lfastmath -> 86_213_355 ticks, 11041 bytes
zcc +test -compiler=sdcc -O2 --max-allocs-per-node100000 --opt-code-speed -m -clib=rabbit -> 49_282_502 ticks, 8320 bytes

There's obviously something wrong with the first sccz80 test - I'm not sure how the 8080 has managed to outpace it.

Given the lack of a context-aware optimiser (or indeed any code analysis) in sccz80 and the difference in compilation time (on my machine sdcc is taking 47 seconds, sccz80 takes just 3 seconds) it's not as bad as it could have been.

Any long tests will be interesting: the sccz80 generated code for longs is much more compact - see https://github.com/z88dk/z88dk/wiki/WritingOptimalCode#sample-results

EDIT: it's the 16 bit multiplication routine causing problems on the default z80 options: 8080 routine here: https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/l/sccz80/crt0_8080/l_mult.asm, z80 routine here: https://github.com/z88dk/z88dk/blob/master/libsrc/_DEVELOPMENT/math/integer/small/l_small_divu_16_16x16.asm)

@feilipu feilipu added the discussion requests for comment and discussion label May 11, 2022
@feilipu
Copy link
Collaborator

feilipu commented May 11, 2022

@suborb Did you publish the modifications required to get STDC Bench to compile with z80? I had a fleeting attempt, but didn't get far.

It would be nice to add at least some compiler runs for this benchmark.

Initially raised in #111

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion requests for comment and discussion
Projects
None yet
Development

No branches or pull requests

3 participants