[macOS] Benchmarks wanted: does hardware acceleration improve performance? #10100

Bouke · 2022-10-23T06:16:40Z

Bouke
Oct 23, 2022

Some time ago hardware acceleration (HW) on macOS was disabled by default. This was decided after a survey showed little or no gain with hardware acceleration enabled. However I recently wondered whether the default is still appropriate as hardware acceleration showed substantial performance improvements on my machines. However before we should consider changing the default, it would be good to gather more performance measurements from a range of hardware configurations. This is where you can help us out! Below you'll find links to a few savegames. Load those games with hardware acceleration enabled and disabled. Run the game, fast-forward- eyeball the fps counters and report your findings here.

Download the savegames: Hilbergen, Wentbourne.
Download this nightly build (20221021-master-g30ff89cac0). Unzip and run it.
Turn hardware acceleration off. Turn off autosave. Restart the game.
For every savegame:
1. Unpause if paused.
2. Open framerate window (Help menu, Show framerate), wait a minute. Really a minute. Wentbourne has a simulation rate of 11 frames per second on my machine, the average will take almost a minute to catch up.
3. Record "Graphics rendering (Average)".
4. Press fast-forward, wait a few seconds.
5. Record estimated average "Current game speed factor".
Turn hardware acceleration on. Restart the game.
Repeat step 3.
Add your findings here using the template below.

In order to get comparable numbers, keep your test setup the same between test runs:

use the nightly version linked above, as performace differs between versions
use the same power source, don't switch between battery and power adapter
use the same window size, don't resize the window
use the same viewport, don't move the map to a different location

Also feel free to report performance metrics for your own savegames instead. Just make sure to load the exact same savegame when comparing.

## System information <!-- see: Apple Menu, About This Mac -->

macOS version: 
Model: 
Processor: 
Graphics: 
Power source: battery / power adapter

## Game options:

Base graphics: Original / OpenGFX / zBase / aBase

## Optional information: <!-- would be nice to have, but ok if left empty -->

Display: built-in Retina display / external display
Resolution: ####x####  <!-- See: System Preferences, Displays -->
Low Power Mode: yes / no   <!-- Only on Montery/macOS 12. See: System Preferences, Battery -->

## Performance metrics

|Savegame           |Graphics rendering|Game speed factor|
|-------------------|-----------------:|----------------:|
|Hilbergen (HW off) |         ###.## ms|           ##.# x|
|Hilbergen (HW on)  |         ###.## ms|           ##.# x|
|Wentbourne (HW off)|         ###.## ms|           ##.# x|
|Wentbourne (HW on) |         ###.## ms|           ##.# x|

Background

Why these savegames?

We wanted to use a variety of savegames. Hillbergen is a rather simple savegame with a decent amount of trains, similar to what a casual player would have. Wentbourne is a savegame that really tries to push your computer to the max.

Specific Game options?

We like to get performance metrics using your actual game options. That's why we don't ask to change certain options as that gives us the widest range of metrics of how the game is being used.

Example report

System information

macOS version: 12.6
Model: MacBook Pro (15-inch, 2016)
Processor: 2,7 GHz Quad-Core Intel Core i7
Graphics: Intel HD Graphics 530 1536 MB
Power source: battery

Game options:

Base graphics: OpenGFX

Optional information:

Display: built-in Retina display
Window size: 1191x751
Low power mode: Yes

Performance metrics

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	6.5 ms	6 x
Hilbergen (HW on)	2.1 ms	11 x
Wentbourne (HW off)	20.4 ms	0.31 x
Wentbourne (HW on)	11.1 ms	0.38 x

Bouke · 2022-10-23T07:34:53Z

Bouke
Oct 23, 2022
Author

System information

macOS version: 11.6.2
Model: MacBook Pro (Retina, 15-inch, Late 2013)
Processor: 2,3 GHz Quad-Core Intel Core i7
Graphics: Intel Iris Pro 1536 MB
Power source: power adapter

Game options:

Base graphics: OpenGFX

Optional information:

Display: built-in Retina display
Resolution: 1680x1050
Low Power Mode: n/a

Performance metrics

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	7.8 ms	4.8 x
Hilbergen (HW on)	5.5 ms	18 x
Wentbourne (HW off)	33.2 ms	0.27 x
Wentbourne (HW on)	17.4 ms	0.37 x

0 replies

Bouke · 2022-10-23T09:23:50Z

Bouke
Oct 23, 2022
Author

System information

macOS version: 10.15.7
Model: iMac (27-inch, Late 2012)
Processor: 2,9 GHz Quad-Core Intel Core i5
Graphics: NVIDIA GeForce GTX 660M 512 MB
Power source: n/a

Game options:

Base graphics: OpenGFX

Optional information:

Display: built-in display
Resolution: 2560x1440
Low Power Mode: n/a

Performance metrics

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	3.7 ms	6 x
Hilbergen (HW on)	1.2 ms	24 x
Wentbourne (HW off)	17.7 ms	0.35 x
Wentbourne (HW on)	9.1 ms	0.45 x

0 replies

Bouke · 2022-10-25T19:39:12Z

Bouke
Oct 25, 2022
Author

System information

macOS version: 12.6
Model: MacBook Pro (16-inch, 2019)
Processor: 2,4 GHz 8-Core Intel Core i9
Graphics: AMD Radeon Pro 5500M 8 GB
Power source: power adapter

Game options:

Base graphics: OpenGFX

Optional information:

Display: built-in Retina display
Resolution: 1090x700
Low Power Mode: no

Performance metrics

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	3.2 ms	9 x
Hilbergen (HW on)	1.0 ms	29 x
Wentbourne (HW off)	12.8 ms	0.4 x
Wentbourne (HW on)	6.6 ms	0.5 x

0 replies

EmperorJake · 2022-10-26T10:07:43Z

EmperorJake
Oct 26, 2022

Here are my findings on my new Macbook Air. Hardware acceleration does provide a minor performance boost. Also when HW was off I encountered a graphical glitch where the cursor left ghost cursors behind.

As you can see the M2 is very powerful but I'm amazed that Wentbourne is even more resource intensive than NetTrans, which I added because it's what I usually use to test OTTD performance.

System information

macOS version: 12.6 Monterey
Model: MacBook Air 2022
Processor: Apple M2
Graphics: Apple M2
Power source: power adapter

Game options:

Base graphics: OpenGFX

Optional information:

Display: external display
Resolution: 1920x1080 (windowed)
Low Power Mode: no

Performance metrics

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	3.80 ms	80.0 x
Hilbergen (HW on)	2.20 ms	91.0 x
Wentbourne (HW off)	3.20 ms	0.95 x
Wentbourne (HW on)	2.15 ms	0.98 x
NetTrans (HW off)	3.70 ms	1.11 x
NetTrans (HW on)	2.80 ms	1.15 x

2 replies

James103 Oct 26, 2022

Can you please post the exact savegame file you used to test "NetTrans" if possible?
That way, other people can test with the file and get a fair comparison.

EmperorJake Oct 26, 2022

Can you please post the exact savegame file you used to test "NetTrans" if possible? That way, other people can test with the file and get a fair comparison.

It's not easy because it requires some obscure and customised NewGRFs but I'll try later

EmperorJake · 2022-10-27T01:04:19Z

EmperorJake
Oct 27, 2022

NetTrans, 5363-07-01.sav.zip

OK, here's the NetTrans save ~~and the GRFs required to run it~~. However, during testing on a fresh config file I've found a different performance problem: the save is completely unplayable (runs at 0.1fps) without first increasing the sprite_cache_size_px from 128 to 512. This is probably because it's a very GRF-heavy game compared to the other two, but that might make it a more interesting benchmark for graphics rendering. Increasing the sprite_cache_size seemed to have no noticeable performance impact on the unmodded saves.

3 replies

James103 Oct 27, 2022

With the provided NewGRFs and save, upon loading the save, the game automatically pauses and gives the following warning:
"Behavior of NewGRF 'Dutch Trainset NetTrans Edition' is likely to cause desyncs and/or crashes - It changed vehicle capacity for 'NetTrains B-set (Electric)' when not inside a depot or refitting"

Why is that the case? Is it safe for me to unpause and continue the savegame?

I'm using nightly version bfa155e and had no other NewGRFs besides the ones you posted and those I could find on BaNaNaS at the time of loading.

EmperorJake Oct 27, 2022

Yes I get that error too, just ignore it. I did fix it in a later save but couldn't upload that one because it was made in JGRPP

LordAro Oct 27, 2022
Maintainer

Please do not distribute GRFs on here - you generally don't have permission to do so, and especially not with the removal of all licensing information.

andythenorth · 2022-10-28T19:14:06Z

andythenorth
Oct 28, 2022

System information

macOS version: 12.6
Model: MacBook Pro (16-inch, 2021)
Processor: Apple M1 Pro (8 performance and 2 efficiency)
Graphics: Apple M1 Pro (16 core)
Power source: battery

Game options:

Base graphics: Original TTD

Optional information:

Display: built-in Retina display
Window size: 1728x1117
Low power mode: No
OpenTTD interface size: Double size
OpenTTD font size: Double size
Font: original base set pixel font
Full animation: enabled
Other system CPU load: minimal (< 10%)

Graphics rendering averages are highly variable, I took median values.

Savegame	Graphics rendering	Game speed factor
Hilbergen (HW off)	2.5 ms	80 x
Hilbergen (HW on)	1.5 ms	85 x
Wentbourne (HW off)	2.5 ms	0.83 x
Wentbourne (HW on)	1.5 ms	0.86 x

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[macOS] Benchmarks wanted: does hardware acceleration improve performance? #10100

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 6 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

[macOS] Benchmarks wanted: does hardware acceleration improve performance? #10100

Background

Why these savegames?

Specific Game options?

Example report

System information

Game options:

Optional information:

Performance metrics

Replies: 6 comments · 5 replies

Bouke Oct 23, 2022 Author

System information

Game options:

Optional information:

Performance metrics

Bouke Oct 23, 2022 Author

System information

Game options:

Optional information:

Performance metrics

Bouke Oct 25, 2022 Author

System information

Game options:

Optional information:

Performance metrics

System information

Game options:

Optional information:

Performance metrics

LordAro Oct 27, 2022 Maintainer

System information

Game options:

Optional information:

Replies: 6 comments 5 replies

Bouke
Oct 23, 2022
Author

Bouke
Oct 23, 2022
Author

Bouke
Oct 25, 2022
Author

LordAro Oct 27, 2022
Maintainer