Factorio ships with a headless version of the game designed for hosting servers. This version of the game supports the --benchmark mode.
Pros to switching to the headless version:
The game loads much quicker, because a majority of the assets are not loaded.
The download size is smaller, allowing quicker redeploys.
Cons to switching to the headless version:
The headless version performs around 10% faster than the regular version. This can lead to a discrepancy between results.
It is more of a pain to build maps for testing from the headless version. Might be worked around by placing both headless and regular versions of the executable in the same directory.
Unknown:
The headless version on Windows may not need output redirection workarounds. If it does not, it is a strong case for switching.
EDIT: The headless version is not available for Windows
Factorio ships with a headless version of the game designed for hosting servers. This version of the game supports the --benchmark mode.
Pros to switching to the headless version:
The game loads much quicker, because a majority of the assets are not loaded.
The download size is smaller, allowing quicker redeploys.
Cons to switching to the headless version:
The headless version performs around 10% faster than the regular version. This can lead to a discrepancy between results.
It is more of a pain to build maps for testing from the headless version. Might be worked around by placing both headless and regular versions of the executable in the same directory.
Unknown:
The headless version on Windows may not need output redirection workarounds. If it does not, it is a strong case for switching.
EDIT: The headless version is not available for Windows