Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing benchmark data at the end of the game #16328

Merged
merged 1 commit into from Jun 1, 2019

Conversation

@teinarss
Copy link
Contributor

commented Mar 19, 2019

Updated the benchmark mode to handle all perfsamples and writing to disk at the end.

OpenRA.Game/Support/Benchmark.cs Show resolved Hide resolved
OpenRA.Game/Support/Benchmark.cs Outdated Show resolved Hide resolved

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from bbef283 to f4ae266 Mar 20, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented Mar 20, 2019

Update: Fixed the issues

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented Mar 24, 2019

On hold! Got some new ideas that i want to implement in this PR before merge.

@pchote

This comment has been minimized.

Copy link
Member

commented Apr 21, 2019

Do you plan to revisit this soon? Might be best to close this if not.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from f4ae266 to ad663c1 Apr 30, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

Updated

Added support to launch a map directly and exit when the game is over

The use case is to support automate running bechmarks several times for getting better data.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from ad663c1 to fa49bad Apr 30, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented Apr 30, 2019

Updated

@abcdefg30

This comment has been minimized.

Copy link
Member

commented May 16, 2019

Needs a rebase.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from fa49bad to 4420a05 May 25, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 25, 2019

Rebased.
Did some more changes after developing a more automated system to run it several times.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from 4420a05 to 93627a1 May 25, 2019

@pchote pchote removed the PR: Rebase me! label May 25, 2019

@pchote
Copy link
Member

left a comment

Some thoughts on streamlining this:

OpenRA.Game/Support/LaunchArguments.cs Show resolved Hide resolved
OpenRA.Game/Game.cs Outdated Show resolved Hide resolved
OpenRA.Game/Game.cs Outdated Show resolved Hide resolved
@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 26, 2019

@pchote

This comment has been minimized.

Copy link
Member

commented May 26, 2019

How important is the ability to set the benchmark output folder?
Another option could be to have just Benchmark which takes either a map uid or replay.

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 26, 2019

Its most for convenience, im using it to mark the data dumps like this:
baseline-0-Pathfinder.csv

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from 93627a1 to fbd6f1e May 28, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 28, 2019

Updated and rebased

OpenRA.Game/Game.cs Show resolved Hide resolved
OpenRA.Game/Game.cs Outdated Show resolved Hide resolved
public string Benchmark;

[Desc("Automatically start playing the given map.")]
public string Map;

This comment has been minimized.

Copy link
@pchote

pchote May 31, 2019

Member

This will also be really useful for people working on scripted maps, so good call not merging it with Benchmark.

Maybe this will also perk @MustaphaTR or @abcdefg30's interest to review this PR.

This comment has been minimized.

Copy link
@abcdefg30

abcdefg30 Jun 1, 2019

Member

Fwiw, editing a Lua script does not change the hash (or path), but one can just use the restart map function to reload the script anyway.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from fbd6f1e to 4960d63 May 31, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 31, 2019

updated!

@pchote
Copy link
Member

left a comment

One last simplification then LGTM

OpenRA.Game/Support/Benchmark.cs Outdated Show resolved Hide resolved
OpenRA.Game/Support/Benchmark.cs Show resolved Hide resolved

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from 4960d63 to ef99129 May 31, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented May 31, 2019

Updated

@abcdefg30
Copy link
Member

left a comment

Looks good to me. From the IRC discussion: We want to write benchmark data when using "Abort Mission", too.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from ef99129 to 2074bbc Jun 1, 2019

@teinarss

This comment has been minimized.

Copy link
Contributor Author

commented Jun 1, 2019

Updated: Added support to abort mission, using World.Dispose() required a manuel input when using a scripted map with a timed win condition.

@teinarss teinarss force-pushed the teinarss:benchmark_mode branch from 2074bbc to 3fedb48 Jun 1, 2019

@pchote
pchote approved these changes Jun 1, 2019

@abcdefg30 abcdefg30 merged commit 4fae77e into OpenRA:bleed Jun 1, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@abcdefg30

This comment has been minimized.

Copy link
Member

commented Jun 1, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.