New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a Rendering Benchmark Sample Package #549

Closed
ilexp opened this Issue Jul 9, 2017 · 5 comments

Comments

1 participant
@ilexp
Member

ilexp commented Jul 9, 2017

Summary

In order to keep track of performance and compare it across machines and Duality versions, it would be very nice to have a benchmark that can easily be installed and run.

Analysis

  • This came up originally in issue #326 and depends on some of the v3.0 rendering pipeline improvements.
  • Use a render-to-texture approach.
    • Full control over multisampling and size, allowing to perform tests with a constant number of pixels to fill, vs. varying resolutions and settings per-user.
    • Allow users to change resolutions / multisampling / etc. via hotkey inside each benchmark.
  • Benchmark scenes / cases:
    • Screen overlay vs world space
    • Lots of small (vertex count) renderers.
    • Few very big (vertex count) renderers.
  • Use procedurally generated objects to keep the scene files small.

@ilexp ilexp added this to the v3.0 milestone Jul 9, 2017

@ilexp ilexp changed the title from Create a Rendering Benchmark Sample to Create a Rendering Benchmark Sample Package Jul 9, 2017

@ilexp ilexp self-assigned this Jul 10, 2017

@ilexp

This comment has been minimized.

Member

ilexp commented Jul 10, 2017

Progress

  • Created a benchmarks-wip feature branch for working on this.
  • Set up a benchmarks project.
  • Set up a benchmarks sample package.

Immediate ToDo

  • Create a "baseline" testing scene with a simple sprite in the center of the screen and nothing else.
  • Implement a special RenderSetup that renders everything to an internal offscreen target and then blits it to output, followed by drawing stats in screen space.
  • Display minimalistic perf stats.
  • Display instructions, similar to physics sample.
  • Allow to adjust the setups internal offscreen target size and multisampling at runtime.
  • Allow to switch between scenes via keyboard input, similar to physics sample.
  • Implement various benchmarks.
@ilexp

This comment has been minimized.

Member

ilexp commented Jul 15, 2017

Progress

  • Created a BenchmarkRenderSetup to control rendering resolution, view size and antialiasing quality.
  • Set up a baseline scene containing only a SpriteRenderer.

Immediate ToDo

  • Add diagnostics overlay to BenchmarkRenderSetup.
  • Display minimalistic perf stats.
    • Current ms / frame
    • Lowpass ms / frame
  • Display instructions, similar to physics sample.
  • Implement a sample controller Component
    • Switch scenes, similar to physics sample
    • Adjust benchmark settings (resolution, AA, ...)
    • Set overlay text / instructions displayed by BenchmarkRenderSetup
  • Implement various benchmarks.
    • Applicable-to-all strategies:
      • When displaying "sprites", try to keep the count constant across samples so different rendering methods / paths are comparable.
      • Keep about half the renderers on screen, the other half off screen to include culling efficiency in the test.
    • Large number of SpriteRenderers, alternating materials
    • Large number of SpriteRenderers, same material
    • Small number of "MultiSpriteRenderers"
    • Large number of screen space single sprite renderers, alternating materials
    • Large number of screen space single sprite renderers, same material
    • Small number of screen space multi sprite renderers
    • Very large number of "dummy renderers", which do not add any vertices at all, but otherwise behave as if they would, including positive BoundRadius, IsVisible calls, CalcZOffset calls and similar.
@ilexp

This comment has been minimized.

Member

ilexp commented Jul 16, 2017

Progress

  • Added render setup info overlay.
  • Added performance info overlay.
  • Added benchmark info overlay.
  • Created benchmark scenes for:
    • Large number of SpriteRenderers, alternating materials
    • Large number of SpriteRenderers, same material
    • Small number of "MultiSpriteRenderers"
  • Removed the screen space sprite benchmarks from the ToDo for now, as they likely don't have a lot of additional value due to similar codepaths.

Immediate ToDo

  • Display GCs per minute instead of GCs overall.
  • Implement benchmarks for:
    • Large number of "dummy renderers", which do not add any vertices at all, but otherwise behave as if they would, including positive BoundRadius, IsVisible calls, CalcZOffset calls and similar.
    • Others..?
  • Implement an automated test run that goes through all available benchmark scenes, waits a few seconds (~30? for average GC count), samples the displayed values and stores them in a text report.
  • Run the benchmark, take note about all the measurements and post them both here and in every rendering optimization related issue before starting.
@ilexp

This comment has been minimized.

Member

ilexp commented Aug 6, 2017

Progress

  • Measuring GCs per simulated minute (3600 frames) instead of the total count since startup.
  • Added a dummy renderer sample where 10000 renderer objects are created, but none of them actually submit vertices to the device.

Immediate ToDo

  • Implement an automated test run that goes through all available benchmark scenes, waits a few seconds (~30? for average GC count), samples the displayed values and stores them in a text report.
  • Run the benchmark, take note about all the measurements and post them both here and in every rendering optimization related issue before starting.
  • Merge back to develop-3.0
@ilexp

This comment has been minimized.

Member

ilexp commented Aug 12, 2017

Progress

  • Moved benchmark controller code to the core plugin, so it will remember things across scenes.
  • Added an automated test run with text report generation.
  • Merged back to develop-3.0.

@ilexp ilexp closed this Aug 12, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment