Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance testing app #361

Merged
merged 15 commits into from
Jun 23, 2023
Merged

Performance testing app #361

merged 15 commits into from
Jun 23, 2023

Conversation

lilleyse
Copy link
Contributor

@lilleyse lilleyse commented Jun 21, 2023

Fixes #261

Follow up to #347 that fills out the performance testing application.

The UI is split into three categories:

  • Options - Different options to test with. More options could be added here in the future. These only apply when you click a new scenario. The Tracing Enabled option is there to remind you if you have tracing enabled. I would have preferred using a read-only checkbox but didn't see an option in omni.ui.
  • Scenarios - a list of scenarios to test. Most scenarios load a single tileset, set the camera at a specific location, and wait for the tileset to load (waiting for the cesium.omniverse.TILESET_LOADED event to fire). The "Tour" scenario loads a single tileset and jumps between different locations. You can click Stop to stop the current scenario. This is useful for stopping tours early.
  • Stats - stats measured during a run. For now only the duration is measured, but in the future we should add memory usage, number of tile requests, average FPS, etc.

Other notes:

  • The ion access token and google access token are hardcoded in the .py file. These are the same keys used in cesium-omniverse-samples and will need to be rotated monthly.

Known issues:

  • Sometimes the timer stops early. In some cases this is because Fabric materials are still loading even after cesium-native thinks it's done, but other times it seems like a bug in cesium-native computeLoadProgress.
  • Clicking a scenario while it's currently running often causes a crash, which is a bug in cesium-omniverse.
  • The code is in a workable state but I think it could be cleaned up and organized better

Future work:

  • Scenarios that follow camera paths
  • Option to disable cesium-native cache
  • Multiple trials per-scenario
  • More options and stats

image

And a video of it in action. I accidentally took this video in Debug mode so it's a bit slow.

perf-testing.mp4

Copy link
Contributor

@weegeekps weegeekps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment.

@@ -17,6 +17,7 @@ app.useFabricSceneDelegate = true
app.usdrt.scene_delegate.enableProxyCubes = false
app.usdrt.scene_delegate.geometryStreaming.enabled = false
app.fastShutdown = true
app.viewport.grid.enabled = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we not default this to off? It's easy enough to turn off manually and then it persists between sessions. I typically leave it on because it can be useful in some cases. By setting it off in this file, it turns it off every time for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I removed this line

Comment on lines +25 to +26
ION_ACCESS_TOKEN = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiIyZTA0MDlmYi01Y2RhLTQ0MjQtYjBlOS1kMmZhMzQ0OWRkNGYiLCJpZCI6MjU5LCJpYXQiOjE2ODU2MzExMTF9.y2CrqatkaHKHcj6NIDJ8ioll-tnOi-2CblnzI6iUays" # noqa: E501
GOOGLE_3D_TILES_URL = "https://tile.googleapis.com/v1/3dtiles/root.json?key=AIzaSyC2PMYr_ZaMJT5DdZ8WJNYMwB0lDyvx5q8"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought for later. We ought to set up a way to use environment variables for this sort of thing, rather than hard coding it in the code. Easier to rotate values when we need to do so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that would be better. I don't like hardcoding it either. I'll have a follow up PR for that.

Copy link
Contributor

@mattelser mattelser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lilleyse lilleyse merged commit 1a6cfbb into main Jun 23, 2023
@lilleyse lilleyse deleted the performance-testing-app branch June 23, 2023 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance testing application
3 participants