Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance measurement usability #102189

Open
dnfield opened this issue Apr 20, 2022 · 4 comments
Open

Improve performance measurement usability #102189

dnfield opened this issue Apr 20, 2022 · 4 comments
Labels
a: tests "flutter test", flutter_test, or one of our tests c: performance Relates to speed or footprint issues (see "perf:" labels) d: devtools DevTools related - suite of performance and debugging tools engine flutter/engine repository. See also e: labels. framework flutter/packages/flutter repository. See also f: labels. P2 Important issues not at the top of the work list team-engine Owned by Engine team triaged-engine Triaged by Engine team

Comments

@dnfield
Copy link
Contributor

dnfield commented Apr 20, 2022

As a developer, I want to run some portion of my app and get information about:

  • How many frames were rendered
    • Average/P90/P99 build times
    • Average/P90/P99 raster times
    • Information about the top N worst build/raster frames
  • How much time was spent in network or disk IO from dart:io
    • The ability to break this down by what files/hosts were accessed and how long it took
  • How much time was spent in platform channels
    • The ability to break this down by channel name/method call
  • GC stats:
    • Total UI thread time in GC
    • Total time of other thread(s) in GC
    • How many GCs ran/what types
  • Average/P90/P99 memory usage during the run
  • Average/P90/P99 CPU usage during the run
  • Average/P90/P99 GPU usage during the run

I would like to get this information with minimal impact on actual performance of my application.

A lot, but not all of this information is available in the timeline today. Some of this information is currently provided on some platforms, whereas on other platforms it is missing or only available via polling that devtools may do.

It would probably be best to start with some of our existing performance benchmarks and see what we can do to get the output to look more like this. We should also evaluate whether we can use timeline/perfetto tracing or not and how much of an impact it is having on performance as part of this bug.

/cc @gaaclarke @iskakaushik - I think this intersects with some of the work you're currently doing.
/cc @jiahaog fyi
/cc @kenzieschmoll @polina-c @jacob314 for devtools input
@dnfield dnfield added a: tests "flutter test", flutter_test, or one of our tests framework flutter/packages/flutter repository. See also f: labels. engine flutter/engine repository. See also e: labels. c: performance Relates to speed or footprint issues (see "perf:" labels) d: devtools DevTools related - suite of performance and debugging tools P2 Important issues not at the top of the work list labels Apr 20, 2022
@dnfield dnfield mentioned this issue Apr 20, 2022
Closed
@dnfield dnfield mentioned this issue May 9, 2022
Open
@ueman
Copy link
Contributor

ueman commented May 10, 2022

Ideally I'd like to have this data also available in release builds, so I can integrate this data with application performance monitoring tools, like Sentry, DataDog and so on.

How much time was spent in network or disk IO from dart:io

I'm currently experimenting with using HttpOverrides and IOOverrides to achieve this. That works quite good.

@dnfield
Copy link
Contributor Author

dnfield commented May 11, 2022

Is it that you want to test in release, or you want your release app sending back metrics for use?

@ueman
Copy link
Contributor

ueman commented May 11, 2022

Is it that you want to test in release, or you want your release app sending back metrics for use?

The latter one. I want to send back metrics for use.

@marandaneto
Copy link

It'd be awesome if at runtime you could collect telemetry data and send it over to observability services such as sentry.io to aggregate and extract information about the performance of your app and compare it across new versions of the app, device models, brands, etc...

@dnfield dnfield mentioned this issue May 24, 2022
Merged
8 tasks
@flutter-triage-bot flutter-triage-bot bot added multiteam-retriage-candidate team-engine Owned by Engine team triaged-engine Triaged by Engine team labels Jul 8, 2023
@jiahaog jiahaog mentioned this issue Sep 18, 2023
Merged
8 tasks
fluttermirroringbot pushed a commit that referenced this issue Sep 21, 2023
@jiahaog
Rename debugProfilePlatformChannels to a constant that works in rel…
f92884c 
…ease mode (#134922)

When it comes to startup profiling, it is very helpful to look at platform channels. `debugProfilePlatformChannels` today only works in debug and profile mode. Unfortunately, using profile mode is less accurate for startup profiling, because of the service isolate introducing additional overhead.

This PR allows this toggle to work in release mode. Note that there are two parts to `debugProfilePlatformChannels`:

- Adding timeline events
- Logging statistics about platform channels

I also considered adding a separate toggle to limit the scope of this change to the former, but that seems like complexity that we might not need at this time.

Towards #102189
Mairramer pushed a commit to Mairramer/flutter that referenced this issue Oct 10, 2023
@jiahaog @Mairramer
Rename debugProfilePlatformChannels to a constant that works in rel…
fe8e40c 
…ease mode (flutter#134922)

When it comes to startup profiling, it is very helpful to look at platform channels. `debugProfilePlatformChannels` today only works in debug and profile mode. Unfortunately, using profile mode is less accurate for startup profiling, because of the service isolate introducing additional overhead.

This PR allows this toggle to work in release mode. Note that there are two parts to `debugProfilePlatformChannels`:

- Adding timeline events
- Logging statistics about platform channels

I also considered adding a separate toggle to limit the scope of this change to the former, but that seems like complexity that we might not need at this time.

Towards flutter#102189
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
a: tests "flutter test", flutter_test, or one of our tests c: performance Relates to speed or footprint issues (see "perf:" labels) d: devtools DevTools related - suite of performance and debugging tools engine flutter/engine repository. See also e: labels. framework flutter/packages/flutter repository. See also f: labels. P2 Important issues not at the top of the work list team-engine Owned by Engine team triaged-engine Triaged by Engine team
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Hixie @ueman @marandaneto @dnfield