New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wire up Metal shader precompilation from offline training runs. #79298
Comments
The precompilation of harvested SKSL's has been wired up. Working on testing and benchmarking this in the host harness. |
This commit depends on [Skia support for Metal SKSL precompilation][1]. When a new Metal onscreen context is acquired, the engine will attempt to load shaders stored in the `io.flutter.shaders.json` file packaged in the Flutter asset bundle. This file can be [obtained via offline training runs][2]. This works very similarly to the OpenGL backend. However, in the OpenGL backend, shell initialization waits for precompilation to be completed. On the other hand, in the Metal backend, precompilation happens immediately after shell initialization. This is in an attempt to parallelize SKSL precompilation with Dart isolate and framework setup. While it may be possible to parallelize precompilation of SKSLs in the future, they are processed serially today. Fixes flutter/flutter#79298. [1]: https://bugs.chromium.org/p/skia/issues/detail?id=11392 [2]: https://flutter.dev/docs/perf/rendering/shader#how-to-use-sksl-warmup
This commit depends on [Skia support for Metal SKSL precompilation][1]. When a new Metal onscreen context is acquired, the engine will attempt to load shaders stored in the `io.flutter.shaders.json` file packaged in the Flutter asset bundle. This file can be [obtained via offline training runs][2]. This works very similarly to the OpenGL backend. However, in the OpenGL backend, shell initialization waits for precompilation to be completed. On the other hand, in the Metal backend, precompilation happens immediately after shell initialization. This is in an attempt to parallelize SKSL precompilation with Dart isolate and framework setup. While it may be possible to parallelize precompilation of SKSLs in the future, they are processed serially today. Support for testing shells (that hold the Skia persistent cache) using the Metal backend did not exist and has been added in this patch. However, I don't believe this testing is sufficient because it only verifies that SKSLs can be dumped and subsequently reused with a Metal backed Shell. Importantly, it doesn't verify where is shell setup this precompilation happens. For that, the persistent cache interface will need to be reworked to better fit with the shell unittests. I considered it out of scope and will followup with those test updates in a later patch. I have also made tracing of precompilation consistent with OpenGL and Metal (and Vulkan when support for that is added). The `PersistentCache::PrecompileKnownSKSLs` trace will summarize the number of SKSLs being precompiled (it is a counter trace) and the time taken to precompile each. Fixes flutter/flutter#79298. [1]: https://bugs.chromium.org/p/skia/issues/detail?id=11392 [2]: https://flutter.dev/docs/perf/rendering/shader#how-to-use-sksl-warmup
Is this available in master to try now? |
will this fix iOS shader jank issue? (using metal) |
See the guide here on how to utilize it: https://flutter.dev/docs/perf/rendering/shader#how-to-use-sksl-warmup |
I thought that tutorial doeisnt work on metal @acoutts |
@alectogeek ok cool thanks lol that's what I figured sorry back to the original question is this already available in master ? And are any changes required from the initial tutorial? |
Seems awesome! Do you know, how to use shaders warmup with |
You just need to add it to your build command, as per that linked guide. |
I don't use build command. My CI script:
and my Fastfile: default_platform(:ios)
platform :ios do
desc "Push a new beta build to TestFlight"
lane :beta do
increment_build_number(build_number: "$CI_PIPELINE_IID")
build_app(workspace: "Runner.xcworkspace", scheme: "Runner")
upload_to_testflight
end
end hmm, looked up the documentation for xcargs | Pass additional arguments to xcodebuild for the build phase. Be sure to quote the setting names and values e.g. OTHER_LDFLAGS="-ObjC -lstdc++" |
Is it possible to run the app with the pre-compiled shaders and not building it? To test out if the "fix-yank-step" actually works before doing the build command. Not sure if i make sense or this is a silly question. |
You need to build an adhoc-signed IPA and drag/drop it in the Xcode devices window onto the device to install it. |
It would not be possible to add a command like: |
It actually works! Device : Iphone 8 What I did was switch to flutter master branch and flutter upgrade, followed the same steps on https://flutter.dev/docs/perf/rendering/shader & built an adhoc-signed IPA and drag/drop it in the Xcode devices window onto the device to install VIdeo: Wihout shader warm up(first run): https://drive.google.com/file/d/1_cHkMleeI-eRhxehq4ikF0Gv3ZE9CTBE/view?usp=sharing PS. I used the code from #76180 |
A huge improvement for iOS. Nice job, Flutter team. |
What about portability to other devices? |
is this available in the dev branch or still only on master? |
@m-j-g Yes, this is available on the dev branch. |
Is there anyway to run this locally? |
Is this available in 2.2 on stable? I can't seem to figure out exactly where this is. |
It's currently on the dev channel @lukepighetti |
Yes please use a command like this:
please make sure you are on the DEV channel as of right now I have tried this solution on the example project as mentioned above and also on our production version which is already a huge app. It is MASSIVELY improving the first jank issue. It is basically gone. |
I followed the instructions on https://flutter.dev/docs/perf/rendering/shader#how-to-use-sksl-warmup but when I press M on the command line nothing happen, no file or folder created. Can someone hep me? Thank you. [✓] Flutter (Channel dev, 2.3.0-16.0.pre, on macOS 11.4 20F71 darwin-x64, locale en-GB) [✓] Xcode - develop for iOS and macOS [✓] Chrome - develop for the web [✓] Android Studio (version 4.2) |
@FelixYew you have to press Shift + m which results in M |
I have tested the SKSL Shader on my latest app version and the results are amazing, however, I noticed also that the boot up time of the app to start after the splash screen is significantly longer. Would that be because of the dev channel? |
@MarcoSoares11 The bundled SkSL shaders are precompiled to machine code on first-run when the application starts. This delay will persist even after the feature makes it to the beta and stable channels. You can reduce the delay by bundling fewer shaders. (We understand this is not a great user experience and we are continuing to pursue improvements in this area.) |
@zanderso Dumb question, but how do I have multiple devices in one file if it creates a new file every time I hit |
The animations are stored as shader code. The code is compiled on startup for your device specific. So in theory you have to do it only on one device because it is not device specific. |
@bennibau It says "For best results, capture SkSL shaders on actual Android and iOS devices separately." in the guide and on top of that animations are usually a bit different based on tablet vs phone etc. Is there not a way to combine all the shaders in one file? |
@SM2DevLLC I use my personal iPhone to capture the SkSL shaders and it has an incredible performance when I test on Android. For best practices if you have access to both an Android and an iOS device it would be recommended to do the capturing in separate devices as the device is specified in the SkSL file. |
@MarcoSoares11 Thanks. Yeah I have been doing both devices, but that's the issue. Each device makes a different file. So iPhone, iPad, Android, etc. Is there a way to combine the files? The other issue is that the landscape iPad view is different, so my assumption would be the shaders are slightly different, so i'd need to have the iPad and iPhone version in one file. |
@SM2DevLLC Bundling multiple SkSLs is not yet implemented. Please give a thumbs-up to #73098 to indicate your interest in that feature. |
@zanderso Oh ok. So I can't really do iPad and iPhone since they would need to be in the same file, so i'd need to pick one for now to build for release? |
I have the problem that I publish my app via Xcode and how can I attach the sksl file there? If I go via Xcode -> Product -> Archive then I can't set anything else there. |
@b3nni97 you don't have to attach the sksl file manually. It is enough to use |
Tried it on my app, it really reduce jank (Not all, there still jank, I know is jank because it dissapear after third open)... but, now my app takes more than 10 secs on the splash screen, and sometime when I restart the phone, it can take more thank 30 sec. Only happens on first app install, and when I restart the phone. On a iPhone 11 Pro Max |
Question, when I do the warm process, Do i only need to trigger the animation with jank only one time? or do I need to run it mutiple times until it runs smooth? does it affect at all the size of the .json and app start time? |
I'm having a hard time tracking where this is committed. Is there a version number that has this implemented? Trying to stay on top of this so that I'm aware when it makes it to beta/stable |
Same here, is there a way to know when we can get out of the dev channel while keeping Metal shader caching? |
On a Mac, you need to use Shift + m. Yeah who would have thought M means Shift + m? Probably only me. Kind of silly when the command line literally needs the capital M instead of just m. Flutter team as a whole is really awesome! But sometimes I find the new guy coming in and can't code the command line properly. There's nothing in the command line output that says press Shift + m. Also one more thing, we have flutter build ios --release --bundle-sksl-path but why didn't they also make flutter run --release --bundle-sksl-path too, just to make our lives less miserable so that we won't need to archive into ipa, sign with adhoc and then manually deploy to the device. To make it more head spinning, flutter run --release --no-build actually ignores the --no-build flag and rebuilds anyway. I mean, whoever writes this command line tool seriously needs to sit down and reflect what's going on since 2019 until now 2021. |
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of |
Now that https://bugs.chromium.org/p/skia/issues/detail?id=11392 has landed and been rolled in, preparation of Metal pipelines from harvested SKPs can be implemented. This is a tracking bug to make sure lifecycle and threading implications of the attempt at engine launch are reasoned about before enabling this on all renderer launch attempts.
The text was updated successfully, but these errors were encountered: