Support custom shaders

[dnfield]

From internal discussion, a user would like to apply a custom GL vertex or fragment shader program. This seems like it should be doable - although one thing I'm not sure about is what happens if you're running on a Vulkan or Metal backend.

/cc @liyuqian @chinmaygarde

/cc @clocksmith

[liyuqian]

Is it more or less similar to PlatformView? For example, create a gl surface so the custom GL shaders will render into that surface, and Flutter will then composite normal Flutter canvas(es) with that gl surface.

[dnfield]

I don't know if this is possible, but my thought was that you'd somehow be able to apply the shader as the shader on a Paint object. Maybe it doesn't work like that though.

Basically, you'd like to apply an arbitrary vertex or fragment shader to a widget.

[liyuqian]

I'm not sure if Skia supports putting arbitrary GL shaders into somewhere. I guess not so it won't break its Vulkan backend. It seems that you can only create an SkShader to put into a SkPaint object (which is basically what Paint is).

[dnfield]

Ahh I see. I guess in that case it would just make sense to have a PlatformView.

[cbenhagen]

@dnfield can you maybe push me into the right direction on how I would use a PlatformView to apply a fragment shader to a widget?

This could be a solution for #55738.

[dnfield]

You cannot use a platform view to apply a custom shader to a widget. Your platform view would have to apply it's custom shader to whatever it knew how to draw.

[cbenhagen]

@dnfield can we reopen this issue then? Being able to apply shaders would indeed be useful for many applications. Having to apply them on the platform side of the video_player plugin for example would mean that one has to fork and maintain the player and implement the shader for every single platform.

Maybe you might want to chime in on this Skia issue I opened https://crbug.com/skia/10274.

[cbenhagen]

@liyuqian It looks like things have changed since this issue was closed. In Skia milestone 81 SkRuntimeEffect and friends have been added which would allow custom SkSL shaders and color filters.

[liyuqian]

Yes, we've discussed with the Skia team about the new Skia APIs allowing custom SkSL shaders several months ago. At that time, it seems to be still in the experimental phase and not ready for exposing them through Flutter's public APIs.

@bsalomon : I wonder how it is now from Skia's perspective?
@chinmaygarde: I saw that you raised the SkSL custom shader in go/flutter-skia-sync on April 20th. I wonder if you have more updates on that?

[roman-petrov]

I found that some work is done on custom shaders by Flutter Team on https://github.com/chriscraws/ssir repository based on Flutter shaders design document. Are there any future plans to move this work forward or it will be freezed / just dropped?..

Pixel shaders is a great feature allowing to create amazing high-performance visual effects & animations so I would be really exited to see pixel shader support in Flutter.

[chriscraws]

@roman-petrov Hello! After a hiatus, we are resuming work on this! We are currently focusing on landing the necessary changes to the engine (we just had a review with the flutter team this morning). Once that is landed, development on the Dart library for generating shaders at the repository you linked will resume. Please feel free to add comments to that design doc, and follow code changes here https://github.com/chriscraws/engine/tree/shader-cc

The open issue to keep track of this is here: #58361

[roman-petrov]

Thank you, this is fantastic news! Flutter is one step closer to become most attractive and powerful UI development framework!

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.