Provide a pluggable method for asset transformation

[dnfield]

The tool already does transformation of icon fonts at build time.

It would be good to make an interface like that open and pluggable, so that packages external to the tool could decide to transform assets. For example, package:vector_graphics will want to transform SVG files into a different kind of file at build time by invoking a Dart program to process them.

@christopherfujino @jonahwilliams for input

[dnfield]

Some rough notes:

• The icon font shaker currently operates by getting fed all the files going into the asset bundle and sniffing the mime type. It then transforms the file and outputs a file with the same name and extension.
• Some transformers might want to replace existing files with brand new ones that have a different file name, but I feel like there's something I haven't thought all the way through here. Maybe keeping the file name the same is a good thing.
• We'll probably want ways to allow the manifest to specify which files get transformed and by which transformers, perhaps with regexps or globs for files?

[dnfield]

@jonahwilliams says he might be able to write a onepager on this. We talked some more this morning and agreed we don't need to make this work for the icon shaking, but it might help support some other features the tool has had requests for like platform specific assets.

[christopherfujino]

@jonahwilliams says he might be able to write a onepager on this. We talked some more this morning and agreed we don't need to make this work for the icon shaking, but it might help support some other features the tool has had requests for like platform specific assets.

SGTM

[Hixie]

It would be great if we could make both the icon font transformer and the license file munger components that just plug into this more generic framework.

[jonahwilliams]

one of these ways I tried to make this tractable was to massively simplify the scope of what we could run over. By just allowing asset -> asset transformations, we allow for a very simple API with almost no configuration required.

We must avoid making a generic build system, otherwise we'd probably be better taking something off the shelf. These usually come with severe tradeoffs and don't work in the same out of the box way that Flutter does today.

[Hixie]

oh totally, i wasn't suggesting making a build system. the icon transformer is asset->asset, right?
i agree that the license is more assets->assets, which is less of a good fit if the goal is simplicity.

[jonahwilliams]

its asset + compiled dart kernel -> asset, which relies on const finder and some tricky assumptions about how and when tree shaking applies to Dart code.

Potentially a better fit would be something like the shader features which use impellerc to do shader compilation. The complications there have to do with

• communicating the right set of configuration to the tooing (i.e. which backend to target, which features to add) which will change over time
• managing the (likely) breaking changes we'll need to make internal to impellerc in order to add support for more backends and to fix bugs once the feature is used more.

[Hixie]

Ah if you mean asset -> asset with literally no other inputs, not even inputs that would be the same for all assets, then yeah, the icon transformation doesn't fit either. A clean asset -> asset transformation with no dependencies at all is a very interesting subset of the problem space, because it can be parallelized very efficiently, so that makes sense to me.

Shader compilation seems like it has the same kind of needs (i.e. additional build-wide information) as the icon transformation.

[jonahwilliams]

Yeah, it has global configuration requirements, but it doesn't depend on the outputs of other build steps. So definitely a step in complexity from the asset -> asset transformations but not as complex as icons.

[Hixie]

Ah, fair. That distinction makes sense.

[andrewkolos]

but it might help support some other features the tool has had requests for like platform specific assets.

@dnfield Do you recall where to find such requests? I'm interested in known potential use cases for asset transformation outside of easier use of package:vector_graphics.

[marandaneto]

It'd be cool if this feature provides a pluggable API that you can extend and run your own code.
An example that comes to mind is associating the generated obfuscation-map (via save-obfuscation-map) file when sending eg error reports such that those reports can symbolicate obfuscated class names, types, etc, because the final app knows its obfuscation map id that was generated and uploaded during build time, but the event should be sent at runtime so it should know its id.
We already do this for Android and Proguard, but that requires hooking up into Gradle directly, or, for each platform, ideally there'd be hooks in the Flutter build process that you can do such things.

[sivang]

@christopherfujino @jonahwilliams @dnfield If this work is still up for grabs, I'm interested investigating if I can take on this "Good Starter Project" but would require some guidance as to what&where to read code pertaining to the enhancement discussed here. In particular, I'm keen on devising up a generic use case that would ideally be platform-independent. (@marandaneto use case is one such case I'd think could be catered for)

[andrewkolos]

It'd be cool if this feature provides a pluggable API that you can extend and run your own code...

The current scope of this feature is limited to pure asset-to-asset transformation, where "asset" refers to a file defined under assets in a project's pubspec.yaml (see Adding assets and images).

[andrewkolos]

If this work is still up for grabs, I'm interested investigating if I can take on this "Good Starter Project" but would require some guidance as to what&where to read code pertaining to the enhancement discussed here.

Hi @sivang, thanks for your interest in contributing; however, I'm already working on this feature.

[sivang]

Thanks @andrewkolos , I'm still interested in understanding the code behind this mechanism in the flutter build system, where to start reading? :)