[css-images] Mesh gradient / Freeform gradient / 2D gradient

[JoshTumath]

Mesh gradients and freeform gradients are becoming a popular design feature, and it would be useful to be able to generate mesh gradient images dynamically with a function in CSS. The nomenclature of 'mesh' vs 'freeform' seems to get conflated, but they are different things in tools like Adobe Illustrator.

Here's an example of a complex mesh gradient from our branding kit at the BBC:

Why do we need a new feature?

Tools such as CSS Hero try to replicate mesh gradients, but they only 'fake' it by layering radial gradients. You wouldn't be able to make a complex mesh like the one above using radial gradients.

How they work

Creating the gradients

Freeform gradients are generated by specifying any number of nodes at arbitrary coordinates on a canvas. Each node is given a colour.

Mesh gradients seem to be more complex, and work using a 2D grid of nodes. The grid lines can be bent using bezier curves.

It's easier to understand how they work by playing around with them yourself. Here are some tools and videos to explain them better:

• Freeform gradient tool that generates them in a HTML canvas
• YouTube video that explains how both mesh and freeform gradients are generated in Adobe Illustrator

How they are created

See #7648 (comment)

Possible syntax

For creating a freeform gradient, the gradient function could have a comma-separated list of coloured coordinates/nodes. I'm suggesting using <length-percentage> units for consistency with other gradient function syntaxes.

background-image: freeform-gradient(red 2% 2%, gold 60% 40%, yellow 90% 100%);

The syntax definition would be the following: (Thanks @SebastianZ)

freeform-gradient([<color> <length-percentage>{2}]#{2,})

I don't have a proposal for a mesh gradient syntax because they're more complex. EDIT: See discussion in the comments about how mesh gradients might be better represented using SVG markup.

[faceless2]

There was a <meshgradient> element suggested for SVG2, although it's been withdrawn. It remains in the svg-next proposals, see https://svgwg.org/svg-next/pservers.html#MeshGradients. It was originally proposed by the Inkscape team.

We've implemented it, it's complex and trying to represent it as a function would be horrendous. If it ever makes it back into SVG it will be usable as a background-image and I would suggest that's probably the best approach.

For future reference, these are also called "Coons Patch" (or "tensor-product patch" mesh gradients, for the bicubic interpolation mode) and the maths is all defined in section 8.7.4.5.6 of the PDF spec, older versions of which are still publicly available. The SVG model is almost identical.

[JoshTumath]

Thanks @faceless2 for that background on the SVGWG's proposal. It sounds like the amount of data you have to specify to create a Coons Patch mesh gradient is quite complex, so using SVG images rather than a function in CSS to make one sounds like a better approach.

As for what Adobe calls 'freeform gradients', the way they are specified seems much simpler. I would be interested to see if there is a published spec or mathematical model for them.

[SebastianZ]

We've implemented it, it's complex and trying to represent it as a function would be horrendous.

@JoshTumath already provided an example for a more simplistic approach.

Written in CSS syntax, it would be

mesh-gradient([<length-percentage>{2} <color>]#{2,})

I assume that would restrict the interpolation between the color stops to be linear, though that probably already covers a lot of use cases.
And the syntax could still be extended in the future to allow Coons Patches if required.

As I understand it, Adobe's freeform gradients are restricted to four colors, which probably also cover many use cases but it's a big restriction if you want to create arbitrary gradients.

Sebastian

[JoshTumath]

As I understand it, Adobe's freeform gradients are restricted to four colors, which probably also cover many use cases but it's a big restriction if you want to create arbitrary gradients.

If that was true in the past, it looks as though that restriction doesn't exist in Illustrator any more. A designer in my team, Ollie Buchanan, just created this example that has 7 points:

[faceless2]

@JoshTumath already provided an example for a more simplistic approach.
I assume that would restrict the interpolation between the color stops to be linear, though that probably already covers a lot of use cases.
And the syntax could still be extended in the future to allow Coons Patches if required.

If we're going to define a 2D gradient syntax I would rather we do it once, in such a way that it covers the options. Make the simple things easy and the difficult things possible.

So, fond as I am of CSS, I'm not sure it's the right place for this. There are all sorts of interesting uses for 2D gradients and the end result of all of them is going to be a very large function with hundreds of arguments. CSS tooling isn't designed for these - they're going to be impossible to prototype, edit or understand by hand.

If a concept requires structure to usefully represent it then we should start with a structured container, like XML (in the form of SVG), rather than trying to force it into a function where it will be represented as a very long list of numbers.

[bradkemper]

The simple, free form version in the syntax suggested seems reasonable to me.

[svgeesus]

Freeform meshes take a set of points and then perform a Delauney trianglation to produce a mesh. They then use bilinear interpolation to generate the intermediate colors.

Thus, our current linear gradients are 1D linear gradients, and freeform are 2D linear gradients.

See also

• https://people.eecs.berkeley.edu/~jrs/meshpapers/delnotes.pdf
• https://www.jstor.org/stable/43693155

[svgeesus]

Skia supports mesh gradients and GLSL also.

• https://meshgradient.com/

[LeaVerou]

+1. I think this syntax works, covers a lot of mesh gradient use cases, and is much easier to implement than mesh gradients.

Notably, it enables this use case which is currently impossible (any color pickers on the Web fake it by overlaying semi-transparent gradients or use WebGL):

Based on recent resolutions and syntax, I’d adjust the grammar to:

freeform-gradient( [<color-interpolation-method>]?, [ <color> <bg-position> ]#{1,} )

Changes:

• We want <color-interpolation-method> on every gradient
• We recently resolved to allow gradients with 1 color stop and no positions
• Using <bg-position> also allows keywords, with established rules about defaulting.

[ydaniv]

Wix also has this tool, kinda like CSS Hero's Mesher, that overlays radial gradients - called Fluid Gradients:

It's like ~3 years old now, though our intent was actually faking mesh gradients.

I also tried playing with voronois on canvas and faking the bilinear interpolation by adding blur on top of it:
https://codepen.io/ydaniv/pen/VwMrKme?editors=0010

I think it's somewhat close to the syntax above, generating points, triangulate using Delanuay, interpolate.
Big +1!

[neneodonkor]

This will be a cool and needful feature.

[SebastianZ]

Given the very positive feedback from outside and inside the working group and @LeaVerou's concrete proposal, I think it's ready to be discussed on a call.

Sebastian

[bernaferrari]

Apple has now implemented Mesh Gradient in SwiftUI: https://developer.apple.com/documentation/SwiftUI/MeshGradient

[xmacinfo]

Good use cases for mesh gradients are heatmaps displayed as overlays on web pages.

I do not have any example screen capture, but searching for “heatmap” Google should return a lot of examples.

[JoshTumath]

Apple has now implemented Mesh Gradient in SwiftUI: https://developer.apple.com/documentation/SwiftUI/MeshGradient

Thank you. And this seems like a true 'mesh gradient' like in Adobe Illustrator.

On a side note, earlier in the issue, we discounted mesh gradients as being too complex to implement in CSS and maybe more suited for future versions of SVG (which are unlikely to happen, given browser vendors' current stance on SVGs 😞). But I'd be interested to see if anyone would be willing to attempt a syntax for mesh gradients - maybe using the <cubic-bezier-easing-function> type.

It could be similar to the SwiftUI function, where the first argument is a matrix specifying the grid and the second is a corresponding matrix of colours.

[JoshTumath]

I published a blog post about this yesterday. I'd thought of some questions around the details of how a freeform gradient function might work:

The name

Is the name right? Is 'freeform gradient' just an Adobe term?

Spread

Adobe Illustrator supports increasing the spread of the colour stops (i.e. how dominant the colour is). How would we specify that? Could each colour stop have an optional <percentage> or <length-percentage> value for specifying this?

background-image: freeform-gradient(
  in oklab,
  red top left / 2%,
  gold 60% 40% / 4px,
  yellow bottom right %5
);

Specifying lines

Adobe Illustrator supports setting lines as well as points as colour stops. Is that something we would want to support in CSS, too? These would be some kind of linear path bezier path.

I know we have the <cubic-bezier-easing-function>, but that only accepts four arguments and doesn't allow you to create a continuous line with many bends on a set of coordinates. I imagine a new kind of positioning function might be needed. Maybe something like the path() function used in CSS Shapes?

Animation

We often see apps and websites use animated mesh gradients. I'd imagine this is computationally expensive to do unless it's pre-rendered.

I suppose, with this proposal, you could animate a freeform gradient by registering CSS custom properties using @property and animating the value changing.

[LeaVerou]

Wrt implementability, we need to investigate whether the graphics libraries used by browsers today have a primitive that makes this easy. E.g. can mesh gradients emulate freeform gradients? What about the libraries beyond Swift UI?

[chriskirknielsen]

I love this proposal, it would be a nice addition! As far as naming goes, I had cloud-gradient in mind (though I like freeform and mesh, too), as you're kind of painting soft blobs that look like clouds.

And to @JoshTumath's point, if spread were an option, we could end up creating really interesting and organic backgrounds (see Stripe's homepage for inspiration), maybe even end up painting something that looks like cells/scales, a bit Voronoi noise-like, perhaps?

[AmeliaBR]

Thanks for proposing this up @JoshTumath, and for your summary article bringing more attention.

As others have mentioned, a mesh gradient proposal was previously added to SVG & was implemented in Inkscape. I have never liked that syntax. It is both far too verbose to write by hand and (IMO) simplified in ways that obscure the logical structure. I like the proposed array of 2D colour points much better.

But, the SVG proposal is a direct representation of the model used in PDF and Illustrator & implemented in many rendering engines. Therefore, for ease of implementation, any CSS 2D gradient should be rendered in terms of an equivalent Coons-patch mesh gradient.

I assume that is true for the Adobe Illustrator "freeform" gradient. It would be helpful if someone from Adobe could confirm that & describe how the points are converted into a mesh. @svgeesus mentioned that it used Delauney triangulation, but are those triangles then somehow arranged into rows and columns? Does the "spread" factor affect the triangulation, or is it used like a colour hint to adjust the colour interpolation along the triangle edges, according to the relative weights of the spread factors at each vertex?

Before settling on a syntax, it is also worth considering the full potential of mesh gradients, so that CSS can be intentional about which subset it supports and why. Josh's blog post mentions that in addition to simple points, Adobe's "freeform" tool supports colour stops defined as curves or lines of solid colour. Should that also be supported?

The scope of the feature also influences the name. We shouldn't borrow Adobe's name if we're not going to fully represent their feature (and their feature set might change). If we might in future add a more mesh-like mesh gradient, we shouldn't use that name for the simple version. For the array of 2D colour stop points, I'd prefer a descriptive name like points-gradient() or scatterplot-gradient(). But if some of the colour stops are curves, then a more generic name might be better.

So, let's consider the potential scope. To quickly summarize how these Coons-patch mesh gradients work:

• Gradient colour stops are positioned at the intersections of set of row and column lines.
• Each line can actually be a curve, or angled in any direction, so they might not end up being vertical and horizontal, but the row and column relationships still define how colours are interpolated along the line and in between them.
• In between the lines, there is very complex but clearly defined math for how the colours fill in that grid cell, which is called a patch.
• The patches (grid cells) can be all sorts of shapes, so long as each can be described by a boundary of four Bézier path components (straight lines or single curve commands) connecting four colour stop positions.
• Triangular patches can be created by making one of the four colour stops overlap with another, with a zero-length path in between.
• Radial and conic-gradient effects can be created by bending either the row or column lines around into a loop where the end points overlap the starting points.

The markup structure from the SVG proposal is inspired by HTML table markup: It defines rows of cells (patches). The columns are implicit, based on the number of patches per row. Each patch logically has four stop points, but some of them are implicitly borrowed from the previous column in the row or row in the column. Each stop point also defines the curve of a segment of one of the row/column lines reaching to the next point. Figure from the spec shows the order in which each patch edge is defined, skipping the implicit ones:

Markup for that gradient

<meshgradient x="50" y="50" id="example"> <!-- x, y used for initial point in first patch. -->
  <meshrow> <!-- No attributes, used only to define begin/end of row. -->
	<meshpatch>
	  <stop path="c  25,-25  75, 25  100,0" stop-color="lightblue" />
	  <stop path="c  25, 25 -25, 75  0,100" stop-color="purple" />
	  <stop path="c -25, 25 -75,-25 -100,0" stop-color="red" />
	  <stop path="c -25,-25, 25,-75"        stop-color="purple" /> <!-- Last point not needed (closed path). -->
	</meshpatch>
	<meshpatch>
	  <stop path="c  25,-25  75, 25  100,0" /> <!-- stop-color from previous patch. -->
	  <stop path="c  25, 25 -25, 75  0,100" stop-color="lightblue" />
	  <stop path="c -25, 25 -75,-25"        stop-color="purple" /> <!-- Last point not needed (closed path). -->
	  <!-- Last path (left side) taken from right side of previous path (with points reversed). -->
	</meshpatch>
  </meshrow>
  <meshrow> <!-- New row. -->
	<meshpatch>
	  <!-- First path (top side) taken from bottom path of patch above. -->
	  <stop path="c  25, 25 -25, 75  0,100" /> <!-- stop-color from patch above. -->
	  <stop path="c -25, 25 -75,-25 -100,0" stop-color="purple" />
	  <stop path="c -25,-25, 25,-75"        stop-color="lightblue" /> <!-- Last point not needed (closed path). -->
	</meshpatch>
	<meshpatch>
	  <!-- First path (top side) taken from bottom path of patch above (with points reversed). -->
	  <stop path="c  25, 25 -25, 75  0,100" /> <!-- stop-color from patch above. -->
	  <stop path="c -25, 25 -75,-25"        stop-color="lightblue" /> <!-- Last point not needed (closed path). -->
	  <!-- Last path (left side) taken from right side of previous path (with points reversed). -->
	</meshpatch>
  </meshrow>
</meshgradient>
  

I don't like that the mesh curves are defined piece-meal for each stop. I don't like that the number of stops and curves per patch varies according to which row and column it is in. I don't like that the point defined within the stop geometry (the end-point of the curve segment) is not the point that receives the colour defined by that stop. I don't like that if you want a more complex curve between stops (more than a single path segment) you need to split a row or column in two and figure out the in-between colour not just for that segment, but for all the others. I don't like that it doesn't include colour-hints or interpolation colour spaces to adjust the colour change along each curve segment (those could probably be added, but might require changes to rendering engines). I don't like that if you want to create a radial or conic effect, you may nonetheless need to duplicate stops and segments, with the segment curve reversed.

As I mentioned at the top: I really don't like this syntax! It's only really suitable as output from other software.

But I do like all the results you can get from editing the mesh-line structure in a visual editor! Here are some examples I created in Inkscape for a future-focused sidebar in the book Using SVG with CSS3 and HTML5:

[bernaferrari]

@AmeliaBR how much do you like the Apple syntax? https://developer.apple.com/documentation/SwiftUI/MeshGradient

[tabatkins]

The Apple, syntax, fwiw, has a few forms:

• a list of points, associated with colors: the grid is straight-edged polygons, connecting the points.
• a list of "Bezier points", associated with colors: the grid is connecting the points with beziers; each "point" also contains (up to) 4 control points for the four beziers anchored at this point. (The other control point for each bezier is attached to the other "Bezier point".

Both syntaxes specify the width and the height of the grid points array, for interpreting the list into 2d. They allow adding a bg color which is defined as giving the color for any points outside the mesh, and a colorspace for interpolating the colors inside the mesh.

There's also a "smoothsColors" bool, which says it controls "Whether cubic (smooth) interpolation should be used for the colors in the mesh (rather than only for the shape of the mesh).". That appears to suggest that it interpolates the interior grid points via bicubic interpolation (to determine what points on the surrounding beziers contribute to that point, and to what amount), and can optionally do bicubic interpolation on the colors as well (rather than linear interpolation in the provided colorspace).

---

This syntax solves the most basic problem of the SVG syntax, which is differently-shaped arguments for different points of the grid - rather than define each patch one by one (requiring 2-4 edges), you define the points one by one and the beziers defining each patch are half-defined by each point.

It doesn't solve the "I want to split one patch edge, now I have to manually split all the edges in this row/col and manually interpolate their colors" issue, tho. But that's only important if we really want to make this meaningfully human-authorable, and I'm not sure to what extent this even can be.

[AmeliaBR]

@bernaferrari asked:

how much do you like the Apple syntax?

I agree with @tabatkins (and thanks for the summary, Tab): the Swift UI mesh gradient definition solves many of my complaints with the SVG syntax, although there remain some unavoidable complexities from the mesh-gradient structure:

• The grid is defined by the network of points (colour stop positions), not the patches between them.
• The curve control points (or none for straight lines) are clearly associated with each point.
• The order of points and how they relate to rows/columns of the grid is consistent.
• There's no worry about implicit points or curves: control points for curves that don't connect anywhere can be specified and ignored.
• The geometry and colours are given as separate objects, so that the same geometry (stored in a variable) can be re-used with a different colour set.
• But: it still requires an explicit row & column grid to be defined, and is therefore more complicated to author and adjust than the freeform scatterplot of colour points.

I don't like the practice of specifying a 2D structure as a 1D array with width & height attributes to parse it in a grid, but that's common in graphics programming & CSS has a lot more syntactic options to more clearly separate rows of values.

As a potential CSS value-definition syntax version of that, we'd be looking at something like:

mesh-gradient( [<color-interpolation-method> || <mesh-gradient-method>, ]?
                         mesh-points(<mesh-point-grid>) || mesh-colors(<mesh-color-grid>)  || <base-color> )

<mesh-gradient-method> = bilinear | bicubic

<mesh-color-grid> = <mesh-color>* [ / <mesh-color>* ]* 

<mesh-color> = <color> | auto 

<base-color> = <color>

<mesh-point-grid> = <mesh-point>* [ / <mesh-point>* ]* 

<mesh-point> = [ <position> || auto ] ( <mesh-control-point>#{1,4} )?

<mesh-control-point> = [<length-percentage> <length-percentage>]  | <angle>? && <length-percentage> 

The mesh-gradient-method would be bilinear by default, because I think that's simpler to render and I believe it's the default for PDF. (Although I don't really know the difference, so I defer to those who do!)

The four control points in the mesh-point data structure would be ordered & expanded in a TRBL way. Each control point is defined either as relative lengths in the X and Y axis from the point with percentages relative to the distance in that axis to the next point, or (optionally, my proposal) as an angle (defaulting to straight up, right, down, left as appropriate) and a length with percentages relative to the straight-line distance to the adjacent point on that edge. If omitted (or 0 length), the control point would be the point itself, creating a straight-line approach to that point.

(I'm assuming that if CSS adds a logical-directions mode to gradients in general, it would apply as a single keyword that then mirror-flips all geometry as necessary to convert top/bottom to block-start/block-end and right-left to inline-start/inline-end. In other words, I'm not worrying about logical directions at the micro-syntax level.)

Auto positions, or those that are omitted (that is, there aren't enough points in any row/column, or the whole position grid is skipped), would be automatically distributed similar to linear gradients: a first/last value in either axis defaulting to 0 or 100%, other stops without positions would be evenly spaced on each axis. (Which means it might be useful to extend the position syntax to support auto in one axis but not the other, but I'm not going to define the Value syntax for that.)

The base color fills in the space outside the mesh geometry (it's called background color in the Swift API, but I didn't want to confuse with CSS background layers: this is a color that is part of the gradient image). It defaults to transparent. Auto colors use the base color for the corner-points of the mesh, or are automatically interpolated from adjacent color points otherwise (exact algorithm to be defined, but the idea is that it would make it much easier to insert an extra row/column into the grid and only specify colours where you want to tweak them).

If colors are omitted (not enough values in each row/column to match the grid points) they would be auto. If you omitted the entire color grid, you'd just get a solid color gradient in the base color.

The only thing missing is a way to snap adjacent color points together to get a sharp color-change stripe. In other CSS gradients, you can do this by setting the position to 0 and letting the fix-up algorithm make the length/angle position match the previous point, since it can't be any less. For a mesh grid, I'm not sure whether that fix-up applies. Maybe there could be another position keyword, similar to auto, but to snap to the previous point in a row/column instead of to distribute points evenly?

That syntax definition is a lot. What would it look like in actual CSS code?

A simple 2D gradient like in @LeaVerou's colour picker example — where you define four corner colors and let the color interpolation space do the rest — would be defined like:

mesh-gradient(oklab, mesh-colors(lime, red / cyan, magenta) );
/* except you'd probably define the corner colors in oklab() functions */

The 4-patch / 9-point wavy gradient example from the SVG spec (for which I gave the proposed SVG markup, above) could be defined like this, making using of a variable for the repeated control point geometry of the repetitive wave structure:

--wavy: (-25% -25%, 25% -25%, 25% 25%, -25% 25%) ;
background-image: 
  mesh-gradient(
    mesh-points(
      50px  50px var(--wavy), 150px  50px var(--wavy), 250px  50px var(--wavy) /
      50px 150px var(--wavy), 150px 150px var(--wavy), 250px 150px var(--wavy) /
      50px 250px var(--wavy), 150px 250px var(--wavy), 250px 250px var(--wavy)
    )
    mesh-colors (
      lightBlue, purple, lightBlue /
      purple, red, purple /
      lightBlue, purple, lightBlue 
    )
  );

As far as CSS function values go, there's a lot of parts there! But I'd still say it's much more readable (and editable) than the proposed SVG version, and in 1/3 of the code (even if the repetitive geometry helped a lot for this example). I especially like how the colours can easily be read as a grid to get an idea of what it might look like.

The absolute dimensions for the points in this example aren't very CSS-y, but they could easily be replaced by percentages, side keywords, or calc expressions to make them offsets from the sides of the box. Any mix of relative, absolute, and percentage lengths can be resolved down to absolute lengths and positions at used-value time, which would then get sent to a rendering engine that's built for PDF-style mesh gradients.

If CSS were to add both something like this detailed mesh-gradient syntax and a simpler freeform/scatterplot syntax, I think it would meet the objective to "make easy things easy and make difficult things possible." (But we still need a clear definition of exactly what Adobe Illustrator is doing with those scatterplot points & whether they can be rendered as an equivalent mesh gradient.)

And then SVG would just have to support CSS gradients as fill / stroke, and it would get them too! 😜

[AmeliaBR]

For the CSS WG meeting, if it is decided to pursue this further, people should be tasked to collect the following information:

• What is the best open specification for mesh gradient rendering that is consistent with PDF & other common rendering environments?
• For the Adobe Illustrator team: How exactly are the "freeform" gradients rendered, and is this defined somewhere that can be incorporated freely into CSS?
• For the browser teams & other rendering engines: Do you already have an implementation of mesh gradients for PDF in the browser? Or is one available in the underlying graphics rendering engines you use? What features does it support (bicubic interpolation, multiple colour spaces, etc)?

Basically, the question is: If CSS defines a syntax for mesh gradients, will the CSS layout & rendering engine be able to call on the underlying graphics library to render them with existing code, after converting the CSS syntax into a mesh of absolutely-positioned grid and control points?

[bernaferrari]

Additional point: mesh is probably (even if slightly) computationally heavy. I saw people on Apple devices using a blur hash transition to the mesh gradient. A fallback API for the blur hash would potentially be useful. https://x.com/DLX/status/1801489961033552336

[AmeliaBR]

A fallback API for the blur hash would potentially be useful

Regardless of whether or not it is useful as a pre-render fallback for mesh gradients, the BlurHash algorithm is used on the web to create placeholders for lazy-loading images. It could be helpful to define it directly in CSS & save a bunch of JavaScript/canvas code.

The BlurHash technique creates a visual effect similar to a 2D gradient, so it seems reasonable to discuss it here. (It is actually a compressed string notation for a very small pixel-grid image, which is then drawn scaled up, letting the rendering environment's image-scaling algorithms blur the pixels together to create a gradient-like effect.)

See the BlurHash algorithm. That GitHub repo is MIT licensed & has rendering implementations in many languages, so hopefully it would be OK for incorporation into CSS (but I am not a lawyer).

[bernaferrari]

Technically if Mesh Gradient is implemented in a "fast enough" way (even if at loss of some quality), it would be possible to implement a BlurHash fallback in CSS for images. Would be super super cool.

[JoshTumath]

For the CSS WG meeting, if it is decided to pursue this further, people should be tasked to collect the following information:

• For the Adobe Illustrator team: How exactly are the "freeform" gradients rendered, and is this defined somewhere that can be incorporated freely into CSS?

CC @dirkschulze are you able to help us find out how freeform gradients in Illustrator are rendered? Is it still ultimately a mesh gradient or something else entirely?

[JoshTumath]

For reference, here is Apple's WWDC talk on SwiftUI's new mesh gradients API.
https://www.youtube.com/watch?v=alhFwkbsxrs&t=492s