Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce SkiaVisual, SKCanvasElement and GLCanvasElement to allow externally adding sophisticated graphics #16621

Draft
wants to merge 32 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5996d88
feat: add a SkiaVisual that allows external use of SkiaSharp
ramezgerges Jan 25, 2024
29593f0
chore: add SKCanvasElement and a lot of details
ramezgerges Jan 26, 2024
e791d0f
chore: remove unneeded dpi logic in SKCanvasElement
ramezgerges Jan 29, 2024
24c13f5
test: add tests for SKCanvasElement
ramezgerges Jan 29, 2024
d152d00
chore: rename RespectFlowDirection to MirroredWhenRightToLeft
ramezgerges Jan 29, 2024
6767e38
docs: add documentation for SkiaCanvas and SKCanvasElement
ramezgerges Jan 30, 2024
d37eff8
chore: minor adjustments
ramezgerges May 7, 2024
8154ac1
chore: relocate implementation
ramezgerges May 7, 2024
f5d0093
feat: introduce the ability to draw using raw OpenGL on skia
ramezgerges Feb 21, 2024
e1ee33d
chore: refactoring and major cleanup
ramezgerges May 7, 2024
47a9bd1
docs: typos and tiny changes
ramezgerges May 7, 2024
6a92cdf
chore: session.Surface.Canvas => session.Canvas
ramezgerges May 7, 2024
b9e66cd
chore: RespectFlowDirectionChanged => OnMirroredWhenRightToLeftChanged
ramezgerges May 7, 2024
8d4f174
chore: use Visual.Size instead of LayoutSlot in SKCanvasElement
ramezgerges May 7, 2024
e9ca77d
test: add ui sample for GLCanvasElement
ramezgerges May 7, 2024
d6bafa5
docs: formatting
ramezgerges May 7, 2024
3527ce4
chore: formatting
ramezgerges May 8, 2024
b3ea410
chore: move GLCanvasElementImpl to a separate skia-only file
ramezgerges May 8, 2024
e2e742f
chore: reduce unsafe usage and restore Viewport after render
ramezgerges May 8, 2024
27c7689
chore: add GLCanvasElement support for wpf
ramezgerges May 8, 2024
f133c61
chore: clear canvas when rendering GLCanvasElement
ramezgerges May 8, 2024
0061e6d
docs: more changes
ramezgerges May 8, 2024
df5109c
chore: explicitly dispose GLVisual's pixmap
ramezgerges May 8, 2024
997470d
chore: move GKCanvasElement.GLVisual to a separate file
ramezgerges May 8, 2024
e16e747
chore: remove MirroredWhenRightToLeftProperty and rename sample
ramezgerges May 8, 2024
a0d197c
chore: adjust sample
ramezgerges May 8, 2024
b0c4862
docs: add a sample to the SKCanvasElement docs
ramezgerges May 8, 2024
fef0691
chore: make the sample animated
ramezgerges May 9, 2024
6d831a3
chore: licensing and formatting
ramezgerges May 9, 2024
b2b1ead
docs: tabs to spaces
ramezgerges May 9, 2024
ff1a3c9
chore: refactor timestamp calculations
ramezgerges May 10, 2024
a681ee9
chore: DispatchQueueRender on Idle to prevent choking the dispatcher …
ramezgerges May 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 177 additions & 13 deletions doc/articles/controls/SkiaCanvas.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,20 @@
uid: Uno.Controls.SKCanvasElement
---

# Introduction
## Introduction

During creating an Uno application, users might want to create elaborate 2D graphics that are more suitable to a 2D graphics library such as [Skia](https://skia.org) or [Cairo](https://www.cairographics.org), rather than using, for example, a simple [Canvas](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.controls.canvas). To support this use case, SkiaSharp comes with an [SKXamlCanvas](https://learn.microsoft.com/dotnet/api/skiasharp.views.windows.skxamlcanvas) element that allows for drawing in an area using SkiaSharp.
When creating an Uno Platform application, developers might want to create elaborate 2D graphics using a library such as [Skia](https://skia.org) or [Cairo](https://www.cairographics.org), rather than using, for example, a simple [Canvas](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.controls.canvas). To support this use case, SkiaSharp comes with an [SKXamlCanvas](https://learn.microsoft.com/dotnet/api/skiasharp.views.windows.skxamlcanvas) element that allows for drawing in an area using SkiaSharp.

On Uno Skia targets, we can utilize the pre-existing Skia canvas that is used internally by Uno to render the application window instead of creating additional Skia surfaces and then copying the resulting renderings to the application (e.g. using a [BitmapImage](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.media.imaging.bitmapimage)). This way, a lot of Skia functionally can be acquired "for free". For example, no additional additional packages are needed, and setup for hardware acceleration is not needed if the Uno application is already using OpenGL to render.
On Uno Platform Skia Desktop targets, we can utilize the pre-existing internal Skia canvas used to render the application window instead of creating additional Skia surfaces. It is then possible to copy the resulting renderings to the application (e.g. using a [BitmapImage](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.media.imaging.bitmapimage)). This way, a lot of Skia functionally can be acquired "for free". For example, no additional additional packages are needed, and setup for hardware acceleration is not needed if the Uno application is already using OpenGL to render.

This functionality is exposed in two parts. `SkiaVisual` is a [Visual](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.composition.visual) that gets an [SKCanvas](https://learn.microsoft.com/dotnet/api/skiasharp.skcanvas) to draw on and is almost completely unmanaged. For more streamlined usage, an `SKCanvasElement` is provided that internally wraps a `SkiaVisual` and can be used like any FrameworkElement, with support for sizing, clipping, RTL, etc. You should use `SKCanvasElement` for most scenarios. Only use a raw `SkiaVisual` if your use case is not covered by `SKCanvasElement`.
This functionality is exposed in two parts. `SkiaVisual` is a [Composition API Visual](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.composition.visual) that gets an [SKCanvas](https://learn.microsoft.com/dotnet/api/skiasharp.skcanvas) to draw on and is almost completely unmanaged. For more streamlined usage, an `SKCanvasElement` is provided that internally wraps a `SkiaVisual` and can be used like any `FrameworkElement`, with support for sizing, clipping, RTL, etc. You should use `SKCanvasElement` for most scenarios. Only use a raw `SkiaVisual` if your use case is not covered by `SKCanvasElement`.

We stress that this functionality is only available on Uno targets that are based on Skia.
> [!IMPORTANT]
> This functionality is only available on Skia Desktop (`net8.0-desktop`) targets.

# SkiaVisual
## SkiaVisual

A `SkiaVisual` is an abstract `Visual` that provides Uno applications the ability to utilize SkiaSharp to draw directly on the Skia canvas that is used internally by Uno to draw the window. To use `SkiaVisual`, create a subclass of `SkiaVisual` and override the `RenderOverride` method.
A `SkiaVisual` is an abstract `Visual` that provides Uno Platform applications the ability to utilize SkiaSharp to draw directly on the Skia canvas that is used internally by Uno to draw the window. To use `SkiaVisual`, create a subclass of `SkiaVisual` and override the `RenderOverride` method.

```csharp
protected abstract void RenderOverride(SKCanvas canvas);
Expand All @@ -26,9 +27,9 @@ Note that you will need to add your own logic to handle sizing and clipping.

When adding your drawing logic in `RenderOverride` on the provided canvas, you can assume that the origin is already translated so that `0,0` is the origin of the visual, not the entire window.

Additionally, `SkiaVisual` has an `Invalidate` method that can be used at any time to tell the Uno runtime to redraw the visual, calling `RenderOverride` in the process.
Additionally, `SkiaVisual` has an `Invalidate` method that can be used at any time to tell the Uno Platform runtime to redraw the visual, calling `RenderOverride` in the process.

# SKCanvasElement
## SKCanvasElement

`SKCanvasElement` is a ready-made `FrameworkElement` that creates an internal `SkiaVisual` and maintains its state as one would expect. To use `SKCanvasElement`, create a subclass of `SKCanvasElement` and override the `RenderOverride` method, which takes the canvas that will be drawn on and the clipping area inside the canvas. Drawing outside this area will be clipped.

Expand All @@ -40,17 +41,180 @@ By default, `SKCanvasElement` takes all the available space given to it in the `

Note that since `SKCanvasElement` takes as much space as it can, it's not allowed to place an `SKCanvasElement` inside a `StackPanel`, a `Grid` with `Auto` sizing, or any other element that provides its child(ren) with infinite space. To work around this, you can explicitly set the `Width` and/or `Height` of the `SKCanvasElement`.

`SKCanvasElement` also comes with a `MirroredWhenRightToLeftProperty`. If `true`, the drawing will be reflected horizontally when the `FlowDirection` of the `SKCanvasElement` is right-to-left. By default, this property is set to `false`, meaning that the drawing will be the same regardless of the `FlowDirection`.
## Full example

# Full example
To see this in action, here's a complete sample that uses `SKCanvasElement` to draw 1 of 3 different drawings based on the value of a [Slider](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.controls.slider). Note how you have to be careful with surrounding all the Skia-related logic in platform-specific guards. This is the case for both the [XAML](platform-specific-xaml) and the [code-behind](platform-specific-csharp).

To see this in action, here's a complete sample that uses `SKCanvasElement` to draw 1 of 3 different drawings based on the value of a [Slider](https://learn.microsoft.com/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.controls.slider). Note how you have to be careful with surrounding all the Skia-related logic in platform-specific guards. This is the case for both [xaml](platform-specific-xaml) and the [code-behind](platform-specific-csharp).
XAML:

Xaml:
```xaml
<!-- SKCanvasElementExample.xaml -->
<UserControl x:Class="BlankApp.SKCanvasElementExample"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:BlankApp"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:skia="http://uno.ui/skia"
xmlns:not_skia="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
mc:Ignorable="d"
d:DesignHeight="300"
d:DesignWidth="400">

<skia:Grid>
<Grid.RowDefinitions>
<RowDefinition Height="Auto" />
<RowDefinition Height="Auto" />
<RowDefinition Height="*" />
</Grid.RowDefinitions>
<Slider Grid.Row="1" x:Name="slider" Header="Sample" Minimum="0" Maximum="{x:Bind MaxSampleIndex}" />
<local:SKCanvasElementImpl Grid.Row="2" Sample="{x:Bind slider.Value, Mode=OneWay}" />
</skia:Grid>
<not_skia:TextBlock Text="This sample is only supported on skia." />
</UserControl>
```

Code-behind:

```csharp
// SKCanvasElementExample.xaml.cs
public partial class SKCanvasElementExample : UserControl
{
public int MaxSampleIndex => SKCanvasElementImpl.SampleCount - 1;

public SKCanvasElement_Simple()
{
this.InitializeComponent();
}
}
```

ramezgerges marked this conversation as resolved.
Show resolved Hide resolved
```csharp
// SKCanvasElementImpl.skia.cs <-- NOTICE the `.skia`
public class SKCanvasElementImpl : SKCanvasElement
{
public static int SampleCount => 3;

public static DependencyProperty SampleProperty { get; } = DependencyProperty.Register(
nameof(Sample),
typeof(int),
typeof(SKCanvasElementImpl),
new PropertyMetadata(0, (o, args) => ((SKCanvasElementImpl)o).SampleChanged((int)args.NewValue)));

public int Sample
{
get => (int)GetValue(SampleProperty);
set => SetValue(SampleProperty, value);
}

private void SampleChanged(int newIndex)
{
Sample = Math.Min(Math.Max(0, newIndex), SampleCount - 1);
}

protected override void RenderOverride(SKCanvas canvas, Size area)
{
var minDim = Math.Min(area.Width, area.Height);
// rescale to fit the given area, assuming each drawing takes is 260x260
canvas.Scale((float)(minDim / 260), (float)(minDim / 260));

switch (Sample)
{
case 0:
SkiaDrawing0(canvas);
break;
case 1:
SkiaDrawing1(canvas);
break;
case 2:
SkiaDrawing2(canvas);
break;
}
}

// https://fiddle.skia.org/c/@shapes
private void SkiaDrawing0(SKCanvas canvas)
{
canvas.DrawColor(SKColors.White);

var paint = new SKPaint();
paint.Style = SKPaintStyle.Fill;
paint.IsAntialias = true;
paint.StrokeWidth = 4;
paint.Color = new SKColor(0xff4285F4);

var rect = SKRect.Create(10, 10, 100, 160);
canvas.DrawRect(rect, paint);

var oval = new SKPath();
oval.AddRoundRect(rect, 20, 20);
oval.Offset(new SKPoint(40, 80));
paint.Color = new SKColor(0xffDB4437);
canvas.DrawPath(oval, paint);

paint.Color = new SKColor(0xff0F9D58);
canvas.DrawCircle(180, 50, 25, paint);

rect.Offset(80, 50);
paint.Color = new SKColor(0xffF4B400);
paint.Style = SKPaintStyle.Stroke;
canvas.DrawRoundRect(rect, 10, 10, paint);
}

// https://fiddle.skia.org/c/@bezier_curves
private void SkiaDrawing1(SKCanvas canvas)
{
canvas.DrawColor(SKColors.White);

var paint = new SKPaint();
paint.Style = SKPaintStyle.Stroke;
paint.StrokeWidth = 8;
paint.Color = new SKColor(0xff4285F4);
paint.IsAntialias = true;
paint.StrokeCap = SKStrokeCap.Round;

var path = new SKPath();
path.MoveTo(10, 10);
path.QuadTo(256, 64, 128, 128);
path.QuadTo(10, 192, 250, 250);
canvas.DrawPath(path, paint);
}

// https://fiddle.skia.org/c/@shader
private void SkiaDrawing2(SKCanvas canvas)
{
var paint = new SKPaint();
using var pathEffect = SKPathEffect.CreateDiscrete(10.0f, 4.0f);
paint.PathEffect = pathEffect;
SKPoint[] points =
{
new SKPoint(0.0f, 0.0f),
new SKPoint(256.0f, 256.0f)
};
SKColor[] colors =
{
new SKColor(66, 133, 244),
new SKColor(15, 157, 88)
};
paint.Shader = SKShader.CreateLinearGradient(points[0], points[1], colors, SKShaderTileMode.Clamp);
paint.IsAntialias = true;
canvas.Clear(SKColors.White);
var path = Star();
canvas.DrawPath(path, paint);

SKPath Star()
{
const float R = 60.0f, C = 128.0f;
var path = new SKPath();
path.MoveTo(C + R, C);
for (var i = 1; i < 15; ++i)
{
var a = 0.44879895f * i;
var r = R + R * (i % 2);
path.LineTo((float)(C + r * Math.Cos(a)), (float)(C + r * Math.Sin(a)));
}
return path;
}
}
}
```
8 changes: 5 additions & 3 deletions src/SamplesApp/UITests.Shared/UITests.Shared.projitems
Original file line number Diff line number Diff line change
Expand Up @@ -4638,7 +4638,7 @@
<SubType>Designer</SubType>
<Generator>MSBuild:Compile</Generator>
</Page>
<Page Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\SkiaVisualShowcase.xaml">
<Page Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\SKCanvasElement_Simple.xaml">
<SubType>Designer</SubType>
<Generator>MSBuild:Compile</Generator>
</Page>
Expand Down Expand Up @@ -5964,6 +5964,8 @@
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI.Xaml_Automation\AutomationProperties_Name.xaml.cs">
<DependentUpon>AutomationProperties_Name.xaml</DependentUpon>
</Compile>
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\GlCanvasElementImpl.skia.cs" />
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\SKCanvasElementImpl.skia.cs" />
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Core\CloseRequestedTests.xaml.cs">
<DependentUpon>CloseRequestedTests.xaml</DependentUpon>
</Compile>
Expand Down Expand Up @@ -8192,8 +8194,8 @@
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\VisualSurfaceTests.xaml.cs">
<DependentUpon>VisualSurfaceTests.xaml</DependentUpon>
</Compile>
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\SkiaVisualShowcase.xaml.cs">
<DependentUpon>SkiaVisualShowcase.xaml</DependentUpon>
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\SKCanvasElement_Simple.xaml.cs">
<DependentUpon>SKCanvasElement_Simple.xaml</DependentUpon>
</Compile>
<Compile Include="$(MSBuildThisFileDirectory)Windows_UI_Composition\GLCanvasElement_Simple.xaml.cs">
<DependentUpon>GLCanvasElement_Simple.xaml</DependentUpon>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,12 @@
xmlns:local="using:UITests.Shared.Windows_UI_Composition"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:skia="http://uno.ui/skia"
xmlns:skia="http://uno.ui/skia#using:UITests.Shared.Windows_UI_Composition"
xmlns:not_skia="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
mc:Ignorable="d"
mc:Ignorable="d skia"
d:DesignHeight="300"
d:DesignWidth="400">

<skia:Border>
<local:GlCanvasElementImpl />
</skia:Border>
<skia:GlCanvasElementImpl />
<not_skia:TextBlock Text="This sample is only supported on skia." />
</UserControl>
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
using System;
using System.Drawing;
using Uno.UI.Samples.Controls;
using Microsoft.UI.Xaml.Controls;
using Size = Windows.Foundation.Size;

#if __SKIA__
using Silk.NET.OpenGL;
#endif

namespace UITests.Shared.Windows_UI_Composition
{
Expand All @@ -18,111 +11,4 @@ public GLCanvasElement_Simple()
this.InitializeComponent();
}
}

#if __SKIA__
public class GlCanvasElementImpl() : GLCanvasElement(new Size(800, 600))
#else
public class GlCanvasElementImpl : FrameworkElement
#endif
{
#if __SKIA__
private int _counter;
private uint _vao;
private uint _vbo;
private uint _program;

unsafe protected override void Init(GL gl)
{
_vao = gl.GenVertexArray();
gl.BindVertexArray(_vao);

float[] vertices =
{
0.5f, -0.5f, 0.0f, // bottom right
-0.5f, -0.5f, 0.0f, // bottom left
0.0f, 0.5f, 0.0f // top
};

_vbo = gl.GenBuffer();
gl.BindBuffer(BufferTargetARB.ArrayBuffer, _vbo);
gl.BufferData(BufferTargetARB.ArrayBuffer, new ReadOnlySpan<float>(vertices), BufferUsageARB.StaticDraw);

gl.VertexAttribPointer(0, 3, GLEnum.Float, false, 3 * sizeof(float), (void*)0);
gl.EnableVertexAttribArray(0);

const string vertexCode = @"
#version 330 core

layout (location = 0) in vec3 aPosition;
out vec4 vertexColor;

void main()
{
gl_Position = vec4(aPosition, 1.0);
vertexColor = vec4(aPosition.x + 0.5, aPosition.y + 0.5, aPosition.z + 0.5, 1.0);
}";

const string fragmentCode = @"
#version 330 core

out vec4 out_color;
in vec4 vertexColor;

void main()
{
out_color = vertexColor;
}";

uint vertexShader = gl.CreateShader(ShaderType.VertexShader);
gl.ShaderSource(vertexShader, vertexCode);
gl.CompileShader(vertexShader);

gl.GetShader(vertexShader, ShaderParameterName.CompileStatus, out int vStatus);
if (vStatus != (int) GLEnum.True)
throw new Exception("Vertex shader failed to compile: " + gl.GetShaderInfoLog(vertexShader));

uint fragmentShader = gl.CreateShader(ShaderType.FragmentShader);
gl.ShaderSource(fragmentShader, fragmentCode);
gl.CompileShader(fragmentShader);

gl.GetShader(fragmentShader, ShaderParameterName.CompileStatus, out int fStatus);
if (fStatus != (int) GLEnum.True)
throw new Exception("Fragment shader failed to compile: " + gl.GetShaderInfoLog(fragmentShader));

_program = gl.CreateProgram();
gl.AttachShader(_program, vertexShader);
gl.AttachShader(_program, fragmentShader);
gl.LinkProgram(_program);

gl.GetProgram(_program, ProgramPropertyARB.LinkStatus, out int lStatus);
if (lStatus != (int) GLEnum.True)
throw new Exception("Program failed to link: " + gl.GetProgramInfoLog(_program));

gl.DetachShader(_program, vertexShader);
gl.DetachShader(_program, fragmentShader);
gl.DeleteShader(vertexShader);
gl.DeleteShader(fragmentShader);
}

protected override void OnDestroy(GL gl)
{
gl.DeleteVertexArray(_vao);
gl.DeleteBuffer(_vbo);
gl.DeleteProgram(_program);
}

protected override void RenderOverride(GL gl)
{
gl.ClearColor(Color.FromArgb(_counter++ % 255, 0, 0));
gl.Clear(ClearBufferMask.ColorBufferBit);

gl.UseProgram(_program);

gl.BindVertexArray(_vao);
gl.DrawArrays(PrimitiveType.Triangles, 0, 3);

Invalidate(); // continuous redrawing
}
#endif
}
}
Loading
Loading