Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skwasm scene #40330

Merged
merged 46 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
cfccaa7
WIP add screenshot goldens to ui_tests
harryterkelsen Feb 15, 2023
294cff9
Started implementing skwasm scene stuff.
eyebrowsoffire Mar 13, 2023
6dc5144
Scene builder mostly coded up.
eyebrowsoffire Mar 14, 2023
13816e4
Stuff.
eyebrowsoffire Mar 15, 2023
ab7e937
Running into TFA issues.
eyebrowsoffire Mar 15, 2023
6184bec
More scene changes.
eyebrowsoffire Mar 21, 2023
fd915b3
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 21, 2023
aade26b
We drawing now.
eyebrowsoffire Mar 22, 2023
d49f73d
Copy stuff, but skwasm.worker.js doesn't copy yet.
eyebrowsoffire Mar 22, 2023
c473192
Add a few flags.
eyebrowsoffire Mar 23, 2023
bbd206b
Bundle skwasm.
eyebrowsoffire Mar 23, 2023
b651d65
Use newer buildroot.
eyebrowsoffire Mar 23, 2023
6fbb8d0
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 23, 2023
4796b64
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 29, 2023
d79c143
Use the normal toolchain so we get LTO.
eyebrowsoffire Mar 29, 2023
8350e48
Merge remote-tracking branch 'hterkelsen/ui-golden-tests' into skwasm…
eyebrowsoffire Mar 29, 2023
d8bfb0f
Analyzer cleanup and Kevin's comments.
eyebrowsoffire Mar 29, 2023
f7a22db
Update license golden.
eyebrowsoffire Mar 29, 2023
ad64278
Some fixes for goldens, although skwasm renderer has async issues.
eyebrowsoffire Mar 29, 2023
17a8e6c
Some async stuff.
eyebrowsoffire Mar 29, 2023
c89584c
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 29, 2023
d446202
Fixed cull rect calculation.
eyebrowsoffire Mar 30, 2023
4ca5636
Update licenses and fix an analyzer issue.
eyebrowsoffire Mar 30, 2023
4f85df1
Wrote unit tests that cover the remainder of what skwasm has implemen…
eyebrowsoffire Mar 31, 2023
a722c25
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 31, 2023
b4f98fc
Update README
eyebrowsoffire Mar 31, 2023
a5d798d
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 31, 2023
1cb3514
Wait one frame after rendering to ensure the screenshot captures prop…
eyebrowsoffire Mar 31, 2023
6cb0f12
Use switch expression.
eyebrowsoffire Mar 31, 2023
32299b2
Merge branch 'main' into skwasm_scene
eyebrowsoffire Mar 31, 2023
c2ca50c
Tweak build parameters for skwasm.
eyebrowsoffire Mar 31, 2023
64d5728
Update lib/web_ui/lib/src/engine/skwasm/skwasm_impl/surface.dart
eyebrowsoffire Apr 4, 2023
a239266
Address some comments from Mouad.
eyebrowsoffire Apr 4, 2023
3d90da0
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 4, 2023
da3be30
Fix formatting.
eyebrowsoffire Apr 4, 2023
e0892d6
Change name of variable for clarity.
eyebrowsoffire Apr 4, 2023
07f6f75
Fix async semantics of skwasm surface.
eyebrowsoffire Apr 4, 2023
0d94512
Reinstate old transformRect algorithm.
eyebrowsoffire Apr 4, 2023
cd56076
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 4, 2023
85674bc
Shift cullRect by offset when using opacity layer.
eyebrowsoffire Apr 5, 2023
457059f
Renderer.renderScene should return FutureOr.
eyebrowsoffire Apr 5, 2023
a0dbca2
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 5, 2023
a6f8d1e
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 6, 2023
26a54c0
Get rid of duplicate `skwasm_group` definitino from merge.
eyebrowsoffire Apr 6, 2023
6a41070
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 6, 2023
2d52b90
Merge branch 'main' into skwasm_scene
eyebrowsoffire Apr 10, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions ci/licenses_golden/licenses_flutter
Original file line number Diff line number Diff line change
Expand Up @@ -1980,7 +1980,9 @@ ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/services/serialization.dart +
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/shadow.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/canvas.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/font_collection.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/image.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/layers.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/paint.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/paragraph.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/path.dart + ../../../flutter/LICENSE
Expand All @@ -1995,6 +1997,7 @@ ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_pa
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_picture.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_surface.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/renderer.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/scene_builder.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/surface.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/vertices.dart + ../../../flutter/LICENSE
ORIGIN: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_stub.dart + ../../../flutter/LICENSE
Expand Down Expand Up @@ -4551,7 +4554,9 @@ FILE: ../../../flutter/lib/web_ui/lib/src/engine/services/serialization.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/shadow.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/canvas.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/font_collection.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/image.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/layers.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/paint.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/paragraph.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/path.dart
Expand All @@ -4566,6 +4571,7 @@ FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_path
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_picture.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/raw/raw_surface.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/renderer.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/scene_builder.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/surface.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_impl/vertices.dart
FILE: ../../../flutter/lib/web_ui/lib/src/engine/skwasm/skwasm_stub.dart
Expand Down
32 changes: 8 additions & 24 deletions lib/web_ui/dev/build.dart
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,6 @@ import 'environment.dart';
import 'pipeline.dart';
import 'utils.dart';

enum RuntimeMode {
profile,
release,
}

const Map<String, String> targetAliases = <String, String>{
'sdk': 'flutter/web_sdk',
'web_sdk': 'flutter/web_sdk',
Expand Down Expand Up @@ -45,6 +40,12 @@ class BuildCommand extends Command<bool> with ArgUtils<bool> {
'output will be located at "out/wasm_profile".\nThis only applies to '
'the wasm build. The host build is always built in release mode.',
);
argParser.addFlag(
kevmoo marked this conversation as resolved.
Show resolved Hide resolved
'debug',
help: 'Build in debug mode instead of release mode. In this mode, the '
'output will be located at "out/wasm_debug".\nThis only applies to '
'the wasm build. The host build is always built in release mode.',
);
}

@override
Expand All @@ -57,9 +58,6 @@ class BuildCommand extends Command<bool> with ArgUtils<bool> {

bool get host => boolArg('host');

RuntimeMode get runtimeMode =>
boolArg('profile') ? RuntimeMode.profile : RuntimeMode.release;

eyebrowsoffire marked this conversation as resolved.
Show resolved Hide resolved
List<String> get targets => argResults?.rest ?? <String>[];

@override
Expand Down Expand Up @@ -112,15 +110,6 @@ class GnPipelineStep extends ProcessStep {
@override
bool get isSafeToInterrupt => false;

String get runtimeModeFlag {
switch (runtimeMode) {
case RuntimeMode.profile:
return 'profile';
case RuntimeMode.release:
return 'release';
}
}

List<String> get _gnArgs {
if (host) {
return <String>[
Expand All @@ -130,7 +119,7 @@ class GnPipelineStep extends ProcessStep {
} else {
return <String>[
'--web',
'--runtime-mode=$runtimeModeFlag',
'--runtime-mode=${runtimeMode.name}',
];
}
}
Expand Down Expand Up @@ -169,12 +158,7 @@ class NinjaPipelineStep extends ProcessStep {
if (host) {
return environment.hostDebugUnoptDir.path;
}
switch (runtimeMode) {
case RuntimeMode.profile:
return environment.wasmProfileOutDir.path;
case RuntimeMode.release:
return environment.wasmReleaseOutDir.path;
}
return getBuildDirectoryForRuntimeMode(runtimeMode).path;
}

@override
Expand Down
7 changes: 7 additions & 0 deletions lib/web_ui/dev/environment.dart
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ class Environment {
io.Directory(pathlib.join(outDir.path, 'wasm_release'));
final io.Directory wasmProfileOutDir =
io.Directory(pathlib.join(outDir.path, 'wasm_profile'));
final io.Directory wasmDebugOutDir =
io.Directory(pathlib.join(outDir.path, 'wasm_debug'));
final io.Directory hostDebugUnoptDir =
io.Directory(pathlib.join(outDir.path, 'host_debug_unopt'));
final io.Directory dartSdkDir = dartExecutable.parent.parent;
Expand All @@ -57,6 +59,7 @@ class Environment {
outDir: outDir,
wasmReleaseOutDir: wasmReleaseOutDir,
wasmProfileOutDir: wasmProfileOutDir,
wasmDebugOutDir: wasmDebugOutDir,
hostDebugUnoptDir: hostDebugUnoptDir,
dartSdkDir: dartSdkDir,
);
Expand All @@ -71,6 +74,7 @@ class Environment {
required this.outDir,
required this.wasmReleaseOutDir,
required this.wasmProfileOutDir,
required this.wasmDebugOutDir,
required this.hostDebugUnoptDir,
required this.dartSdkDir,
});
Expand Down Expand Up @@ -103,6 +107,9 @@ class Environment {
/// The output directory for the wasm_profile build.
final io.Directory wasmProfileOutDir;

/// The output directory for the wasm_debug build.
final io.Directory wasmDebugOutDir;

/// The output directory for the host_debug_unopt build.
final io.Directory hostDebugUnoptDir;

Expand Down
8 changes: 3 additions & 5 deletions lib/web_ui/dev/steps/copy_artifacts_step.dart
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ import '../pipeline.dart';
import '../utils.dart';

class CopyArtifactsStep implements PipelineStep {
CopyArtifactsStep(this.artifactDeps, { required this.isProfile });
CopyArtifactsStep(this.artifactDeps, { required this.runtimeMode });

final ArtifactDependencies artifactDeps;
final bool isProfile;
final RuntimeMode runtimeMode;

@override
String get description => 'copy_artifacts';
Expand Down Expand Up @@ -167,9 +167,7 @@ class CopyArtifactsStep implements PipelineStep {
}
}

String get outBuildPath => isProfile
? environment.wasmProfileOutDir.path
: environment.wasmReleaseOutDir.path;
String get outBuildPath => getBuildDirectoryForRuntimeMode(runtimeMode).path;

Future<void> copySkwasm() async {
final io.Directory targetDir = io.Directory(pathlib.join(
Expand Down
8 changes: 4 additions & 4 deletions lib/web_ui/dev/steps/run_suite_step.dart
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ import '../utils.dart';
/// them from another bot.
class RunSuiteStep implements PipelineStep {
RunSuiteStep(this.suite, {
required this.isDebug,
required this.isDebugBrowser,
required this.isVerbose,
required this.doUpdateScreenshotGoldens,
required this.requireSkiaGold,
Expand All @@ -39,7 +39,7 @@ class RunSuiteStep implements PipelineStep {

final TestSuite suite;
final Set<FilePath>? testFiles;
final bool isDebug;
final bool isDebugBrowser;
final bool isVerbose;
final bool doUpdateScreenshotGoldens;
final String? overridePathToCanvasKit;
Expand Down Expand Up @@ -76,7 +76,7 @@ class RunSuiteStep implements PipelineStep {
...<String>['-r', 'compact'],
// Disable concurrency. Running with concurrency proved to be flaky.
'--concurrency=1',
if (isDebug) '--pause-after-load',
if (isDebugBrowser) '--pause-after-load',
'--platform=${browserEnvironment.packageTestRuntime.identifier}',
'--precompiled=$bundleBuildPath',
'--configuration=$configurationFilePath',
Expand Down Expand Up @@ -167,7 +167,7 @@ class RunSuiteStep implements PipelineStep {
final Renderer renderer = suite.testBundle.compileConfig.renderer;
final CanvasKitVariant? variant = suite.runConfig.variant;
final SkiaGoldClient skiaClient = SkiaGoldClient(
environment.webUiSkiaGoldDirectory,
getSkiaGoldDirectoryForSuite(suite),
dimensions: <String, String> {
'Browser': suite.runConfig.browser.name,
if (isWasm) 'Wasm': 'true',
Expand Down
1 change: 1 addition & 0 deletions lib/web_ui/dev/test_dart2wasm.js
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ window.onload = async function () {
document.body.appendChild(skwasmScript);
skwasmScript.addEventListener('load', async () => {
const skwasmInstance = await skwasm();
window._flutter_skwasmInstance = skwasmInstance;
resolve({
"skwasm": skwasmInstance.asm,
"ffi": {
Expand Down
1 change: 1 addition & 0 deletions lib/web_ui/dev/test_platform.dart
Original file line number Diff line number Diff line change
Expand Up @@ -429,6 +429,7 @@ class BrowserPlatform extends PlatformPlugin {
screenshot,
doUpdateScreenshotGoldens,
filename,
getSkiaGoldDirectoryForSuite(suite),
skiaClient,
isCanvaskitTest: isCanvaskitTest,
verbose: isVerbose,
Expand Down
13 changes: 9 additions & 4 deletions lib/web_ui/dev/test_runner.dart
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
TestCommand() {
argParser
..addFlag(
'debug',
'debug-browser',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flutter tool calls this --start-paused which I think is a more self-explanatory name and keeps names consistent across repos/tools.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that is probably a better name. One detail is that this option also causes us to run in a window instead of headless, but that probably is fine to just leave implied.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that's how it works in the flutter tool too.

help: 'Pauses the browser before running a test, giving you an '
'opportunity to add breakpoints or inspect loaded code before '
'running the code.',
Expand Down Expand Up @@ -80,6 +80,11 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
help:
'Use artifacts from the profile build instead of release.'
)
..addFlag(
'debug',
help:
'Use artifacts from the debug build instead of release.'
)
..addFlag(
'require-skia-gold',
help:
Expand Down Expand Up @@ -152,7 +157,7 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
///
/// In this mode the browser pauses before running the test to allow
/// you set breakpoints or inspect the code.
bool get isDebug => boolArg('debug');
bool get isDebugBrowser => boolArg('debug-browser');

bool get isVerbose => boolArg('verbose');

Expand Down Expand Up @@ -364,7 +369,7 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
final Set<FilePath>? testFiles = targetFiles.isEmpty ? null : Set<FilePath>.from(targetFiles);
final Pipeline testPipeline = Pipeline(steps: <PipelineStep>[
if (isWatchMode) ClearTerminalScreenStep(),
if (shouldCopyArtifacts) CopyArtifactsStep(artifacts, isProfile: boolArg('profile')),
if (shouldCopyArtifacts) CopyArtifactsStep(artifacts, runtimeMode: runtimeMode),
if (shouldCompile)
for (final TestBundle bundle in bundles)
CompileBundleStep(
Expand All @@ -376,7 +381,7 @@ class TestCommand extends Command<bool> with ArgUtils<bool> {
for (final TestSuite suite in filteredSuites)
RunSuiteStep(
suite,
isDebug: isDebug,
isDebugBrowser: isDebugBrowser,
isVerbose: isVerbose,
doUpdateScreenshotGoldens: doUpdateScreenshotGoldens,
requireSkiaGold: requireSkiaGold,
Expand Down
41 changes: 41 additions & 0 deletions lib/web_ui/dev/utils.dart
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ import 'environment.dart';
import 'exceptions.dart';
import 'felt_config.dart';

enum RuntimeMode {
debug,
profile,
release,
}

class FilePath {
FilePath.fromCwd(String relativePath)
: _absolutePath = path.absolute(relativePath);
Expand Down Expand Up @@ -328,6 +334,32 @@ mixin ArgUtils<T> on Command<T> {

/// Extracts a string argument from [argResults].
String stringArg(String name) => argResults![name] as String;

RuntimeMode get runtimeMode {
final bool isProfile = boolArg('profile');
final bool isDebug = boolArg('debug');
if (isProfile && isDebug) {
throw ToolExit('Cannot specify both --profile and --debug at the same time.');
}
if (isProfile) {
return RuntimeMode.profile;
} else if (isDebug) {
return RuntimeMode.debug;
} else {
return RuntimeMode.release;
}
}
}

io.Directory getBuildDirectoryForRuntimeMode(RuntimeMode runtimeMode) {
switch (runtimeMode) {
kevmoo marked this conversation as resolved.
Show resolved Hide resolved
case RuntimeMode.debug:
return environment.wasmDebugOutDir;
case RuntimeMode.profile:
return environment.wasmProfileOutDir;
case RuntimeMode.release:
return environment.wasmReleaseOutDir;
}
}

/// There might be proccesses started during the tests.
Expand Down Expand Up @@ -390,6 +422,15 @@ io.Directory getBundleBuildDirectory(TestBundle bundle) {
);
}

io.Directory getSkiaGoldDirectoryForSuite(TestSuite suite) {
return io.Directory(
path.join(
environment.webUiSkiaGoldDirectory.path,
suite.name,
)
);
}

extension AnsiColors on String {
static bool shouldEscape = io.stdout.hasTerminal && io.stdout.supportsAnsiEscapes;

Expand Down
2 changes: 1 addition & 1 deletion lib/web_ui/lib/src/engine/canvaskit/renderer.dart
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,7 @@ class CanvasKitRenderer implements Renderer {
CkParagraphBuilder(style);

@override
void renderScene(ui.Scene scene) {
Future<void> renderScene(ui.Scene scene) async {
// "Build finish" and "raster start" happen back-to-back because we
// render on the same thread, so there's no overhead from hopping to
// another thread.
Expand Down
2 changes: 1 addition & 1 deletion lib/web_ui/lib/src/engine/html/renderer.dart
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ class HtmlRenderer implements Renderer {
CanvasParagraphBuilder(style as EngineParagraphStyle);

@override
void renderScene(ui.Scene scene) {
Future<void> renderScene(ui.Scene scene) async {
_viewEmbedder.addSceneToSceneHost((scene as SurfaceScene).webOnlyRootElement);
frameTimingsOnRasterFinish();
}
Expand Down
3 changes: 1 addition & 2 deletions lib/web_ui/lib/src/engine/html/scene_builder.dart
Original file line number Diff line number Diff line change
Expand Up @@ -106,8 +106,7 @@ class SurfaceSceneBuilder implements ui.SceneBuilder {
throw ArgumentError('"matrix4" must have 16 entries.');
}

// TODO(yjbanov): make this final after NNBD ships definite assignment.
/*final*/ Float32List? matrix;
final Float32List matrix;
if (_surfaceStack.length == 1) {
// Top level transform contains view configuration to scale
// scene to devicepixelratio. Use identity instead since CSS uses
Expand Down
2 changes: 1 addition & 1 deletion lib/web_ui/lib/src/engine/renderer.dart
Original file line number Diff line number Diff line change
Expand Up @@ -213,5 +213,5 @@ abstract class Renderer {

ui.ParagraphBuilder createParagraphBuilder(ui.ParagraphStyle style);

void renderScene(ui.Scene scene);
Future<void> renderScene(ui.Scene scene);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is being called from a synchronous method:

void render(ui.Scene scene, [ui.FlutterView? view]) {
renderer.renderScene(scene);
}

I don't think we should change the signature of Renderer.renderScene. Since skwasm is the only one that needs the asynchronicity, let's only change SkwasmRenderer.renderScene's signature? And I would argue that even SkwasmRenderer.renderScene doesn't need to be asynchronous. Nothing is awaiting it, so effectively, we are firing the future and forgetting about it.

One concern that I have here though is race conditions. E.g. what if the previous renderScene call takes a while and we receive a new one? Should we somehow force these calls to go in sequence (by pushing them into a queue for example)? Again, if this is something we plan to handle later, let's add a TODO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is fine for this to be called and not awaited from a synchronous method in a "fire and forget" manner, and that's what is still going on right now. Making this method async at the renderer level only makes things a bit more expressive and flexible. The existing renderers which do not asynchronously render can return an immediately resolved future.

The main impetus behind doing this is because the new golden screenshot capabilities in the golden tests need to wait for the rendering to be actually complete before attempting a screenshot. So really, this future is only being awaited in tests.

As for your concern in the last paragraph, I have a lot of the same thoughts. That's sort of the reason why I'd like to expose this as an async method. In situations where the render thread is slower than our target FPS, we may want to keep track of pending futures and stop actually submitting new frames if it starts getting backed up. Otherwise, if we continue pushing more frames, the render thread will get continually more backed up, when it's better to to just drop frames. We can't do this with our current "fire-and-forget" method, but making the method async opens us up to doing this in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with Mouad, this should be synchronous for consistency with the mobile Flutter engine and our public facing API. The Skwasm renderer can have a way to await for the scene to actually be rendered which can be used in tests. As for the problem with the render thread getting backed up, this is already something that can happen in Flutter mobile and I think they just drop everything but the latest frame.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a suggested alternative? This Renderer.renderScene API is web-specific, so there is nothing in the mobile Flutter engine to keep it consistent with, and I'm also not changing the public facing API (PlatformDispatcher.render is still a synchronous method). Basically the reason I want to return a Future here is:

  • To allow us to wait for the render to complete in the tests.
  • If we do want to "drop everything but the latest frame" like the native engine does, the UI thread still needs a way to tell whether we are actually getting backed up.

A Future returned directly from the method seems like the most natural way to achieve this, but if there is something different you all have in mind I'm open to suggestions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I could have Renderer.renderScene return FutureOr<void>, so that the subclasses of Renderer that aren't asynchronous can still just return void?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should've clarified my motivation for keeping things as sync as possible. It's because I want future maintainers to be aware that this is used as a sync API and it's better to stay that way. And if they really need the asynchronicity, then they can make the deliberate decision of converting it to async.

If we make all of them async today, I'm afraid it becomes too easy in the future to add async stuff in there without thinking about the consequences. I want to conversion to async to be a deliberate decision based on a real need (like in the case of skwasm).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the FutureOr<void> change not address your issue? SkwasmRenderer's render will return Future<void>, but the other renderers still return void. So it would require us to explicitly change a renderer's return value to make it async

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't address the issue. The developer making change to renderScene will think it's okay to add an await in there because the signature already "supports" async. I want them to have the extra burden of changing the signature, which will make them look around and check whether it's fine to convert it to async or not.

That said, I don't feel strongly about it, and maybe this is all just me doing premature design optimizations :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, if a developer is editing the renderScene function on one of the existing renderers (whose renderScene method is synchronous), and they want to add an await, they will have the extra burden of changing the signature, since the signature of the existing renderers is staying void renderScene(...). They would have to explicitly change it to Future<void> renderScene(...). The abstract Renderer class has the signature FutureOr<void> renderScene(...), but the actual concrete renderer classes themselves either explicitly return void or explicitly return Future<void>

Copy link
Contributor

@mdebbar mdebbar Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh okay got it now. I misunderstood your previous comment, thought you were making subclasses FutureOr<void> too (should've looked more closely at the code 🤦‍♂️)

Ok, this actually addresses my concerns, thanks!

}
7 changes: 7 additions & 0 deletions lib/web_ui/lib/src/engine/skwasm/skwasm_impl.dart
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,23 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

@DefaultAsset('skwasm')
// The web_sdk/sdk_rewriter.dart uses this directive.
// ignore: unnecessary_library_directive
library skwasm_impl;

import 'dart:ffi';

export 'skwasm_impl/canvas.dart';
export 'skwasm_impl/font_collection.dart';
export 'skwasm_impl/image.dart';
export 'skwasm_impl/layers.dart';
export 'skwasm_impl/paint.dart';
export 'skwasm_impl/paragraph.dart';
export 'skwasm_impl/path.dart';
export 'skwasm_impl/path_metrics.dart';
export 'skwasm_impl/picture.dart';
export 'skwasm_impl/raw/js_functions.dart';
export 'skwasm_impl/raw/raw_canvas.dart';
export 'skwasm_impl/raw/raw_geometry.dart';
export 'skwasm_impl/raw/raw_memory.dart';
Expand All @@ -22,5 +28,6 @@ export 'skwasm_impl/raw/raw_path_metrics.dart';
export 'skwasm_impl/raw/raw_picture.dart';
export 'skwasm_impl/raw/raw_surface.dart';
export 'skwasm_impl/renderer.dart';
export 'skwasm_impl/scene_builder.dart';
export 'skwasm_impl/surface.dart';
export 'skwasm_impl/vertices.dart';