Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some missing ops to the SPIR-V Transpiler #26336

Closed
wants to merge 27 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
26460e5
Add dart library for transpiling SPIR-V
chriscraws Apr 7, 2021
b4fdec5
Address comments from @clockwork.
chriscraws Apr 7, 2021
1262278
update readme
chriscraws Apr 7, 2021
95319c5
add exception tests to spir-v transpiler
chriscraws Apr 20, 2021
23b828d
Merge remote-tracking branch 'flutter/master' into spirv-dart
chriscraws Apr 20, 2021
93a1412
Add some ops to the transpiler
clocksmith May 15, 2021
b8c139c
Add some ops to the transpiler
clocksmith May 15, 2021
09b4d66
Add some ops to the transpiler
clocksmith May 15, 2021
fccc268
Add some ops to the transpiler
clocksmith May 15, 2021
ba5ac64
Add some ops to the transpiler
clocksmith May 15, 2021
b28cfdc
Add some ops to the transpiler
clocksmith May 15, 2021
d823233
Add some ops to the transpiler
clocksmith May 15, 2021
97afbe0
Delete fragment_shader.cc
clocksmith May 15, 2021
8db822c
Typos
clocksmith May 15, 2021
fe2869f
Typos
clocksmith May 15, 2021
78acab6
Add dart library for transpiling SPIR-V
chriscraws Apr 7, 2021
49b879b
Address comments from @clockwork.
chriscraws Apr 7, 2021
3dab21f
update readme
chriscraws Apr 7, 2021
c7b1729
add exception tests to spir-v transpiler
chriscraws Apr 20, 2021
84a7c92
Merge branch 'spirv-dart' of github.com:chriscraws/engine into spirv-…
chriscraws May 21, 2021
b8ab5ba
Update review changes
chriscraws May 21, 2021
dee3b25
Add comment addressing opAccessChain
chriscraws May 21, 2021
a7b7791
address comments and nit: adjust some string concats
clocksmith May 21, 2021
f75cc23
Merge branch 'spirv-dart' of https://github.com/chriscraws/engine int…
clocksmith May 21, 2021
29cc86d
Address comments
clocksmith May 21, 2021
6639301
Address comments
clocksmith May 21, 2021
89db8d1
Address comments
clocksmith May 21, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ group("flutter") {
public_deps += [
"//flutter/flow:flow_unittests",
"//flutter/fml:fml_unittests",
"//flutter/lib/spirv/test/exception_shaders:spirv_compile_exception_shaders",
"//flutter/lib/ui:ui_unittests",
"//flutter/runtime:no_dart_plugin_registrant_unittests",
"//flutter/runtime:runtime_unittests",
Expand Down
6 changes: 6 additions & 0 deletions ci/analyze.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,12 @@ analyze \
--options "$FLUTTER_DIR/analysis_options.yaml" \
"$SRC_DIR/out/host_debug_unopt/gen/sky/bindings/dart_ui/ui.dart"

echo "Analyzing spirv library..."
analyze \
--packages="$FLUTTER_DIR/lib/spirv/.dart_tool/package_config.json" \
--options "$FLUTTER_DIR/analysis_options.yaml" \
"$FLUTTER_DIR/lib/spirv"

echo "Analyzing flutter_frontend_server..."
analyze \
--packages="$FLUTTER_DIR/flutter_frontend_server/.dart_tool/package_config.json" \
Expand Down
46 changes: 46 additions & 0 deletions lib/spirv/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# SPIR-V Transpiler

Note: This library is currently considered experimental until shader compilation is verified by engine unit tests, see the Testing section below for more details.

A dart library for transpiling a subset of SPIR-V to the shader languages used by Flutter internally.

- [SkSL](https://skia.org/docs/user/sksl/)
- [GLSL ES 100](https://www.khronos.org/files/opengles_shading_language.pdf)
- [GLSL ES 300](https://www.khronos.org/registry/OpenGL/specs/es/3.0/GLSL_ES_Specification_3.00.pdf)

All exported symbols are documented in `lib/spirv.dart`.

The supported subset of SPIR-V is specified in `lib/src/constants.dart`.

If you're using GLSL to generate SPIR-V with `glslangValidator` or `shaderc`,
the code will need to adhere to the following rules.

- There must be a single vec4 output at location 0.
- The output can only be written to from the main function.
- `gl_FragCoord` can only be read from the main function, and its z and w components
have no meaning.
- Control flow is prohibited aside from function calls and `return`.
`if`, `while`, `for`, `switch`, etc.
- No inputs from other shader stages.
- Only float, float-vector types, and square float-matrix types.
- Only square matrices are supported.
- Only built-in functions present in GLSL ES 100 are used.
- Debug symbols must be stripped, you can use the `spirv-opt` `--strip_debug` flag.

These rules may become less strict in future versions. Confirmant SPIR-V should succesfully transpile from the current version onwards. In other words, a spir-v shader you use now that meets these rules should keep working, but the output of the transpiler may change for that shader.

Support for textures, control flow, and structured types is planned, but not currently included.

## Testing

## Exception Tests

Exception tests are compiled to the `spirv_unittests` target. These tests rely on the `.spvasm` (SPIR-V Assembly) and `.glsl` files contained under `test/exception_shaders` in this directory. They are compiled to binary SPIR-V using `spirv-asm`, from the SwiftShader dependency, and each is verified by a test described in `test/spirv_exception_test.dart`. The purpose of these tests is to exercise every explicit failure path for shader transpilation. Each `glsl` or `spvasm` file should include a comment describing the failure that it is testing. The given files should be valid apart from the single failure case they are testing.

These tests can be run alone by executing `./spirv_unittests` in the build-output directory.

## Pixel Tests

Pixel test are not yet checked in, and should run as part of unit-testing for each implementation of `dart:ui`. These tests aim to validate the correctness of transpilation to each target language. Each shader should render the color green #00FF00 for a correct transpilation, and any other color for failure. They will be a combination of `.spvasm` files and more-readable GLSL files that are compiled to SPIR-V via `glslang`, provided by the SwiftShader dependency. Information for pixel tests will be expanded in a follow-up PR.

These tests will be able to be run alone by executing `./ui_unittests` in the build-output directory.
70 changes: 70 additions & 0 deletions lib/spirv/lib/spirv.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

/// This library defines a transpiler for converting SPIR-V into SkSL or GLSL.
// @dart = 2.12
library spirv;

import 'dart:convert';
import 'dart:math';
import 'dart:typed_data';

// These parts only contain private members, all public
// members are in this file (spirv.dart)
part 'src/constants.dart';
part 'src/transpiler.dart';
part 'src/types.dart';

/// The language to transpile to.
enum TargetLanguage {
/// SkSL, for Skia.
sksl,

/// GLSL ES 1.00, for WebGL 1.
glslES,

/// GLSL ES 3.00, for WebGL 2.
glslES300,
}

/// The result of a transpilation.
class TranspileResult {
/// Source code string in [language].
final String src;

/// The shader language in [src].
final TargetLanguage language;

/// The number of float uniforms used in this shader.
final int uniformFloatCount;

TranspileResult._(this.src, this.uniformFloatCount, this.language);
}

/// Thrown during transpilation due to malformed or unsupported SPIR-V.
class TranspileException implements Exception {
/// The SPIR-V operator last read, or zero if there was none.
final int op;

/// Human readable message explaining the exception.
final String message;

@override
String toString() => '$op: $message';

TranspileException._(this.op, this.message);
}

/// Transpile the provided SPIR-V buffer into a string of the [target] lang.
/// Throws an instance of [TranspileException] for malformed or unsupported
/// SPIR-V.
TranspileResult transpile(ByteBuffer spirv, TargetLanguage target) {
final _Transpiler t = _Transpiler(spirv.asUint32List(), target);
t.transpile();
return TranspileResult._(
t.src.toString(),
t.uniformFloatCount,
target,
);
}
132 changes: 132 additions & 0 deletions lib/spirv/lib/src/constants.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
// Copyright 2013 The Flutter Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// @dart = 2.12

part of spirv;

// This file contains a subset of SPIR-V constants defined at
// https://www.khronos.org/registry/spir-v/specs/unified1/SPIRV.html

// Header constants
const int _magicNumber = 0x07230203;

// Supported ExecutionModes
const int _originLowerLeft = 8;

// Supported memory models
const int _addressingModelLogical = 0;
const int _memoryModelGLSL450 = 1;

// Supported capabilities
const int _capabilityMatrix = 0;
const int _capabilityShader = 1;

// Supported storage classes
const int _storageClassUniformConstant = 0;
const int _storageClassInput = 1;
const int _storageClassOutput = 3;
const int _storageClassFunction = 7;

// Explicity supported decorations, others are ignored
const int _decorationBuiltIn = 11;
const int _decorationLocation = 30;

// Explicitly supported builtin types
const int _builtinFragCoord = 15;

// Ignored ops that have no semantic impact
const int _opSource = 3;
const int _opSourceExtension = 4;
const int _opName = 5;
const int _opMemberName = 6;
const int _opString = 7;
const int _opLine = 8;

// Supported instructions
const int _opExtInstImport = 11;
const int _opExtInst = 12;
const int _opMemoryModel = 14;
const int _opEntryPoint = 15;
const int _opExecutionMode = 16;
const int _opCapability = 17;
const int _opTypeVoid = 19;
const int _opTypeBool = 20;
const int _opTypeFloat = 22;
const int _opTypeVector = 23;
const int _opTypeMatrix = 24;
const int _opTypePointer = 32;
const int _opTypeFunction = 33;
const int _opConstantTrue = 41;
const int _opConstantFalse = 42;
const int _opConstant = 43;
const int _opConstantComposite = 44;
const int _opFunction = 54;
const int _opFunctionParameter = 55;
const int _opFunctionEnd = 56;
const int _opFunctionCall = 57;
const int _opVariable = 59;
const int _opLoad = 61;
const int _opStore = 62;
const int _opAccessChain = 65;
const int _opDecorate = 71;
const int _opVectorShuffle = 79;
const int _opCompositeConstruct = 80;
const int _opCompositeExtract = 81;
const int _opFNegate = 127;
const int _opFAdd = 129;
const int _opFSub = 131;
const int _opFMul = 133;
const int _opFDiv = 136;
const int _opFMod = 141;
const int _opVectorTimesScalar = 142;
const int _opMatrixTimesScalar = 143;
const int _opVectorTimesMatrix = 144;
const int _opMatrixTimesVector = 145;
const int _opMatrixTimesMatrix = 146;
const int _opDot = 148;
const int _opLabel = 248;
const int _opReturn = 253;
const int _opReturnValue = 254;

// GLSL extension constants defined at
// https://www.khronos.org/registry/spir-v/specs/unified1/GLSL.std.450.html

// Supported GLSL extension name
const String _glslStd450 = 'GLSL.std.450';

// Supported GLSL ops
const int _glslStd450Trunc = 3;
const int _glslStd450FAbs = 4;
const int _glslStd450FSign = 6;
const int _glslStd450Floor = 8;
const int _glslStd450Ceil = 9;
const int _glslStd450Fract = 10;
const int _glslStd450Radians = 11;
const int _glslStd450Degrees = 12;
const int _glslStd450Sin = 13;
const int _glslStd450Cos = 14;
const int _glslStd450Tan = 15;
const int _glslStd450Asin = 16;
const int _glslStd450Acos = 17;
const int _glslStd450Atan = 18;
const int _glslStd450Atan2 = 25;
const int _glslStd450Pow = 26;
const int _glslStd450Exp = 27;
const int _glslStd450Log = 28;
const int _glslStd450Exp2 = 29;
const int _glslStd450Log2 = 30;
const int _glslStd450Sqrt = 31;
const int _glslStd450InverseSqrt = 32;
const int _glslStd450FMin = 37;
const int _glslStd450FMax = 40;
const int _glslStd450FClamp = 43;
const int _glslStd450FMix = 46;
const int _glslStd450Step = 48;
const int _glslStd450SmoothStep = 49;
const int _glslStd450Length = 66;
const int _glslStd450Distance = 67;
const int _glslStd450Cross = 68;
const int _glslStd450Normalize = 69;
const int _glslStd450FaceForward = 70;
const int _glslStd450Reflect = 71;
Loading