Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up first asset load by encoding asset manifest in binary rather than JSON #113637

Merged
Merged
Show file tree
Hide file tree
Changes from 108 commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
9a33150
initial attempt to use binary encoding for assetmanifest
andrewkolos Oct 18, 2022
d8e32a6
working
andrewkolos Oct 18, 2022
9dccdc6
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Oct 18, 2022
9c1782b
trim trailing whitespace
andrewkolos Oct 19, 2022
54ff2ed
update asset decoding test
andrewkolos Oct 20, 2022
866b972
Delete generated_plugins.cmake
andrewkolos Oct 20, 2022
3739c09
try to use compute
andrewkolos Oct 20, 2022
7cd4ca0
experiment with compute
andrewkolos Oct 21, 2022
0d4110f
remove json version of asset manifest
andrewkolos Oct 21, 2022
7c835fd
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Oct 21, 2022
4debadb
remove padding from asset manifest
andrewkolos Oct 21, 2022
ba3b36d
fix buffer length issue
andrewkolos Oct 22, 2022
92dbb3d
fix messed up method name
andrewkolos Oct 22, 2022
e7d4d78
trim trailing whitespace
andrewkolos Oct 22, 2022
f434777
remove trailing whitespace
andrewkolos Oct 22, 2022
23decc2
run flutter update-packages --force-upgrade
andrewkolos Oct 24, 2022
13986f8
add money_asset_manifest.bin to _legacyBinaries
andrewkolos Oct 24, 2022
3ffdf4d
Revert "run flutter update-packages --force-upgrade"
andrewkolos Oct 25, 2022
41233c2
restore JSON version of asset manifest
andrewkolos Oct 26, 2022
a7b7c52
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Oct 26, 2022
04aea55
restore iteration count
andrewkolos Oct 26, 2022
acec0c3
remove pointless bundle.clear call
andrewkolos Oct 26, 2022
9168628
normalize API
andrewkolos Oct 27, 2022
6c2c854
rewrite loadStructuredDataBinary to avoid event loop if data is in cache
andrewkolos Oct 27, 2022
3c282d9
provide base implementation for loadStructuredDataBinary
andrewkolos Oct 27, 2022
3065049
remove asynchrony from decodeAssetManifest and weaken types
andrewkolos Oct 27, 2022
02d7f69
make import for ServicesBinding more specific
andrewkolos Oct 27, 2022
3410cc0
fix test
andrewkolos Oct 28, 2022
91726f1
fix Failed image loads in debug mode test
andrewkolos Oct 28, 2022
35cbbb5
adjust verifyNoBinaries
andrewkolos Oct 28, 2022
451b32a
remove unnecessary await
andrewkolos Oct 28, 2022
8b6c072
fix tests that inspect bundle.entries
andrewkolos Oct 28, 2022
a0da1b5
fix more tests
andrewkolos Oct 28, 2022
1ed348a
adjust binding import
andrewkolos Oct 28, 2022
43ec1ce
Revert "adjust verifyNoBinaries"
andrewkolos Oct 31, 2022
1da88c3
(broken) have asset manifest use lists of dynamic objects for values
andrewkolos Nov 1, 2022
6f3d639
fix previous commit
andrewkolos Nov 1, 2022
7c84f49
clean up previous commit
andrewkolos Nov 1, 2022
450879b
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 1, 2022
e91cd51
centralize asset manifest parsing logic within image resolution logic
andrewkolos Nov 2, 2022
1b5d5a0
remove nullability of asset manifest
andrewkolos Nov 2, 2022
40033ce
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 2, 2022
c3e0fe1
remove abbreviation
andrewkolos Nov 2, 2022
6bc4091
revert update to binary allow list
andrewkolos Nov 2, 2022
99ce7fa
resolve library_private_types_in_public_api lint
andrewkolos Nov 2, 2022
0d19ec3
update asset_bundle_package_test.dart tests
andrewkolos Nov 3, 2022
50d21fb
update more tests
andrewkolos Nov 3, 2022
aba0650
restore AssetManifest.json tests
andrewkolos Nov 3, 2022
b84c921
fix tests in asset_bundle_test.dart
andrewkolos Nov 3, 2022
2b95265
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 3, 2022
104c5bb
fix image_resolution tests
andrewkolos Nov 3, 2022
eb82a46
fix tests in asset_bundle_test
andrewkolos Nov 4, 2022
a7c9850
fix tests in image_resolution_test.dart
andrewkolos Nov 4, 2022
1eb83df
nits
andrewkolos Nov 4, 2022
1e2fe39
nit
andrewkolos Nov 4, 2022
24231f1
add deprecation message and rename to loadStructuredBinaryData
andrewkolos Nov 7, 2022
8f58c27
make deprecation message slightly less ambiguous
andrewkolos Nov 7, 2022
566c7ed
Revert "make deprecation message slightly less ambiguous"
andrewkolos Nov 7, 2022
ce4edd6
Revert "add deprecation message and rename to loadStructuredBinaryData"
andrewkolos Nov 7, 2022
e41806b
add documentation of the asset manifest format
andrewkolos Nov 7, 2022
342fa71
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 8, 2022
17b5b9f
broken
andrewkolos Nov 8, 2022
abc800b
simplify asset manifest loading logic
andrewkolos Nov 8, 2022
3347063
fix future chain
andrewkolos Nov 8, 2022
a7fe460
nit
andrewkolos Nov 8, 2022
369447e
perf nit
andrewkolos Nov 8, 2022
44a066c
remove unnecessary late
andrewkolos Nov 8, 2022
ca70c1b
refactor _LegacyAssetManifest
andrewkolos Nov 8, 2022
35d85a3
Merge branch 'add-binary-encoding-for-asset-manifest' of https://gith…
andrewkolos Nov 8, 2022
1637fa4
formatting nit
andrewkolos Nov 8, 2022
d265cdf
nit
andrewkolos Nov 8, 2022
48cdf03
remove uses of dynamic
andrewkolos Nov 8, 2022
f4bfd3e
add cleanup issue
andrewkolos Nov 8, 2022
52708a6
remove accidental lint ignores
andrewkolos Nov 9, 2022
6355fb2
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 9, 2022
7658f16
correct linux analyze lints
andrewkolos Nov 9, 2022
091280e
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 9, 2022
152770f
flutter update-packages --force-upgrade
andrewkolos Nov 9, 2022
a6e967b
upgrade assertions to use matchers
andrewkolos Nov 9, 2022
89b93d5
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 9, 2022
3b73c04
run "flutter update-packages --force-upgrade"
andrewkolos Nov 10, 2022
049e782
fix AssetManifest.json fallback
andrewkolos Nov 15, 2022
eed94ec
i dont understand how async works in dart
andrewkolos Nov 15, 2022
2885846
Update asset_bundle.dart
andrewkolos Nov 15, 2022
5d5d245
i am oh so confused
andrewkolos Nov 15, 2022
692cacd
go back to using identity then with onError arg
andrewkolos Nov 16, 2022
2a09f74
fix
andrewkolos Nov 17, 2022
e14a816
fix nullcheck
andrewkolos Nov 17, 2022
1050390
remove redundant error handling
andrewkolos Nov 17, 2022
765182b
make naming more consistent
andrewkolos Nov 17, 2022
89fca21
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 17, 2022
dad9185
add comments
andrewkolos Nov 18, 2022
80941a4
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 18, 2022
ab5de3b
Merge branch 'master' into add-binary-encoding-for-asset-manifest
andrewkolos Nov 18, 2022
a4d80d8
upgrade dependencies
andrewkolos Nov 18, 2022
a3b0304
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 18, 2022
ef4a512
hmm
andrewkolos Nov 18, 2022
19fdf34
restore collection as prod dependency
andrewkolos Nov 18, 2022
23303d0
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 21, 2022
05110fc
trying something out
andrewkolos Nov 22, 2022
c20c436
lint
andrewkolos Nov 22, 2022
ce1699a
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 22, 2022
6468e99
Merge remote-tracking branch 'upstream/master' into add-binary-encodi…
andrewkolos Nov 29, 2022
ded125f
fix json string parsing
andrewkolos Nov 30, 2022
c743c78
add test for AssetManifest.json
andrewkolos Nov 30, 2022
44993cb
remove dependency on collection
andrewkolos Dec 1, 2022
d676d75
add a comment to hacky try-catch
andrewkolos Dec 1, 2022
53a726e
restore collection as dev dependency
andrewkolos Dec 1, 2022
263aaec
nits
andrewkolos Dec 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -3,9 +3,9 @@
// found in the LICENSE file.

import 'dart:convert';
import 'dart:typed_data';

import 'package:flutter/foundation.dart';
import 'package:flutter/services.dart' show PlatformAssetBundle;
import 'package:flutter/services.dart' show PlatformAssetBundle, StandardMessageCodec;
import 'package:flutter/widgets.dart';

import '../common.dart';
Expand All @@ -18,16 +18,14 @@ void main() async {
final BenchmarkResultPrinter printer = BenchmarkResultPrinter();
WidgetsFlutterBinding.ensureInitialized();
final Stopwatch watch = Stopwatch();
final PlatformAssetBundle bundle = PlatformAssetBundle();

final ByteData assetManifestBytes = await bundle.load('money_asset_manifest.json');
final ByteData assetManifest = await loadAssetManifest();

watch.start();
for (int i = 0; i < _kNumIterations; i++) {
bundle.clear();
final String json = utf8.decode(assetManifestBytes.buffer.asUint8List());
// This is a test, so we don't need to worry about this rule.
// This is effectively a test.
// ignore: invalid_use_of_visible_for_testing_member
await AssetImage.manifestParser(json);
AssetImage.parseAssetManifest(assetManifest);
}
watch.stop();

Expand All @@ -40,3 +38,49 @@ void main() async {

printer.printToStdout();
}

final RegExp _extractRatioRegExp = RegExp(r'/?(\d+(\.\d*)?)x$');

Future<ByteData> loadAssetManifest() async {
double parseScale(String key) {
final Uri assetUri = Uri.parse(key);
String directoryPath = '';
if (assetUri.pathSegments.length > 1) {
directoryPath = assetUri.pathSegments[assetUri.pathSegments.length - 2];
}
final Match? match = _extractRatioRegExp.firstMatch(directoryPath);
if (match != null && match.groupCount > 0) {
return double.parse(match.group(1)!);
}
return 1.0;
}

final Map<String, dynamic> result = <String, dynamic>{};
final PlatformAssetBundle bundle = PlatformAssetBundle();

// For the benchmark, we use the older JSON format and then convert it to the modern binary format.
final ByteData jsonAssetManifestBytes = await bundle.load('money_asset_manifest.json');
final String jsonAssetManifest = utf8.decode(jsonAssetManifestBytes.buffer.asUint8List());

final Map<String, dynamic> assetManifest = json.decode(jsonAssetManifest) as Map<String, dynamic>;

for (final MapEntry<String, dynamic> manifestEntry in assetManifest.entries) {
final List<dynamic> resultVariants = <dynamic>[];
final List<String> entries = (manifestEntry.value as List<dynamic>).cast<String>();
for (final String variant in entries) {
if (variant == manifestEntry.key) {
// With the newer binary format, don't include the main asset in it's
// list of variants. This reduces parsing time at runtime.
continue;
}
final Map<String, dynamic> resultVariant = <String, dynamic>{};
final double variantDevicePixelRatio = parseScale(variant);
resultVariant['asset'] = variant;
resultVariant['dpr'] = variantDevicePixelRatio;
resultVariants.add(resultVariant);
}
result[manifestEntry.key] = resultVariants;
}

return const StandardMessageCodec().encodeMessage(result)!;
}
Expand Up @@ -2,6 +2,8 @@
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

import 'dart:async';

import 'package:flutter/services.dart';
import 'package:flutter_gallery/gallery/example_code_parser.dart';
import 'package:flutter_test/flutter_test.dart';
Expand Down Expand Up @@ -58,4 +60,9 @@ class TestAssetBundle extends AssetBundle {

@override
String toString() => '$runtimeType@$hashCode()';

@override
Future<T> loadStructuredBinaryData<T>(String key, FutureOr<T> Function(ByteData data) parser) async {
return parser(await load(key));
}
}