[localization][tests]: use-deferred-loading: true throws _WidgetTypeFinder:<zero widgets with type "Scaffold" exception

[cedvdb]

Details

We have a simple unit test that's failing when it is copy pasted and tested twice.

EG:

    testWidgets('should be properly sorted without favorites', (tester) async {
      await tester.pumpWidget(builder());
      await tester.pumpAndSettle();
      final scaffold = find.byType(Scaffold);
      expect(scaffold, findsOneWidget);
    });

    // // // same test if added, fails
    // testWidgets('should be properly sorted without favorites', (tester) async {
    //   await tester.pumpWidget(builder());
    //   await tester.pumpAndSettle();
    //   final scaffold = find.byType(Scaffold);
    //   expect(scaffold, findsOneWidget);
    // });

We pinpointed the issue to the custom localization delegate used by the library. More specifically the deferred loading

Both test pass when using arb files with flutter --gen-l10n, one of two test pass when using use-deferred-loading: true
One of the two test fail when loading a json for localization instead. So maybe rootBundle.loadString is at cause here.

reproduce

We have two branch here that have been reduced to the minimum to reproduce the issue (at least in the test files);

This one fails when the second test is uncommented. The test is the only one remaining and is in test/_country_selector_test.dart

git clone https://github.com/cedvdb/phone_form_field/tree/localization_json && cd phone_form_field && flutter pub get && flutter test

This one works with use-deferred-loading: false (or not specified) and does not work with it set to true. This can be set in l10n.yaml

git checkout localization_arb && flutter pub get && flutter test

Version:

flutter --version
Flutter 2.6.0-0.0.pre • channel dev • https://github.com/flutter/flutter.git
Framework • revision 83b9e99 (3 weeks ago) • 2021-08-23 19:03:21 +0200
Engine • revision d5adde0
Tools • Dart 2.15.0 (build 2.15.0-42.0.dev)

but I believe it happens on versions anterior to this one.

[darshankawar]

@cedvdb
This doesn't seem to be a performance related issue. I cloned the repo and ran as prescribed which passed, as below:

Cloning into 'phone_form_field'...
remote: Enumerating objects: 1216, done.
remote: Counting objects: 100% (1216/1216), done.
remote: Compressing objects: 100% (860/860), done.
remote: Total 1216 (delta 507), reused 987 (delta 292), pack-reused 0
Receiving objects: 100% (1216/1216), 7.45 MiB | 5.95 MiB/s, done.
Resolving deltas: 100% (507/507), done.
dhs@Dhss-MacBook-Pro ncflutter % cd phone_form_field
dhs@Dhss-MacBook-Pro phone_form_field % flutter pub get
Running "flutter pub get" in phone_form_field...                    7.2s
Running "flutter pub get" in example...                          1,072ms
dhs@Dhss-MacBook-Pro phone_form_field % flutter test
00:11 +17: All tests passed!                                                   
dhs@Dhss-MacBook-Pro phone_form_field % 

Same with other one as well:

dhs@Dhss-MacBook-Pro phone_form_field % git checkout localization_arb && flutter pub get && flutter test
M	example/pubspec.lock
M	pubspec.lock
Branch 'localization_arb' set up to track remote branch 'localization_arb' from 'origin'.
Switched to a new branch 'localization_arb'
Running "flutter pub get" in phone_form_field...                   576ms
Running "flutter pub get" in example...                            566ms
00:02 +2: All tests passed!                                                                        
dhs@Dhss-MacBook-Pro phone_form_field % 

Also checked by directly running country_selector_test.dart which was uncommented and as-is which worked as below:

I verified this on latest stable (2.5.0)

[cedvdb]

In the first branch localization_json you ran flutter test, the second test is commented. Please uncomment or git pull second test to see it failing.

Note: this is not a performance issue, this was mislabeled

[cedvdb]

Actually I found a nice find to help debug this:

When using the second repository (arb) which was working, if I use use-deferred-loading: true when generating localization it also stops working

[darshankawar]

Thanks for the details. From localization_json branch, if I run the tests as-is, I get below exception:

console error log

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TestFailure object was thrown running a test:
  Expected: exactly one matching node in the widget tree
  Actual: _WidgetTypeFinder:<zero widgets with type "Scaffold" (ignoring offstage widgets)>
   Which: means none were found but one was expected

When the exception was thrown, this was the stack:
#4      main.<anonymous closure>.<anonymous closure> (file:///Users/dhs/Downloads/phone_form_field-localization_json/test/_country_selector_test.dart:37:7)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)

This was caught by the test expectation on the following line:
  file:///Users/dhs/Downloads/phone_form_field-localization_json/test/_country_selector_test.dart line 37
The test description was:
  should be properly sorted without favorites
════════════════════════════════════════════════════════════════════════════════════════════════════

Test failed. See exception logs above.
The test description was: should be properly sorted without favorites

From the same test file, if I comment out below snippet and run again, the same tests passes properly:

  // locale: Locale('en'),
          // localizationsDelegates: [
          //   ...GlobalMaterialLocalizations.delegates,
          //   PhoneFieldLocalization.delegate,
          // ],
          // supportedLocales: [Locale('en')],

Is this a correct interpretation of the issue ?

[cedvdb]

That is correct for the localization_json

For the localization_arb one if you use deferred loading for the l10n.yaml you'll also get the same error. The arb one is a better case for reproducity because at that point there is nothing from my app and only vanilla flutter code or code generated by flutter itself. That is the localization is generated by flutter and there is only flutter widgets in the test case.

In any case, this shows something is wrong with the deferred file loading, be it manual or with generated files by flutter.