Automatic retry causes test failures with a duration in pumpAndSettle() in tests

[remax21]

Issue with automatic retry in Riverpod 3 in tests

Problem

Hi,

I've observed unexpected behavior with Riverpod's automatic retry mechanism in my Flutter tests. When using pumpAndSettle() with a long duration (e.g., 30 seconds) to wait for asynchronous operations to complete, automatic retries generate multiple exceptions that escape the try-catch block in my widget and cause the test to fail with the error "Multiple exceptions (3) were detected during the running of the current test, and at least one was unexpected."

Observed behavior

• ✅ Test passes: With pumpAndSettle() without duration, retries don't complete before the test ends, so no error occurs
• ❌ Test fails: With pumpAndSettle(Duration(seconds: 30)), retries complete and generate multiple exceptions that bubble up to the test framework
• ✅ Test passes: By disabling retry with retry: (_, __) => null, the test passes even with a long duration

Question

In my applications, I often need to wait for asynchronous calls in my tests, which requires using pumpAndSettle() with a sufficient duration. In these cases, tests fail due to automatic retries => but it works in my application because I can wait for the retry

Should automatic retry be systematically disabled in tests with Riverpod? Is there a recommended best practice for handling this case?

Reproducible example

import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:flutter_test/flutter_test.dart';

/// Minimal reproducible example for Riverpod automatic retry issue in tests
///
/// Problem: When using `pumpAndSettle()` with a long duration in tests,
/// automatic retries generate multiple exceptions that escape the try-catch
///
/// Question: Should automatic retry be disabled in tests with Riverpod?

// Provider that always throws an exception
// This will trigger automatic retry
final throwingProvider = Provider<String>((ref) {
  print('Provider throwing exception');
  throw Exception('Test error');
});

// Widget that catches the exception
class TestWidget extends ConsumerWidget {
  const TestWidget({super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    var displayText = 'Default';

    try {
      displayText = ref.watch(throwingProvider);
    } catch (error) {
      displayText = 'Error caught';
    }

    return Scaffold(body: Center(child: Text(displayText)));
  }
}

void main() {
  testWidgets(
    'PASSES: pumpAndSettle() without duration - retries do not complete',
    (tester) async {
      await tester.pumpWidget(
        const ProviderScope(child: MaterialApp(home: TestWidget())),
      );

      await tester.pumpAndSettle();

      // The widget catches the exception and displays "Error caught"
      expect(find.text('Error caught'), findsOneWidget);
    },
  );

  testWidgets(
    'FAILS: pumpAndSettle() with 30s duration - retries complete and generate exceptions',
    (tester) async {
      await tester.pumpWidget(
        const ProviderScope(child: MaterialApp(home: TestWidget())),
      );

      // With a longer duration, pumpAndSettle() waits for retries to complete
      // This allows automatic retries to generate multiple exceptions
      // that escape the try-catch and cause the test to fail
      await tester.pumpAndSettle(const Duration(seconds: 30));

      // The widget catches the exception and displays "Error caught"
      expect(find.text('Error caught'), findsOneWidget);
    },
  );

  testWidgets(
    'PASSES: Disabling retry makes the test pass even with long duration',
    (tester) async {
      await tester.pumpWidget(
        ProviderScope(
          retry: (_, __) => null, // ✅ Disable automatic retry
          child: const MaterialApp(home: TestWidget()),
        ),
      );

      await tester.pumpAndSettle(const Duration(seconds: 30));

      expect(find.text('Error caught'), findsOneWidget);
      // ✅ Test passes when retry is disabled
    },
  );
}

[rrousselGit]

  testWidgets(
    'PASSES: pumpAndSettle() without duration - retries do not complete',
    (tester) async {
      await tester.pumpWidget(
        const ProviderScope(child: MaterialApp(home: TestWidget())),
      );

      await tester.pumpAndSettle();

      // The widget catches the exception and displays "Error caught"
      expect(find.text('Error caught'), findsOneWidget);
    },
  );

This shouldn't pass.

Caching errors inside widgets has no impact on whether the exception is considered caught or not. Looks like there's a bug that cause a silent exception to be allowed.

[remax21]

Thanks a lot for your reply, that makes sense.
From what I understand, this means the bug is on Riverpod’s side, right?
If so, do you think this is something that could be fixed in an upcoming release?

[remax21]

And also, if I understand well, this test should not pass too ?

testWidgets(
    'PASSES: Disabling retry makes the test pass even with long duration',
    (tester) async {
      await tester.pumpWidget(
        ProviderScope(
          retry: (_, __) => null, // ✅ Disable automatic retry
          child: const MaterialApp(home: TestWidget()),
        ),
      );

      await tester.pumpAndSettle(const Duration(seconds: 30));

      expect(find.text('Error caught'), findsOneWidget);
      // ✅ Test passes when retry is disabled
    },
    ```