[ERROR:flutter/runtime/dart_vm_initializer.cc(40)] Unhandled Exception: FormatException: Unhandled ContentModality format

[DingQLee]

This is the error code:

E/flutter (29563): [ERROR:flutter/runtime/dart_vm_initializer.cc(40)] Unhandled Exception: FormatException:
Unhandled ContentModality format
E/flutter (29563): AUDIO
E/flutter (29563): #0 ContentModality._parseValue (package:firebase_vertexai/src/api.dart:538:9)
E/flutter (29563): #1 _parseModalityTokenCount (package:firebase_vertexai/src/api.dart:881:45)
E/flutter (29563): #2 MappedListIterable.elementAt (dart:_internal/iterable.dart:435:31)
E/flutter (29563): #3 ListIterator.moveNext (dart:_internal/iterable.dart:364:26)
E/flutter (29563): #4 new _GrowableList._ofEfficientLengthIterable (dart:core-patch/growable_array.dart:189:27)
E/flutter (29563): #5 new _GrowableList.of (dart:core-patch/growable_array.dart:150:28)
E/flutter (29563): #6 new List.of (dart:core-patch/array_patch.dart:39:18)
E/flutter (29563): #7 ListIterable.toList (dart:_internal/iterable.dart:224:7)
E/flutter (29563): #8 _parseUsageMetadata (package:firebase_vertexai/src/api.dart:861:57)
E/flutter (29563): #9 parseGenerateContentResponse (package:firebase_vertexai/src/api.dart:756:7)
E/flutter (29563): #10 _MapStream._handleData (dart:async/stream_pipe.dart:211:31)
E/flutter (29563): #11 _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:151:13)
E/flutter (29563): #12 _RootZone.runUnaryGuarded (dart:async/zone.dart:1609:10)
E/flutter (29563): #13 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:366:11)
E/flutter (29563): #14 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:297:7)
E/flutter (29563): #15 _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:777:19)
E/flutter (29563): #16 _StreamController._add (dart:async/stream_controller.dart:651:7)
E/flutter (29563): #17 _StreamController.add (dart:async/stream_controller.dart:606:5)
E/flutter (29563): #18 _AsyncStarStreamController.add (dart:async-patch/async_patch.dart:80:16)
E/flutter (29563): #19 HttpApiClient.streamRequest (package:firebase_vertexai/src/client.dart)
E/flutter (29563):
E/flutter (29563): #20 _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:150:3)
E/flutter (29563):
E/flutter (29563):

Related packages:

• firebase_vertexai: ^1.3.0
• record: ^5.2.1
  I am using android pixel 3a API 29 for debuggin. I also have an iPhone for debugging too. The error behaves the same, so I guess it is an api problem.

I am trying to get response from gemini using vertex ai in flutter with audio + text input. I continuously get this error.

I tried using all supported audio format and they all didn't work. Plus, I found nothing from this error messag in the internet.

Please also note that I did get some (1 - 2) chunks before the error occurs. The response is relavent too (it says my singing needs more practice but anyway).

Please tell me to provide more info if needed. Thanks a lot.

[DingQLee]

 The related components:

 void getContent(Uint8List audioData) async {
    setState(() {
      answer = '';
      isLoading = true;
    });
    final prompt = TextPart('Please comment on my singing.');
    final audio = await File(_filePath!).readAsBytes();
    final thisAudioPart = InlineDataPart('audio/mp4', audio);
    final response = model.generateContentStream([
      Content.multi([prompt, thisAudioPart])
    ]);
    await for (final chunk in response) {
      print(chunk.text);
      setState(() {
        answer += chunk.text!;
      });
    }
    setState(() {
      isLoading = false;
    });
    if (_filePath != null) {
      try {
        final file = File(_filePath!);
        await file.delete();
        print('Audio file deleted');
      } catch (e) {
        print('Error deleting audio file: $e');
      }
    }
  }

[aradhya2211]

I am having the same issue, but with application/pdf file.. what could be the reason?

FormatException: Unhandled ContentModality format
DOCUMENT

[DingQLee]

Video, audio, pdfs... all have this issue.

For this error message "FormatException: Unhandled ContentModality format", I read about other flutter runtime error. Other developers mentioned that it is a flutter-firebase issue. I also tried all available models. All gave the same error.

The related issue (it is actually not too related but it shows that flutter_vertexai doesn't get enough attention from Google dev team...) [(https://github.com/google-gemini/deprecated-generative-ai-dart/issues/184)]

Some developers with the related issue i mentioned tried updating the firebase version and it worked. FYI: they did flutter upgrade , flutter clean + flutter pub get and flutter run .

I guess we will need to wait until the official Vertexai update... my progress completely stopped for parts that involve video/audio/other file inputs...

[aradhya2211]

Hi, I downgraded the vertex ai plugin from 1.3.0 to 1.0.0, then flutter clean, flutter pub get .. and now it's working.

[SelaseKay]

Hi @DingQLee, thanks for the report and pointing out the possible solution. Is the issue resolved for you now?

[DingQLee]

unfortunately, the problem is not solved... sorry that it didn't help.