Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot get yamnet features in flutter #200

Open
mikkel-convai opened this issue Mar 5, 2024 · 0 comments
Open

Cannot get yamnet features in flutter #200

mikkel-convai opened this issue Mar 5, 2024 · 0 comments

Comments

@mikkel-convai
Copy link

Hi

I am trying to use the yamnet tflite model from: "/kaggle/input/yamnet/tflite/tflite/1/1.tflite", to take in a 500 ms sound clip and extract embeddings and spectrogram.

I can load the model fine with:
final interpreter = await Interpreter.fromAsset('assets/1.tflite');

And get both input and output tensors like so:

interpreter.resizeInputTensor(0, singleSample.shape);
inputTensors = interpreter.getInputTensors();
Tensor inputTensor = inputTensors[0];
outputTensors = interpreter.getOutputTensors();
scores = outputTensors[0];
embeddings = outputTensors[1];
spectrogram = outputTensors[2];

The problem arises in figuring out how to format the input and output. I've tried the following:
First

  final input = List.generate(8000, (_) => List.filled(1, 0));
  final output = {0: scores, 1: embeddings, 2: spectrogram};
  interpreter.runForMultipleInputs(input, output);

Which gives error:

E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1
E/flutter (11660): #0 _Array.[] (dart:core-patch/array.dart:10:36)
E/flutter (11660): #1 ListMixin.elementAt (dart:collection/list.dart:78:33)
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:196
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 get_yamnet_features_single_sample
yamnet_features.dart:31

Second

final input = List.generate(8000, (_) => List.filled(1, 0));
 final output = {0: scores, 1: embeddings, 2: spectrogram};
 interpreter.run(input, output);

Which gives error:

E/tflite (11660): tensorflow/lite/kernels/pad.cc:79 SizeOfDimension(op_context->paddings, 0) != op_context->dims (1 != 2)
E/tflite (11660): Node number 15 (PAD) failed to invoke.
E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Bad state: failed precondition
E/flutter (11660): #0 checkState
check.dart:74
E/flutter (11660): #1 Interpreter.invoke
interpreter.dart:164
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:214
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 Interpreter.run
interpreter.dart:172
E/flutter (11660): #5 get_yamnet_features_single_sample
yamnet_features.dart:31
E/flutter (11660):

Third

  final input = List.generate(8000, (_) => List.filled(1, 0));
  final output = [scores, embeddings, spectrogram];
  interpreter.run(input, output);

Which gives error:

E/tflite (11660): tensorflow/lite/kernels/pad.cc:79 SizeOfDimension(op_context->paddings, 0) != op_context->dims (1 != 2)
E/tflite (11660): Node number 15 (PAD) failed to invoke.
E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Bad state: failed precondition
E/flutter (11660): #0 checkState
check.dart:74
E/flutter (11660): #1 Interpreter.invoke
interpreter.dart:164
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:214
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 Interpreter.run
interpreter.dart:172
E/flutter (11660): #5 get_yamnet_features_single_sample
yamnet_features.dart:31
E/flutter (11660):

Fourth

List<double> scoresList = List.filled(521, 0.0);
List<double> embeddingsList = List.filled(1024, 0.0);
List<double> spectrogramList = List.filled(64, 0.0);

final input = List.generate(8000, (_) => List.filled(1, 0));
final output = {0: scoresList, 1: embeddingsList, 2: spectrogramList};
interpreter.runForMultipleInputs(input, output);

Which again gives error:

E/flutter (11660): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1
E/flutter (11660): #0 _Array.[] (dart:core-patch/array.dart:10:36)
E/flutter (11660): #1 ListMixin.elementAt (dart:collection/list.dart:78:33)
E/flutter (11660): #2 Interpreter.runInference
interpreter.dart:196
E/flutter (11660): #3 Interpreter.runForMultipleInputs
interpreter.dart:180
E/flutter (11660): #4 get_yamnet_features_single_sample
yamnet_features.dart:38
E/flutter (11660):

It all looks like its just the way im setting up outputs, but im still not super experienced with interpretors and models in flutter, so i would appreciate some guidance or help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant