Unable to train new model

[demey]

openwakeword (0.5.1) installed using provided command - 'pip install openwakeword' is not contains functions specified in train.py:
generate_adversarial_texts - from openwakeword.data import generate_adversarial_texts
compute_features_from_generator - from openwakeword.utils import compute_features_from_generator

In other worlds corresponding files data.py and utils.py not contains functions imported in train.py.

In the second part (2. Download Data) of automatic_model_training_simple.ipynb, openwakeword as well can't be installed using following commands:

!git clone https://github.com/dscripka/openwakeword
!pip install -e ./openwakeword

Cloning into 'openwakeword'...
remote: Enumerating objects: 1081, done.
remote: Counting objects: 100% (146/146), done.
remote: Compressing objects: 100% (101/101), done.
remote: Total 1081 (delta 59), reused 66 (delta 45), pack-reused 935
Receiving objects: 100% (1081/1081), 3.10 MiB | 7.15 MiB/s, done.
Resolving deltas: 100% (648/648), done.
Obtaining file:///content/openwakeword
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  error: subprocess-exited-with-error
  
  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Getting requirements to build editable ... error
error: subprocess-exited-with-error

× Getting requirements to build editable did not run successfully.
│ exit code: 1
╰─> See above for output.

As result it fails on third part (3. Train the Model) with error:

Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py

As workaround solution I manually created whl file, which have been used to install correct version of openwakeword (with correct files data.py and utils.py). But even in this case the training process fails on third part (on third step) with following error:

/usr/local/lib/python3.10/dist-packages/torch_audiomentations/utils/io.py:27: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("soundfile")
INFO:root:##################################################
Computing openwakeword features for generated samples
##################################################
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 639, in <module>
    compute_features_from_generator(positive_clips_train_generator, n_total=len(os.listdir(positive_train_output_dir)),
  File "/usr/local/lib/python3.10/dist-packages/openwakeword/utils.py", line 556, in compute_features_from_generator
    F = AudioFeatures(device=device)
  File "/usr/local/lib/python3.10/dist-packages/openwakeword/utils.py", line 84, in __init__
    self.melspec_model = ort.InferenceSession(melspec_model_path, sess_options=sessionOptions,
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from /usr/local/lib/python3.10/dist-packages/openwakeword/resources/models/melspectrogram.onnx failed:Load model /usr/local/lib/python3.10/dist-packages/openwakeword/resources/models/melspectrogram.onnx failed. File doesn't exist
/usr/local/lib/python3.10/dist-packages/torch_audiomentations/utils/io.py:27: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("soundfile")
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 667, in <module>
    F = openwakeword.utils.AudioFeatures(device='cpu')
  File "/usr/local/lib/python3.10/dist-packages/openwakeword/utils.py", line 84, in __init__
    self.melspec_model = ort.InferenceSession(melspec_model_path, sess_options=sessionOptions,
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 452, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from /usr/local/lib/python3.10/dist-packages/openwakeword/resources/models/melspectrogram.onnx failed:Load model /usr/local/lib/python3.10/dist-packages/openwakeword/resources/models/melspectrogram.onnx failed. File doesn't exist

/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning: 

TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP). 

For more information see: https://github.com/tensorflow/addons/issues/2807

[trip5]

Also having the same error... my log:

Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
Traceback (most recent call last):
  File "/content/openwakeword/openwakeword/train.py", line 18, in <module>
    import openwakeword
ModuleNotFoundError: No module named 'openwakeword'
/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/tfa_eol_msg.py:23: UserWarning: 

TensorFlow Addons (TFA) has ended development and introduction of new features.
TFA has entered a minimal maintenance and release mode until a planned end of life in May 2024.
Please modify downstream libraries to take dependencies from other repositories in our TensorFlow community (e.g. Keras, Keras-CV, and Keras-NLP). 

For more information see: https://github.com/tensorflow/addons/issues/2807 

  warnings.warn(
/usr/local/lib/python3.10/dist-packages/tensorflow_addons/utils/ensure_tf_install.py:53: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.12.0 and strictly below 2.15.0 (nightly versions are not supported). 
 The versions of TensorFlow you are currently using is 2.8.1 and is not supported. 
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version. 
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
  warnings.warn(
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-3-cfa7805b268a>](https://localhost:8080/#) in <cell line: 92>()
     90     return None
     91 
---> 92 convert_onnx_to_tflite(f"my_custom_model/{config['model_name']}.onnx", f"my_custom_model/{config['model_name']}.tflite")
     93 
     94 # Automatically download the trained model files

2 frames
[/usr/local/lib/python3.10/dist-packages/onnx/__init__.py](https://localhost:8080/#) in _load_bytes(f)
    143     else:
    144         f = typing.cast(Union[str, os.PathLike], f)
--> 145         with open(f, "rb") as readable:
    146             content = readable.read()
    147     return content

FileNotFoundError: [Errno 2] No such file or directory: 'my_custom_model/hi_sweet_home.onnx'

[demey]

I'm found workaround solution for the first error related to openwakeword installation.
But finally stuck with error as on screenshot.
Process always stops on 75% regardless of number_of_examples, number_of_training_steps and false_activation_penalty values.

[dscripka]

Thank you for sharing the logs, I've found the cause of this issue, have pushed fixes to openWakeWord, and updated the training Colab notebooks accordingly. Please try the latest versions of the Colab notebooks, and let me know if you have any issues.

@demey, I assume this is not on a Colab notebook? See this discussion topic for the same issue, hopefully some of the ideas there are helpful.

[demey]

@dscripka yes. I used docker image (us-docker.pkg.dev/colab-images/public/runtime) with increased shared memory as described here: https://research.google.com/colaboratory/local-runtimes.html but not as local runtime.
With some manual actions it works well, but stuck somewhere in "Run validation and log validation metrics" part (train.py)