Fixed: Cannot build pip dependency 'webrtc-noise-gain' required for HA's integrations

[pdsakurai]

Issue

Since the recent update on Home Assistant brought in v9.1.1 (#6906), the following integrations are no longer working:

1. assist_pipeline
2. bluetooth_adapters
3. cloud
4. mobile_app

This is caused by the missing pip dependency webrtc-noise-gain.
It is not built due to the missing C++11 compiler.

The same issue has been raised in #6914 (comment).

Fix

This pull request adds g++ as one of the dependencies of Home Assistant, by default, regardless of HW architecture.

Workaround

1. Install the C++11 compiler.
   For fresh installation of Home Assistant, do this beforehand. Then, you may skip the second step.

   apt install g++
   

2. Restart Home Assistant service.

   systemctl restart homeassistant
   

Logs containing the error message

This is during the reinstallation of Home Assistant but may also happen on a fresh install on a device with no g++ installed beforehand.

Collecting webrtc-noise-gain
  Downloading webrtc_noise_gain-1.2.3.tar.gz (3.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 6.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: webrtc-noise-gain
  Building wheel for webrtc-noise-gain (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for webrtc-noise-gain (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [77 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-cpython-312
      creating build/lib.linux-aarch64-cpython-312/webrtc_noise_gain
      copying webrtc_noise_gain/__init__.py -> build/lib.linux-aarch64-cpython-312/webrtc_noise_gain
      running build_ext
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/home/homeassistant/.pyenv/versions/3.12.1/include/python3.12 -c -std=c++17 flagcheck.cpp -o flagcheck.o -std=c++17
      gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
      compilation terminated.
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/home/homeassistant/.pyenv/versions/3.12.1/include/python3.12 -c -std=c++17 flagcheck.cpp -o flagcheck.o -std=c++14
      gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
      compilation terminated.
      gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/home/homeassistant/.pyenv/versions/3.12.1/include/python3.12 -c -std=c++17 flagcheck.cpp -o flagcheck.o -std=c++11
      gcc: fatal error: cannot execute ‘cc1plus’: execvp: No such file or directory
      compilation terminated.
      Traceback (most recent call last):
        ...omitted to shorten logs...
        File "/tmp/pip-build-env-14zkjojc/overlay/lib/python3.12/site-packages/pybind11/setup_helpers.py", line 283, in build_extensions
          ext.cxx_std = auto_cpp_level(self.compiler)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-14zkjojc/overlay/lib/python3.12/site-packages/pybind11/setup_helpers.py", line 266, in auto_cpp_level
          raise RuntimeError(msg)
      RuntimeError: Unsupported compiler -- at least C++11 support is needed!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for webrtc-noise-gain
Failed to build webrtc-noise-gain
ERROR: Could not build wheels for webrtc-noise-gain, which is required to install pyproject.toml-based projects

[MichaIng]

I see, pre-compiled wheels are not available for Python 3.12: https://pypi.org/project/webrtc-noise-gain/#files

But, this module is not required for a HA base installation, but only for your optional integrations, isn't it? We have dietpi.txt settings for this, to have additional modules and possibly required APT packages installed automatically: https://github.com/MichaIng/DietPi/blob/a6cddbe/dietpi.txt#L281-L286
We do not intend to install all possible dependencies for all possible integrations that can be used with HA 😉.

We should add this to our documentation 🤔. For the automatic module installation on reinstalls/updates alone, this is quite handy.

[pdsakurai]

I tried doing a fresh installation of HA in VM. The same error happens.

That module pip depedency webrtc-noise-gain becomes implicitly required as it is a prerequisite of the default integrations of HA:

1. assist_pipeline
2. cloud
3. mobile_app

Feb 26 02:13:06 DietPi Home Assistant[16794]: 2024-02-26 02:13:06.471 ERROR (MainThread) [homeassistant.setup] Setup failed for 'cloud': Requirements for assist_pipeline not found: ['webrtc-noise-gain==1.2.3'].
Feb 26 02:13:40 DietPi Home Assistant[16794]: 2024-02-26 02:13:40.911 ERROR (MainThread) [homeassistant.setup] Setup failed for 'mobile_app': Requirements for assist_pipeline not found: ['webrtc-noise-gain==1.2.3'].
Feb 26 02:13:43 DietPi Home Assistant[16794]: 2024-02-26 02:13:43.501 ERROR (MainThread) [homeassistant.setup] Setup failed for 'assist_pipeline': Requirements for assist_pipeline not found: ['webrtc-noise-gain==1.2.3'].
Feb 26 02:13:43 DietPi Home Assistant[16794]: 2024-02-26 02:13:43.502 ERROR (MainThread) [homeassistant.setup] Unable to set up dependencies of 'default_config'. Setup failed for dependencies: assist_pipeline, cloud, mobile_app
Feb 26 02:13:43 DietPi Home Assistant[16794]: 2024-02-26 02:13:43.502 ERROR (MainThread) [homeassistant.setup] Setup failed for 'default_config': (DependencyError(...), 'Could not setup dependencies: assist_pipeline, cloud, mobile_app')

Here's the complete log.

Edit:
Here's the list of HA's default integrations.

Since the pre-compiled pip is not available for some architectures, then I suppose it is best to add g++ module as required for HA's base installation.

Or... adding the instruction for workaround somewhere may suffice in exchange of smooth HA upgrade/installation.

[MichaIng]

Got it, so it's one of these modules which are installed on initial service start, rather than in HA installation. I never understood why they do not add these to requirements.txt. The way it is now, it slips through our test, because the installation finishes, the service starts up and the web UI is accessible.

[MichaIng]

While this solves the warnings in web UI, there is another error on service startup:

Feb 27 23:12:00 DietPi Home Assistant[15777]: 2024-02-27 23:12:00.901 ERROR (MainThread) [haffmpeg.core] FFmpeg fails [Errno 2] No such file or directory: 'ffmpeg'
Feb 27 23:12:00 DietPi Home Assistant[15777]: Traceback (most recent call last):
Feb 27 23:12:00 DietPi Home Assistant[15777]:   File "/mnt/dietpi_userdata/homeassistant/deps/lib/python3.12/site-packages/haffmpeg/core.py", line 137, in open
Feb 27 23:12:00 DietPi Home Assistant[15777]:     self._proc = await self._loop.run_in_executor(None, proc_func)
Feb 27 23:12:00 DietPi Home Assistant[15777]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Feb 27 23:12:00 DietPi Home Assistant[15777]:   File "/home/homeassistant/.pyenv/versions/3.12.1/lib/python3.12/concurrent/futures/thread.py", line 58, in run
Feb 27 23:12:00 DietPi Home Assistant[15777]:     result = self.fn(*self.args, **self.kwargs)
Feb 27 23:12:00 DietPi Home Assistant[15777]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Feb 27 23:12:00 DietPi Home Assistant[15777]:   File "/home/homeassistant/.pyenv/versions/3.12.1/lib/python3.12/subprocess.py", line 1026, in __init__
Feb 27 23:12:00 DietPi Home Assistant[15777]:     self._execute_child(args, executable, preexec_fn, close_fds,
Feb 27 23:12:00 DietPi Home Assistant[15777]:   File "/home/homeassistant/.pyenv/versions/3.12.1/lib/python3.12/subprocess.py", line 1950, in _execute_child
Feb 27 23:12:00 DietPi Home Assistant[15777]:     raise child_exception_type(errno_num, err_msg, err_filename)
Feb 27 23:12:00 DietPi Home Assistant[15777]: FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg'

I wonder why now FFmpeg is expected while not a single integration is installed, which would use it. Same is true for webrtc-noise-gain of course, for volume control in WebRTC video chat sessions, a pretty specific feature. But this just goes inline with other large dependencies added lately, which only a fraction of users will ever use. All around video/audio, actually: https://community.home-assistant.io/t/new-modules-pyturbojpeg-and-ha-av-installed-on-first-ha-core-start/563019

Let's keep it like that for now, because FFmpeg with all its libraries is huge, about 500 MiB.

[pdsakurai]

I ignored that error. 😅 Fortunately it is really not needed, unlike webrtc-noise-gain which breaks integrations when missing. I noticed its impact when my automations aborted due to the mobile_app's failure to send notification.