Custom integrations with requirements fail to start with 2022.7.0

[mfschumann]

The problem

I am using two integrations (myenergi, gardena_smart_system) that have requirements according to their manifest.json. Since the upgrade of home assistant to 2022.7.0, both integrations fail to load:

[homeassistant.setup] Setup failed for custom integration gardena_smart_system: Unable to import component: No module named 'gardena'
[homeassistant.setup] Setup failed for custom integration myenergi: Unable to import component: No module named 'pymyenergi'

What version of Home Assistant Core has the issue?

core-2022.7.0

What was the last working version of Home Assistant Core?

core-2022.6.7

What type of installation are you running?

Home Assistant Container

Integration causing the issue

myenergi, gardena_smart_system

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

2022-07-07 11:17:10 INFO (SyncWorker_3) [homeassistant.util.package] Attempting install of pymyenergi==0.0.24
2022-07-07 11:17:10 DEBUG (SyncWorker_3) [homeassistant.util.package] Running pip command: args=['/usr/local/bin/python3', '-m', 'pip', 'install', '--quiet', 'pymyenergi==0.0.24', '--timeout', '60', '--upgrade', '--constraint', '/usr/src/homeassistant/homeassistant/package_constraints.txt', '--find-links', 'https://wheels.home-assistant.io/musllinux/', '--prefer-binary', '--user', '--prefix=']
2022-07-07 11:17:18 ERROR (MainThread) [homeassistant.setup] Setup failed for custom integration myenergi: Unable to import component: No module named 'pymyenergi'

Additional information

When installing the missing python packages manually inside the home assistant container (pip install pymyenergi py-smart-gardena), the integrations load fine again. Of course that is only temporary until the container is rebuilt.

The pip command executed by homeassistant.util.package (see logs above) seems to install the python packages into /lib/python3.10/site-packages where they cannot be found for importing. In earlier home assistant releases, the packages were instead installed into /config/deps/lib/python3.*/site-packages. So maybe something has been messed up in homeassistant.util.package with 2022.7.0?

[frenck]

This is an issue with a custom integration, please report issues with custom integration with the custom integration author or project. they are not maintained, created, or supported by the Home Assistant Project.

(Ps. Love your commander Keen avatar ❤️ )

../Frenck

[mfschumann]

For me it looks more like a regression in homeassistant.util.package. What's the right place to report this?

[frenck]

There is no change in that part. The right place to report issue with custom integration is with the authors/project of those custom integrations.

[mfschumann]

Are you saying that the custom integrations are responsible themselves for installing their python package dependencies? I thought that would be done by homeassistant.util.package. The issue I am reporting here is that the package dependencies of custom integrations are being installed into a wrong location since 2022.7.0 (/lib instead of /config/deps/lib) so that they cannot be imported. This would affect all custom integrations that have requirements in their manifest.json (that are not already satisfied by core or official integrations that happen to be installed).

I actually tried one more custom integration that requires python packages (wyzeapi). It fails with the same error as the other two integrations mentioned above. To me this strongly looks like the infrastructure for installing requirements for custom integrations broke with the update to 2022.7.0.

[frenck]

Are you saying that the custom integrations are responsible themselves for installing their python package dependencies?

I'm saying that reporting issues with custom integration should be reported with custom integration authors/projects.

[mfschumann]

I get that. According to what I found this issue is probably not a problem in individual custom integrations, but in the infrastructure installing the python package requirements for those. If that code is not in home-assistant/core, can you maybe point me to another issue tracker that is more appropriate?

[martinw72]

This is the first time I‘m not able to update to latest version! Hope you are able to find the right location for reporting this issue and someone is able to repair it!

[onedr0p]

Are you saying that the custom integrations are responsible themselves for installing their python package dependencies?

I'm saying that reporting issues with custom integration should be reported with custom integration authors/projects.

This may fall on deaf ears but if all custom integrations are broken, home assistant changed something purposely and didn't notify developers of custom integrations in the release notes or something was broken by accident or wasn't thoroughly tested. Repeating yourself 3 times with that same comment doesn't help anyone.