Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move OpenZwave cachefile & configuration to the addon/local directory. #1410

Closed
Petro31 opened this issue Jun 15, 2020 · 45 comments
Closed

Move OpenZwave cachefile & configuration to the addon/local directory. #1410

Petro31 opened this issue Jun 15, 2020 · 45 comments

Comments

@Petro31
Copy link

Petro31 commented Jun 15, 2020

Currently running into issues with my Zwave devices. I need to modify the cache file or the configuration files for devices. This is not possible because the samba addon does not expose the /usr/share/hassio/addons/data/, it only exposes /usr/share/hassio/addons/local/. So, both the cache file and the config files should move to the addons/local location. The other option could be to expose /usr/share/hassio/addons/data/core_zwave/ozw/config/ in samba share, but that won't always exist.

Anyone new user who purchases a device will not have access to make the device work properly if it's not fully supported by ozw. Ultimately we don't want users mucking with these files but it will need to be done for some. "Sorry your SOL should never be an option for users".

@frenck
Copy link
Member

frenck commented Jun 16, 2020

The local folder on disk is not what you think it is. It is for adding custom add-ons (like custom_components) and is not related to data at all.

If support for a device needs to be added, it needs to be raised at the OpenZWave project.

@Petro31
Copy link
Author

Petro31 commented Jun 16, 2020

@frenck the fix in question has been 3 years in the waiting so far. It was written against in 2017. Mind you this affects hundreds of different dimming devices. Also, as it stands it doesn't look like the solution is anywhere in site. So when migrating from the 1.4 v built in we are regressing in functionality and giving users no way to fix the problem. I understand the want/need to have this issue fixed in openzwave, but forcing users to wait 'indefinitely will not bode well for many users.

@Petro31
Copy link
Author

Petro31 commented Jun 16, 2020

All I'm asking for is a way for HassOS users to modify the cache file and configuration files. This is currently available with the built in home assistant. It is not available with openzwave and the only limiting factor is because we don't allow configuration modification. This will be needed.

@frenck
Copy link
Member

frenck commented Jun 16, 2020

So, just wondering, you have the device configuration you need right? Why not open a PR @ the OpenZWave project so all those people using those "Mind you this affects hundreds of different dimming devices" can benefit from it?

Like, please don't get me wrong, I'm open for a lot of things, but we should focus on fixing the issues, not on fixing workarounds. Hence my response.

@Petro31
Copy link
Author

Petro31 commented Jun 16, 2020

There is a band-aide in the works that will allow you to set the command class via mqtt I believe. But this is only one of the issues that I've had to adjust files for.

The real fix is a long standing issue with dimmers in general on ozwave products. In that PR, it was mentioned that it was being looked into and kept open in 2018 by the main dev. Now 2.5 years later, there's still no 'real fix'. Only the band aide. I'll wait, but I still think we should allow users to adjust configuration settings. Even if the fix is simply pointing to an override directory for both of these files like the current implementation allows. I.e. /config/ozwave/config.

@Petro31
Copy link
Author

Petro31 commented Jun 16, 2020

Also, I have a completely unrelated issue that I need to solve that appears to be a configuration issue, unless I have to re-include all the devices.

@Fishwaldo
Copy link

@frenck as you probably know, the database is mainly maintained by the community - that means before users can submit a PR to fix something, they need some way to edit their local installations to do the testing.

Even for me - helping users to troubleshoot is impossible - often support consists of “please try this and see if it fixes the issue” is the only way as I don’t own the 1100+ devices in the database. My only option is to point them to my docker image right now which is frustrating for a lot of users.

Exposing the config would:

  1. help me immensely with respect to time spent helping users
  2. ease the path for users to contribute new device support/fixes etc.

Currently this addon is by far the most popular method of getting ozwdaemon running for users, but also the most difficult to support and troubleshoot

@frenck
Copy link
Member

frenck commented Jun 19, 2020

@Fishwaldo I was not against, I even stated I was open for these things. In the meantime, I tried to offer an alternative approach that would likely benefit more people.

This is on my radar, but needs checking with the team implementing the integration to ensure it does not collide with anything on their roadmap.

@frenck frenck self-assigned this Jun 19, 2020
@Petro31
Copy link
Author

Petro31 commented Jun 19, 2020

I think we should also look into adding the log there as well. I'm currently switching to the AIO continainer instead of the addon because I can't get more than ~10k lines of the logs. Which ammounts to ~1 minute of my 6 minute startup process. I only have 50 devices. Some people have upwards of 100 devices.

@jef-pearlman
Copy link

jef-pearlman commented Jun 23, 2020

Adding that I also have been relying on this capability in the old integration and would really appreciate something similar in the new one. It's one of two things left that aren't already as good or better than the old experience for me. While I also hope to push my edits/fixes/additions to the main ozw project, things seem to get stuck in the pipeline and I'd also much prefer to test my changes within your well-maintained setup than try to build my own. (And I would love easy access to the logs too.)

FWIW, I made a thread about this in the forum a few weeks back, which has a few replies:
https://community.home-assistant.io/t/openzwave-beta-custom-device-xml-files/200898

I also brought it up in the #zwave Discord channel and made a little progress there, but not enough to be a replacement for an official solution.

And I do see above that you're open to it/it's on your radar and you're checking with the team--many thanks!

@Petro31
Copy link
Author

Petro31 commented Jun 25, 2020

Another post here that requires access to these files.

https://community.home-assistant.io/t/dimmer-lights-show-as-on-even-when-turned-off/152975/6

FYI, anyone using jasco products prior to 2018 will need to access these files. So this issue will just keep popping up on the forums.

@frenck
Copy link
Member

frenck commented Jun 25, 2020

Thanks, @Petro31. However, it is not needed to build a case. Please have a bit of patience for this one.

@Mattie
Copy link

Mattie commented Jul 4, 2020

Thanks so much for looking into this-- I find this is needed, too, for updates. Looking forward to making the switch!

@stale
Copy link

stale bot commented Aug 22, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 22, 2020
@Petro31
Copy link
Author

Petro31 commented Aug 23, 2020

Still needed.

@stale stale bot removed the stale label Aug 23, 2020
@frenck
Copy link
Member

frenck commented Aug 23, 2020

blocked by home-assistant/supervisor#1842

@wauswaus
Copy link

wauswaus commented Sep 18, 2020

Where are we with this? I also would love to contribute my making configuration files but it is so hard now, especially with limited Linux knowledge.

@storeman
Copy link

storeman commented Oct 4, 2020

I also would love this feature. I finally managed to get to the host terminal and enter the terminal of the homeassistant/armv7-addon-zwave container and edit the xml file.

First I noticed two different xml files in two locations for the same device.
I tried to edit the files to add an extra config option.

When restarting the container, the watchdog acted and deleted the old container and created a new one, so my changes were gone.

I disabled the watchdog, but than the container stops, because of issues with the MQTT broker. I had a lot of trouble getting into the host OS, and than finding the ability to enter docker containers. This next step is a little to much. An easier way to edit the config files is very welcome.

If I could test local configurations I could create pull requests on the project.

@Cyberwizzard
Copy link

I am also blocked in debugging my Z-wave issues because I got recommended to change a device XML, which requires this feature.

So +1 on adding this.

@gongtao0607
Copy link

I have a PR (#1674) to fix this, which I am using for debugging Z-wave devices. But PR is apparently blocked by home-assistant/supervisor#1842.

If anyone needs this urgently (for modifying zwave xml purpose), grab my PR and point docker image to gongtao0607/{arch}-addon-zwave

@frenck
Copy link
Member

frenck commented Nov 8, 2020

Small correction, @gongtao0607 is working around it, it is not fixing it.

@contactcr
Copy link

I also need to be able to edit the ozwcache_*.xml to fix an issue with some motorized blinds I have. They show up as dimmers instead of covers and they are unusable like this.

@frenck
Copy link
Member

frenck commented Nov 9, 2020

@contactcr see all of the above. The best thing to do right now, is get that device fixed in OpenZwave itself.

@contactcr
Copy link

@contactcr see all of the above. The best thing to do right now, is get that device fixed in OpenZwave itself.

If I get it fixed in Open Zwave what do I do to get the addon to fetch new configs?

@frenck
Copy link
Member

frenck commented Nov 12, 2020

We update the add-on with the new definitions on a regular basis

@gongtao0607
Copy link

gongtao0607 commented Nov 12, 2020

If I get it fixed in Open Zwave what do I do to get the addon to fetch new configs?

Try this: https://github.com/OpenZWave/qt-openzwave/blob/7ebd43e246e97851ad20f2b1bb9c62bfd6a4a6ef/docs/MQTT.md#downloadlatestconfigfilerevision

It fetches from openzwave database

@nappyjim
Copy link

Has this been remedied? Looking to move from Z wave to OpenZWave but when testing OpenZWave, it will not natively see scenes on my inovelli switch (ie, if i press up on the paddle 2x)

@frenck
Copy link
Member

frenck commented Nov 14, 2020

Has this been remedied?

The issue is open, so it isn't solved.

@wauswaus
Copy link

So, I made a config file and it is added to the OpenZwave GitHub - untested of course as there is almost no way of testing...
Result: somewhere in my config file there is something wrong, causing the openzwave daemon to crash...

Long story short, adding something to the openzwave project and just hoping it works is a painful process. I now have to make a pull request and wait for the next Home assistant openzwave integration update to see if I fixed things properly. Just wanted this of my chest.

Now the question, can we maybe get an update on where we stand with this? Also with home-assistant/supervisor#1842 as this is blocking, but in that issue there is also not much activity.

@frenck
Copy link
Member

frenck commented Nov 21, 2020

Feel free to help out with the implementation of the RFC, if you want things quicker.

@wauswaus
Copy link

Please, there is no need to get cynical. If I wasn't travelling the world to engineer and commission various Amazon warehouse conveyor systems so people can order their stuff they want, I would.
I appreciate all the work that is being done by the community. I just was curious on where we stand so I can make a better decision on if I want to proceed with the current hardware I have.

@frenck
Copy link
Member

frenck commented Nov 22, 2020

It wasn't cynical, help is really appreciated in an open source project. That is how an open source project works, it relies on contributions.

@nickolas1969
Copy link

This looks like the place. I recently ran into to possible needs for accessing the ozw-XXXXXXX.xml and manufacturer data folder.

I needed to add custome classes for my Z-Wave remote fob, and I am enjoying the Manufacturer clash between Inovelli and Minoston modules for ID: 0312.

The help I have received thus far all seem to have easy edits to the config files behind the Docker Container.

https://www.home-assistant.io/docs/z-wave/device-specific/#fibaro-keyfob-fgkf-601 is the first issue I am looking at. The second is that I have downloaded the manufacturer's .xml files for Minoston, so placing that in the folder for manufacturer specific files and then removing the ID: 0312 from the Inovelli definition.

Thoughts?

@Petro31
Copy link
Author

Petro31 commented Nov 23, 2020

This looks like the place. I recently ran into to possible needs for accessing the ozw-XXXXXXX.xml and manufacturer data folder.

I needed to add custome classes for my Z-Wave remote fob, and I am enjoying the Manufacturer clash between Inovelli and Minoston modules for ID: 0312.

The help I have received thus far all seem to have easy edits to the config files behind the Docker Container.

https://www.home-assistant.io/docs/z-wave/device-specific/#fibaro-keyfob-fgkf-601 is the first issue I am looking at. The second is that I have downloaded the manufacturer's .xml files for Minoston, so placing that in the folder for manufacturer specific files and then removing the ID: 0312 from the Inovelli definition.

Thoughts?

This is not the place to discuss this. This thread is an enhancement request to give users the ability to access the configuration files for the OpenZWave HomeAssistant OS Addon.

@mikedahlgren
Copy link

I did not realize this was a problem until I hit this head on. Bought new door locks, they were not recognized by Home Assistant. Created a patch for openzwave (OpenZWave/open-zwave#2474).

However with no way to test if that patch works I have low expectations that they will accept it untested. Current plan is to just wait 3-4 months and hope it works when if it gets there. (Hope isn't a strategy, but it's all I can do)

@nickolas1969
Copy link

This looks like the place. I recently ran into to possible needs for accessing the ozw-XXXXXXX.xml and manufacturer data folder.
I needed to add custome classes for my Z-Wave remote fob, and I am enjoying the Manufacturer clash between Inovelli and Minoston modules for ID: 0312.
The help I have received thus far all seem to have easy edits to the config files behind the Docker Container.
https://www.home-assistant.io/docs/z-wave/device-specific/#fibaro-keyfob-fgkf-601 is the first issue I am looking at. The second is that I have downloaded the manufacturer's .xml files for Minoston, so placing that in the folder for manufacturer specific files and then removing the ID: 0312 from the Inovelli definition.
Thoughts?

This is not the place to discuss this. This thread is an enhancement request to give users the ability to access the configuration files for the OpenZWave HomeAssistant OS Addon.

Petro31, that is what I am requesting. I was providing additional examples of why this access is needed. With that access I would be able to try what is suggested in that link and for my unknown Minoston devices.

@nickolas1969
Copy link

Another reason to expose the config and manufacturer xml directory would be for folks using the Z-Uno for developing their own modules?

Z-WAVE.ME ZMEEZUNO Z-Uno-Z-Wave Board for Arduino, White https://www.amazon.com/dp/B01IRBDBZY

I’m sure being able to define your own capabilities would be needed, yes?

@frenck
Copy link
Member

frenck commented Nov 24, 2020

We all agree it needs to be exposed. There is no reason to provide more reasoning or defend the proposal.

What needs to happen is implementation, that is another thing. So let's stop providing examples of how this can be used. I think that is covered by now 👍

@edgimar
Copy link

edgimar commented Dec 6, 2020

In the meanwhile, before it is implemented, it would be nice if there were some kind of "official" documentation that explains what is and is not currently possible w.r.t. customizing openzwave config files, and what workarounds are available for those who want to customize them now.

@bigwind777
Copy link

I believe most problems can be fixed if the zwave add-on simply grant (r/w) permission to
/data/addons/data/core_zwave/ozw/config/manufacturer_specific.xml
No need to change ozwcache_xxxx.xml directly if users can make changes to manufacturer_specific.xml
(and if necessary create custom-mfg directories containing diy-devices.xml files).
The correct ozwcache_xxxx.xml will be generated by open z-wave automatically.
like:
-rw-r--r-- 1 root root 164223 Dec 10 19:32 manufacturer_specific.xml

drwxr-xr-x 2 root root 4096 Dec 13 07:30 custom-mfg
|
|- diy-device1.xml
|- diy-device2.xml

Yes, users will have to "docker exec -it [docker ID] /bin/bash" in order to make changes to the directory.

Having limited knowledge of HASSIO and z-wave, I am probably oversimplifying. But I am also totally frustrated because unrecognizable zwave devices is the only problem preventing upgrade from my old Hassbian installation.

@bigwind777
Copy link

I chmod 600 manufacturer_specific.xml and added an entry in the file for product type and id that OZW could not identify. Then restart OZW addon. That works around the issue for now.

@frenck
Copy link
Member

frenck commented Dec 15, 2020

Workarounds above are not recommended to use with the add-on and are at your own risk. None of the above mentioned are supported by the Home Assistant project.

@ohjohnsen
Copy link

ohjohnsen commented Dec 25, 2020

Can somebody quickly explain why getting local access to the OpenZWave config files is an issue needing a lot of development? I've been working with Docker on work projects earlier. And I don't quite understand why this is not just a case of modifying a docker-compose file to map a local folder to the config folder inside the container. Please note that I'm not trying to be a smarta** here - I'm just trying to understand what the core of the problem is.

@jef-pearlman
Copy link

jef-pearlman commented Dec 25, 2020

Can somebody quickly explain why getting local access to the OpenZWave config files is an issue needing a lot of development? I've been working with Docker on work projects earlier. And I don't quite understand why this is not just a case of modifying a docker-compose file to map a local folder to the config folder inside the container. Please note that I'm not trying to be a smarta** here - I'm just trying to understand what the core of the problem is.

I'm sure someone more authoritative than me will answer soon, but here's my understanding: Per
#1410 (comment), this change is blocked by home-assistant/supervisor#1842. My takeaway from that is that they're trying to move to a more unified system where exposed files from addons are put in some sort of organized hierarchy instead of randomly under /config wherever the addon developer chooses.

I think the devs want to put a hold on exposing new exposed configuration files until the new system is in place, as adding them now will create breaking changes when the new system is implemented and the files move. But the new system isn't implemented yet. The net result is that OpenZWave in releases isn't allowed be changed to expose the files in a custom location (that's being deprecated) and can't be changed to the new location (that's not implemented), so it's stuck. And apologies in advance if I've misread!

@stale
Copy link

stale bot commented Mar 11, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 11, 2021
@stale stale bot closed this as completed May 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests