Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Homematic: cannot see/modify variables #18984

Closed
HiranChaudhuri opened this issue Dec 3, 2018 · 18 comments
Closed

Homematic: cannot see/modify variables #18984

HiranChaudhuri opened this issue Dec 3, 2018 · 18 comments
Labels
integration: homematic

Comments

@HiranChaudhuri
Copy link

HiranChaudhuri commented Dec 3, 2018 •
edited

Home Assistant release with the issue:
0.83.2

Last working Home Assistant release (if known):
never worked for me

Operating environment (Hass.io/Docker/Windows/etc.):
Docker running homeassistant/home-assistant:latest

Component/platform:
https://www.home-assistant.io/components/homematic/

Description of problem:
Homematic environment variables are not visible in Home Assistant, devices and their states show up ok. The situation only showed up as a problem when I tried moving automation from Homematic to Home Assistant and needed access to some variables.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

homematic:
  interfaces:
    wireless:
      host: 192.168.178.39
      resolvenames: json
    rf:
      host: 192.168.178.39
      resolvenames: json
  hosts:
    ccu2:
      host: 192.168.178.39
  local_port: 8124

logger:
  default: warning
  logs:
    homeassistant.components.homematic: debug
    pyhomematic: debug

Traceback (if applicable):
not available

Additional information:
Home Assistant logfile shows messages like
2018-12-03 21:28:44 DEBUG (SyncWorker_13) [pyhomematic._hm] ServerThread.getAllSystemVariables: Exception: <Fault -1: 'getAllSystemVariables: unknown method name'>
@danielperna84
Copy link
Contributor

danielperna84 commented Dec 6, 2018 •
edited

If you have a CCU you have to provide a username and password (to the hosts-section) as mentioned here in the documentation. Without proper credentials Home Assistant is not allowed to query the JSON-RPC API.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 8, 2018 •
edited

Thank you for the quick reaction. Now I got to actually test your suggestion. In fact I use Homematic without authentication and authorization - the system is open. You may discuss whether this is good but so far I did not have problems.

Now I configured a guest user, as the documentation suggests and placed the credentials into the configuration like this:

homematic:
  interfaces:
    wireless:
      host: 192.168.178.39
      resolvenames: json
      username: homeassistant
      password: nLGWekp3
    rf:
      host: 192.168.178.39
      resolvenames: json
      username: homeassistant
      password: nLGWekp3
  hosts:
    ccu2:
      host: 192.168.178.39
  local_port: 8124

But after a restart of HomeAssistant the logfile still contains messages like this one:
2018-12-08 12:01:20 DEBUG (SyncWorker_0) [pyhomematic._hm] ServerThread.getAllSystemVariables: Exception: <Fault -1: 'getAllSystemVariables: unknown method name'>
I seem to have no hint as of which method name was even tried.

I noticed though that now I have an entity
sensor.rollosollwert
which is what I was searching for. Unfortunately it does not show the current value, neither can I set the state using the developer's tools (/dev-state).

@HiranChaudhuri
Copy link
Author

Ah, the abovementioned variable rollosollwert may have come through some other configuration change that I tried:

  - platform: template
    sensors:
      rollosollwert:
        value_template: '{{ states.cover.meq0517350.attributes.RolloSollwert }}'
        entity_id: cover.meq0517350A
        friendly_name: Jalousie Wohnzimmer Rechts Sollwert

This would also explain why other variables from the CCU2 are not available on HomeAssistant.

@danielperna84
Copy link
Contributor

In fact I use Homematic without authentication and authorization

That doesn't matter. To access the JSON-RPC API a username and passwort always are required. Even if you don't set them on the CCU, it internally does some authentication transparently. At least I didn't get it to work back then when I have implemented that. So that's why credentials always are required.

The problem with the variables is still caused by your configuration. You also have to add the credentials in the hosts-section. Make it look like this:

homematic:
  interfaces:
    wireless:
      host: 192.168.178.39
      resolvenames: json
      username: homeassistant
      password: nLGWekp3
    rf:
      host: 192.168.178.39
      resolvenames: json
      username: homeassistant
      password: nLGWekp3
  hosts:
    ccu2:
      host: 192.168.178.39
      username: homeassistant
      password: nLGWekp3
  local_port: 8124

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 8, 2018 •
edited

I added credentials to the hosts/ccu2 part as you suggested. After a restart the error message I described is no longer visible, now I can see this in the logfile:

2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] ServerThread.getAllSystemVariables: Getting all System variables via JSON-RPC
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: Session.login
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.178.39:80/api/homematic
.cgi
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: SysVar.getAll
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.178.39:80/api/homematic
.cgi
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: Method: Session.logout
2018-12-08 19:04:55 DEBUG (SyncWorker_6) [pyhomematic._hm] RPCFunctions.jsonRpcPost: API-Endpoint: http://192.168.178.39:80/api/homematic
.cgi

This looks as promising as strange. Pyhomematic writes the method when succesful while it should tell it when unsuccessful. But next I checked the UI to see the values.

The environment variables are still not visible, and all the devices are missing now. Seems like the autodiscovery is impacted now. The "configuration check" function returns ok though.

@danielperna84
Copy link
Contributor

I just noticed another error in your configuration: you have not specified a port for your second interface. You have wireless and rf. I suppose one of them is for the old wireless devices, and the other for HomeMatic IP. For HomeMatic IP you have to specify the port 2010. The way you have it right now you are connecting twice to the same port, which would just load all device two times. I never tested this though, so that might be why you're not seeing anything right now.

The log you have posted is perfectly fine. That's exactly how it looks like if it's working correctly. The variables are all part of the attributes of a single entity. In your case the name should be homematic.ccu2. Go to the states panel (< >-button in the left menu), there you'll find the entity. The variables will be part of the details you can see there. You should also see the variables simply by clicking on the entity in the regular UI.

Please read the documentation again. All of this is described there in great detail. If you follow the instructions closely you shouldn't run into any troubles.
If you run into further issues, please open a thread in the community. The issues here at GitHub are intended for bugs and alike. In the community you'll meet a bigger audience with more chances of getting seen by people who are able to help you.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 8, 2018 via email

@danielperna84
Copy link
Contributor

The variables however do not show up even when I investigate the "homematic.ccu2" entity. It only shows friendly_name and icon attributes.

Then there should be something relevant in the log. You don't need to have debug loggin on. If this fails, then there should be a message in the warning or error level.

I actually only use Homematic with wireless devices. There is no wired device connected, and I have no Homematic IP devices. Would I still have to configure the interfaces?

Of course. Otherwise Home Assistant wouldn't know about the CCU. And in your case you onle need 1 interface, since you only use the regular wireless devices. The documentation says:
(list) (Required) Configuration for each XML-RPC interface to integrate into Home Assistant.
An XML-RPC interface is what Home Assistant is using to read and write data from and to the devices. By desing each type (wired, wireless and IP) has it's own port (2000, 2001 and 2010). 2001 is the default, since most people have the regular wireless devices like you do. That's why in your case you can leave out the port. The documentation I am referring to is this one: https://www.home-assistant.io/components/homematic/
The second paragraph tries to sum up what I have explained here. If you feel the explanation given in the documentation is not clear enough, please tell me how to improve it so other users won't run into the same issues.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 9, 2018 via email

@danielperna84
Copy link
Contributor

Regarding the variables: do you actually allow connections to the JSON API? On the CCU2 the Firewall Settings have to like this:
hms
"Vollzugriff" would be ok as well, although that's not very secure.

@danielperna84
Copy link
Contributor

Also if username/password are definitely required the code coudl emit a warning or error message so users like me get a better hint what may be missing.

They are not required because not everybody needs those. There are people that use Homegear instead of a CCU, and there the reuquirements are different. Hence we can't show a warning because what would be a warning with the CCU is totally fine with Homegear and vice versa.

The Resolvenames option is described as it can be turned on and off. But the impact is not mentioned.

Well, it's not directly stated, that's correct. But in this conext in seems to have never occurred to anyone, that it could be unclear what the option does. If you don't set it you get the device IDs. What "resolve names" does seems intuitive in this context. Besides that it's briefly referenced here (in the paragraph below the example).
Apart from that: you can turn off resolvenames again and get back to the ID-based names. You just have to stick to one or the other. The problem with resolving names is that if for some reason you rename the device on the CCU, it will also change the ID in Home Assistant, and therefore break all related automations etc..

I went through the logs and could not find a single of my names in there.

Variables don't get logged. You could have hundres of them, and logging them would only unnecessarily bloat the log. All that would be relevant would be an error message when it fails to read the variables. But as mentioned in my post above, please check if you even allow access to the needed API.

Maybe it is worth a warning or even error message in the (missing) else block?

It's not missing. There are various legitimate reasons why the code can pass this point. And we only want to gather variables in this specific case. Every other case can, but not must be an error.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 9, 2018 via email

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 9, 2018 via email

@danielperna84
Copy link
Contributor

I did not know about that. Is it possible to query for the device HomeAssistant actually connects to? I've never used a HomeGear...

Using Homegear is an active process. It's an alternative to the CCU. So for CCU users it's irrelevant, and people that use Homegear know that they are using it.

Is that the case? Then I am wondering why this line looks exactly as I suggested for getting variables:

Setting a variable and failing at that could have negative impacts. Lets say you want to trigger some kind of alarm on your CCU. If that fails that could be really bad.
On the other side, the list of varialbes on the CCU can change, and maybe even be empty. It's been too long since I made this, so I don't know exactly what was the issue. But there are responses the CCU gives that are no problem, but are not worth logging. An explicit condition where where has been an error could be logged of course.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 10, 2018 via email

@danielperna84
Copy link
Contributor

So if I understand correctly there is no way pyhomematic can figure out itself whether it is talking to HomeGear or HomeMatic. But this information seem crucial to validate whether other configuration directives make sense or not.
Since users know which of the two systems they have, how about adding
an attribute to the configuration where exactly this is specified?It
seems easy for them to deliver the information, and based on that some
validation can then give further information which items are missing or
bad. This would be an improvement in usability.

It actually is possible to detect, but it doesn't make sense since you have to a) follow the documentation anyways, and b) don't necessarily require all options the CCU provides. I don't resolve names, and you don't do it as well. So we can't just say we need a username just because someone uses the CCU. And there are even more options than CCU and Homegear. hassio has a add-on that acts as an interface as well, and there you also don't need a username or password. It really iss dependent on any individual case. Besides that Home Assistant already has it's config validation for required and optional options. I do see the point in making things easier. But this is outside of the scope for HomeMatic and mor of an architectural decision of how Home Assistant in general handles configurations. For which there are already plans. It's mentioned somewhere in the developer documentation, so it's on their roadmap.

I agree it does not matter why you introduced logging. There was a problem, and there may be problems in the future as well.But are you aware about priorities? If I am concerned of a bloated logfile, I just turn off verbosity. In my case I was worried about functionality so I configured to have all message logged (debug) . Right now I am not happy when the system hides information that can help troubleshooting. If this were my code I would - similar like all the events and values for each device are logged - introduce a logger that logs all environment variables and their values - debug priority is sufficient. As soon as someone feels comfortable with the setup he'd switch to less verbose logging anyway.

Seeing the values of variables wouldn't have helped fixing your problem. If you see them in the entity it did work, so logging them is obsolete. If you don't see them in the entity, there wouldn't be anything to log.
Yes, we could log more error messages. But honestly: in around 2 years of Home Assistant + HomeMatic you're the first user I have witnessed that got stuck because of insufficient permissions of the user. But that's something that should be part of the documentation. Which is lacking this information right now. But that's easy to fix.

@HiranChaudhuri
Copy link
Author

HiranChaudhuri commented Dec 12, 2018 via email

@rgriebl
Copy link

rgriebl commented Dec 19, 2018

Just ran into the same problem and found this thread a little too late. On the plus side, I was down to a TCP dump and this clearly shows the problem:

       {
          "id": 0,
          "version": "1.1",
          "result": null,
          "error": {
            "name": "JSONRPCError",
            "code": 400,
            "message": "access.denied.(\"USER\".needed.1)"
       }

But pyhomematic will not tell you anything about it, even with debug logging enabled.
At least logging this error would have saved me 2 hours...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
integration: homematic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rgriebl @HiranChaudhuri @danielperna84 @cgarwood