home-assistant: missing sensor.mqtt in component-packages.nix

[netixx]

Issue description

Nixos build error when using a MQTT sensor, following this configuration.

error: attribute 'sensor.mqtt' missing, at /nix/store/qxw43qslgcx5g61npzficqqvyciwxrml-unstable-19.03pre169108.36f31600749/unstable/pkgs/servers/home-assistant/default.nix:82:28

Steps to reproduce

services.home-assistant = {
  enable = true;
  package = unstable.home-assistant;
  config = {
    sensor = [ 
      {
        name = "Temperature room";
        platform = "mqtt";
        state_topic = "sensors/temperature/room";
        qos = 0;
        unit_of_measurement = "°C";
      }
    ];
  };
};

unstable being the nixos-unstable channel.

Technical details

The corresponding line seems to have been removed in : 6cb6b90f1a67f0513fa34c3db880684e6ae533a6 which is the commit for the update to 0.86.4.

Maybe the code has changed and the python script no longer recognises sensors correctly ?

• system: "x86_64-linux"
• host os: Linux 4.14.95, NixOS, 18.09pre-git (Jellyfish)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.1.3
• channels(root): "nixos-18.09.2203.9bd45dddf81, unstable-19.03pre169108.36f31600749"
• nixpkgs: /var/nixpkgs

[netixx]

@dotlambda

[dotlambda]

The configuration you listed above does not yield an evaluation error. I have verified that by building it myself. You must have specified sensor.mqtt somewhere else.

[netixx]

I don't have any other mention of sensor.mqtt in my config, although I have also configured an external mqtt brocker with

services.home-assistant.config.mqtt = {
            broker = "127.0.0.1";
            username = "homeassistant";
            # password = "";
          };

which might change the evaluation.

I can confirm that when I remove the sensor config, the evaluation error goes away, which is why I didn't think I needed to include the brocker config.

Those are the only mention of mqtt in my config I promise!

Still it's strange that the line was removed from component-packages.nix no ?

[netixx]

I've just noticed now that I have forgotten autoExtraComponents = true; in my config...

Here is the actual pieces of relevant config:

services.home-assistant = {
  enable = true;
  autoExtraComponents = true;
  package = unstable.home-assistant;
  config = {
    mqtt = {
      broker = "127.0.0.1";
      username = "homeassistant";
      # password = "";
    };
    sensor = [ 
      {
        name = "Temperature room";
        platform = "mqtt";
        state_topic = "sensors/temperature/room";
        qos = 0;
        unit_of_measurement = "°C";
      }
    ];
  };
};

[dotlambda]

I've just noticed now that I have forgotten autoExtraComponents = true; in my config...

That's the default. No need to include it.

After all, I was able to reproduce your error by building NixOS from the stable channel. My bad. It's caused by the following line:

nixpkgs/nixos/modules/services/misc/home-assistant.nix

Line 13 in 1b621b8

availableComponents = pkgs.home-assistant.availableComponents;

which will use the availableComponents from the stable package.
However, if I set availableComponents = cfg.package.availableComponents; I get error: stack overflow (possible infinite recursion) which I still have to investigate.

[dotlambda]

I really have no idea what's going on. The way I test this is by having a file /home/rschuetz/test-nixos/configuration.nix with the following content:

{ ... }:

{
  imports = [
    /etc/nixos/hardware-configuration.nix
  ];

  boot.loader.systemd-boot.enable = true;

  services.home-assistant = {
    enable = true;
    package = (import <unstable> {}).home-assistant;
    config = {
      sensor = [
        {
          name = "Temperature room";
          platform = "mqtt";
          state_topic = "sensors/temperature/room";
          qos = 0;
          unit_of_measurement = "°C";
        }
      ];
    };
  };
}

and a checkout of the release-18.09 branch with the following diff at /home/rschuetz/dev/nixpkgs:

diff --git a/nixos/modules/services/misc/home-assistant.nix b/nixos/modules/services/misc/home-assistant.nix
index 0756e81612a..441fcb30569 100644
--- a/nixos/modules/services/misc/home-assistant.nix
+++ b/nixos/modules/services/misc/home-assistant.nix
@@ -10,7 +10,7 @@ let
     (builtins.toJSON (if cfg.applyDefaultConfig then
     (lib.recursiveUpdate defaultConfig cfg.config) else cfg.config));
 
-  availableComponents = pkgs.home-assistant.availableComponents;
+  availableComponents = cfg.package.availableComponents;
 
   # Given component "parentConfig.platform", returns whether config.parentConfig
   # is a list containing a set with set.platform == "platform".

When building the configuration I get

$ NIXOS_CONFIG=/home/rschuetz/test-nixos/configuration.nix nixos-rebuild build -I nixpkgs=/home/rschuetz/dev/nixpkgs -I unstable=https://github.com/NixOS/nixpkgs-channels/archive/nixos-unstable.tar.gz
building Nix...
building the system configuration...
error: stack overflow (possible infinite recursion)

What's strange is that there is no error when /home/rschuetz/dev/nixpkgs points to the master branch with the diff applied.
Maybe @peterhoeg or @FRidh have a clue.

[netixx]

I am happy you could reproduce this!

Still, I don't see any line for sensor.mqtt in the channels/nixos-unstable (but I see it in the channels/nixos-18.09), which I think it will lead to problems when it will be merged, no ?

What I still find strange, is that the referred file in my original error is from unstable...

Is this making any sense ?

[dotlambda]

Yes, this makes sense: The module is taking the list of availableComponents from the stable channel and passing those used in your config to the package from the unstable channel, which obviously can't find the component since it's no longer in component-packages.nix.

[netixx]

Ok, thank you for your explanations, I understand where the error comes from now !

I did some digging around to know why the sensor.mqtt line has disappeared from the component-packages.nix with the update. This is what I found out:

• on the home-assistant repository, the code for the mqtt sensor was moved from the homeassistant/components/sensor/mqtt.py to homeassistant/components/mqtt/sensor.py between 0.85.1 and 0.86.4
• this means that the sensor.mqtt is now mqtt.sensor in the component-packages.nix

When I define a mqtt sensor in my nixos config, will the nixos module be able to do the conversion from sensor.mqtt to mqtt.sensor (I don't fully understand the code in the nixos module, especially the useComponentPlatform code) ?

[dotlambda]

When I define a mqtt sensor in my nixos config, will the nixos module be able to do the conversion from sensor.mqtt to mqtt.sensor (I don't fully understand the code in the nixos module, especially the useComponentPlatform code) ?

Not yet. I will implement that once I was able to solve the stack overflow issue.

[dotlambda]

@netixx The problem should be solved once you upgrade your system since 753f58d is now in the nixos-18.09 channel. Can you please try again?

[netixx]

I can confirm that I don't have the build error anymore, thank you !

However, the home-assistant environment is missing component dependencies.

Same as stated previously, home-assistant code has been refactored, but config is the same, which means that dependencies are not found.

For example, I have lifx.light declared in config, but dependencies are at the light.lifx key.

[dotlambda]

Same as stated previously, home-assistant code has been refactored, but config is the same, which means that dependencies are not found.

Adjusting the code for useComponentPlatform is on my TODO list. For now, please override the package's extraPackages.

[netixx]

Ok, at the moment, I am using manual config because I am testing home-assistant, using customize.yaml and the automation editor, so I have manually set the extraPackages.

Do you think we could add a home-assistant package (something like home-assistant-all) which would have all possible dependencies (the idea being that I would also be built by hydra) - a little like the docker/distro image that home-assistant.io distributes ?

[dotlambda]

Do you think we could add a home-assistant package (something like home-assistant-all) which would have all possible dependencies (the idea being that I would also be built by hydra) - a little like the docker/distro image that home-assistant.io distributes ?

Sure, no problem. However, I suscpect that will often not build due to some Python packages not building. Do you want to make a PR or should I do it?

[netixx]

I can surely try, but I am not sure how I should proceed :

1. should I define a new option for home-assistant package that populates extraPackages to all packages referenced in component-packages.nix
2. or should I call the home-assistant package with all packages referenced in component-packages.nix (in a new file next to home-assistant/default.nix), and call it in a new callPackage in all-packages.nix.

Which option do you think is best ?

[dotlambda]

I guess the first option comes close to the best one. However, please don't use extraPackages for that but directly add the packages to propagatedBuildInputs. Maybe we could also add a few lines to parse-requirements.py to add a list of all packages to component-packages.nix so that we don't go through all components to generate that list.