Mi Flora Sensor gattool not found

[danieljkemp]

Make sure you are running the latest version of Home Assistant before reporting an issue.

You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:

Home Assistant release (hass --version):
0.29-dev

Python release (python3 --version):

Component/platform:
mi flora

Description of problem:
HA fails to read from sensor, errors indicate executable gattool is missing. This tool is part of Bluez on debian, but not centos/RHEL.

Expected:
No errors. Works on my raspi2.

Edit: Looks like the EPEL package dropped gatttool. Not sure why. Simply digging up a old RPM, pulling out the required executable and throwing it where expected worked to get the sensor online. This would not be a recommended fix. I don't know if there is an ideal fix here other than mark the issue in the docs that all distros may not be supported.

Problem-relevant configuration.yaml entries and steps to reproduce:

sensor:
  - platform: miflora
    mac: XX:XX:XX:XX:XX:XX
    name: YYYY
    force_update: false
    median: 3
    monitored_conditions:
     - moisture
     - light
     - temperature
     - conductivity

1. Run miflora sensor on RHEL-based linux.
2. Observe errors

Traceback (if applicable):

/bin/sh: gatttool: command not found
16-09-19 22:57:30 INFO (ThreadPool Worker 5) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s] XX:XX:XX:XX:XX:XX

Additional info:

[lwis]

I didn't like this when I saw it, @open-homeautomation could you not use pygatt?

[ghost]

I could, but I decided not to. pygatt is just a wrapper around gatttool. Therefore without gatttool, pygatt also won't work.
Using external Bluetooth libraries had already been a problem with the Bluetooth LE presence detection, therefore I decided to call gatttool directly instead of using an external library that calls it.

[lwis]

@Danielhiversen can you run sudo find / -name gatttool

[Danielhiversen]

I do not have the Mi Flora sensors yet.

[ggravlingen]

I had the miflora-library working on my rpi1. Did an upgrade to a rpi3 and experienced the same problems with gatttool. Was able to solve them eventually though. In solving the problem, I did many things and unfortunately have not understood what exactly solved the issue. Two of the things I remember doing are, however:

• A manual reinstall of the latest bluez-library (not through apt-get install bluez)
• Starting the bluetooth service in experimental mode.

I've got three miflora sensors running on 0.28.2 with the miflora-sensor manually downloaded from the 0.29 dev repository. I haven't exactly done a kosher install so I'm waiting for 0.29 before drawing any conclusions on functionality.

[lwis]

@Danielhiversen sorry buddy, meant @danieljkemp

[danieljkemp]

Doing a find / -name gatttool was one of my first steps, no results, as we'll see why. In the latest EPEL bluez rpm gatttool is missing as seen here:

http://rpm.pbone.net/index.php3/stat/6/idpl/31977313/dir/centos_7/com/bluez-5.23-4.el7.x86_64.rpm

vs a previous version of bluez:

http://rpm.pbone.net/index.php3/stat/6/idpl/29070354/dir/centos_7/com/bluez-4.101-13.el7.x86_64.rpm

It is not provided by any other packages.

@ggravlingen My guess is that your manual install of bluez is what did the trick.

Edit: After hearing about issues on other distros, I just recommend a blurb on installing bluez from source in the case that gatttool is missing.

[lwis]

@danieljkemp could you add something to https://github.com/home-assistant/home-assistant.github.io/blob/next/source/_components/sensor.miflora.markdown to close this off?

[Spartan-II-117]

Has this been resolved?

[knowhy]

gattool is deprecated in the most recent bluez versions.

commit b1eb2c4cd057624312e0412f6c4be000f7fc3617
Author: Luiz Augusto von Dentz luiz.von.dentz@intel.com
Date: Wed Jan 4 14:23:58 2017 +0200

build: Hide deprecated tools under --enable-deprecated

This marks the following tools as deprecated as they are not longer
maintained or have been replaced by other tools:

 hciattach hciconfig hcitool hcidump rfcomm sdptool ciptool gatttool

bluez.git

There is an open issue in the pygatt repository.

[atavius]

@open-homeautomation So what is the solution here?

[ciotlosm]

I'm getting the same problem using hass.io:

/bin/sh: gatttool: not found
2017-06-22 23:02:06 INFO (SyncWorker_8) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s] 

Any solution?

[JohanLeirnes]

@open-homeautomation using pygatt would make this component able to run on windows and mac using a Bluegiga USB bluetooth dongle. This would make this component usable to a wider range of users.

[lwis]

@bhaap basnijholt/miflora#23 uses bluepy. pygatt only wraps gatttool itself, so how is it able to avoid platform incompatability?

[JohanLeirnes]

Using a Bluegiga's BGAPI compatible dongle would enable you to run pygatt on any system. Its either a lib for gatttool or for BGAPI.

[knowhy]

I had a lot of trouble compiling bluepy on a Linux ARM system. Looking at the bluepy-helper code it seems like this based on an forked bluez4 codebase. So I don't think bluepy is a good choice in the long term.

From my research bluez has replaced gatttool with a D-bus api. I would love to see a solution utilizing the bluez5 d-bus api. That would of course not avoid platform incompatibility. From my very limited knowledge I don't think that this is an achievable goal at all.

[JohanLeirnes]

I have a created a fork that uses pygatt that works for BGAPI adapters. This should make the code more universal.

https://github.com/bhaap/miflora

I agree with you that Gatttool is not a good choice in the long term. D-bus api and BGAPI based code would be the best atm in both compability and in long term support.

[ViperRNMC]

any idea how to get this working with a Mac mini?

[hifiberry]

This won't work on MacOS.

[ViperRNMC]

when I use the tools from apple to scan blue devices is see the mi flora in the list, so maybe not with this code

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[ChristianKuehnel]

Added this to the issue list in miflora: basnijholt/miflora#50

Since we can now support different bluetooth backends, it should be quite easy to implement a version with pygatt. Unfortunately I do not have a BGAPI compatible Bluetooth device.

[tehbrd]

This is still an issue.

Platform is hass 0.56.2 on hass.io

What I noted was that the docker instance where HA was operating was attempting to use the interface (hci0) before it was properly initialised.

017-11-01 19:59:48 INFO (SyncWorker_15) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s] xx

As that container didnt contain hci* binary tools, I was able to locate them in the host OS and properly initialise the interface at runtime.

root@hassio:~# hcitool dev
Devices:
root@hassio:~# hciconfig hci0 up
root@hassio:~# hcitool dev
Devices:
        hci0    Xx

That said, I appear to be encountering locking issues with a second sensor never able to poll.

2017-11-01 20:04:27 DEBUG (SyncWorker_17) [homeassistant.components.sensor.miflora] Polling data for Plant1 Light intensity
2017-11-01 20:04:27 DEBUG (SyncWorker_17) [miflora.miflora_poller] Using cache (0:06:07.785415 < 0:20:00)
2017-11-01 20:04:27 DEBUG (SyncWorker_17) [homeassistant.components.sensor.miflora] Plant1 Light intensity = 87
2017-11-01 20:04:27 DEBUG (SyncWorker_17) [homeassistant.components.sensor.miflora] Data collected: [87, 87, 87, 87]
2017-11-01 20:04:27 DEBUG (SyncWorker_17) [homeassistant.components.sensor.miflora] Median is: 87
2017-11-01 20:04:27 DEBUG (SyncWorker_2) [homeassistant.components.sensor.miflora] Polling data for Plant2 Light intensity
2017-11-01 20:04:27 DEBUG (SyncWorker_2) [miflora.miflora_poller] Using cache (0:15:37.703623 < 0:20:00)
2017-11-01 20:04:27 INFO (SyncWorker_2) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s] X2
2017-11-01 20:04:27 DEBUG (SyncWorker_7) [homeassistant.components.sensor.miflora] Polling data for Plant2 Moisture
2017-11-01 20:04:27 DEBUG (SyncWorker_7) [miflora.miflora_poller] Using cache (0:15:37.717738 < 0:20:00)
2017-11-01 20:04:27 INFO (SyncWorker_7) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s] X2

[ChristianKuehnel]

Hey @bhaap,

would you be interested in upstreaming your integration of the pygatt in https://github.com/open-homeautomation/miflora? We had a major refactoring and can now support different bluetooth backends.

[JohanLeirnes]

It can be seen here: JohanLeirnes/miflora@6658309
I cannot take credit thou, its thanks to the user rileyhome @ homeassistant community forum.

I dont have the time to do the coding but feel free to check out my implementation and use that to develop yours.
My code is buggy thou. If asking for data from a device and it gets no asnwer the code crashes.
I ended up not developing it anymore because i moved to a linux server for my miflora and then your implementation via gattool works.

[ChristianKuehnel]

Hey @tehbrd

the current version of HASS.io is installing the bluetooth package that should include gatttool: https://github.com/home-assistant/home-assistant/blob/dev/virtualization/Docker/setup_docker_prereqs

Do you still have this issue?

[tehbrd]

I didn't have a material issue as once the interface was bought up in the host os everything worked correctly. Now its all dead as of 60.0, need to figure out what's happened.

[ChristianKuehnel]

Hey @tehbrd,

my first guess: your OS does not have "gatttool" any more. This used to be part of the bluez package and is now deprecated. Can you please check if you can run gatttool from the command line?

So did you change anything in your OS?

You said you're running hass.io. So looking at the docker files from Homeassistant it looks like we're using Debian jessie and installing the "bluetooth" package. And that package should still contain gatttool.

[tehbrd]

This isn't gattool related, never has been. More in relation to getting miflora running on a vanilla build.

hass.io has introduced another issue with the driver during 1.0 -> 1.1

bcm43xx_init
Initialization timed out.

Can't invoke the device anymore, hence the issues...

[tehbrd]

so it looks like my hass install is picking up the hardware correctly, but now it appears to have the exact gattool issue as described. Strange as this was previously a working system until I upgraded (aside from having to bring up hci0).

I don't know how these have gone from being available to gone...

INFO (SyncWorker_3) [homeassistant.components.sensor.miflora] Polling error [Errno Could not read data from Mi Flora sensor %s]
root@hassio:/var/log# docker exec -it homeassistant bash
bash-4.4# find / -name miflora*
/usr/lib/python3.6/site-packages/miflora-0.1.16-py3.6.egg-info
/usr/lib/python3.6/site-packages/miflora
/usr/lib/python3.6/site-packages/miflora/miflora_poller.py
/usr/lib/python3.6/site-packages/miflora/__pycache__/miflora_poller.cpython-36.pyc
/usr/lib/python3.6/site-packages/homeassistant/components/sensor/miflora.py
/usr/lib/python3.6/site-packages/homeassistant/components/sensor/__pycache__/miflora.cpython-36.pyc
bash-4.4# find / -name gatttool
bash-4.44# grep cmd /usr/lib/python3.6/site-packages/miflora/miflora_poller.py
        cmd = "gatttool --device={} --char-write-req -a {} -n {} --adapter={}".format(mac,
            with Popen(cmd,
        cmd = "gatttool --device={} --char-read -a {} --adapter={}".format(mac,
            with Popen(cmd,
bash-4.4#

[ChristianKuehnel]

Hey @tehbrd ,

I just did a clean installation of hassio on my RaspberryPi 2 with a Logilink USB-Bluetooth adapter and I can reproduce the problem there.

The log file contains:

2017-12-24 10:06:46 WARNING (MainThread) [homeassistant.helpers.entity] Update of sensor.<mysensorname> is taking over 10 seconds
/bin/sh: gatttool: not found
/bin/sh: gatttool: not found

So this is as I expected: The binaries of gatttool are not installed there. So the next step is: figure out to get gatttool installed by default. I'll have a look at this later.

[ChristianKuehnel]

The further analysis reveals:

• Hassio is using Alpine Linux 3.7 to as OS for Home Assistant.
• Alpine 3.7 seems to have moved the gatttool to a different package now, which is not installed by default: bluez-deprecated

A quick workaround (until you update HomeAssistant the next time):

1. log into your hassio host using ssh: https://home-assistant.io/developers/hassio/debugging/
2. log into your "homeassistant" docker container: docker exec -it homeassistant sh
3. in your docker container execute: apk update ; apk add bluez-deprecated
4. And with the next update cycle, your sensors should be visible again.

And it looks like this is already fixed in the next release:
home-assistant/hassio-build@d76b24b

➡️ Ticket can be closed