Skip to content

Add Crownstone integration#37005

Closed
RicArch97 wants to merge 12 commits into
home-assistant:devfrom
RicArch97:crownstone
Closed

Add Crownstone integration#37005
RicArch97 wants to merge 12 commits into
home-assistant:devfrom
RicArch97:crownstone

Conversation

@RicArch97
Copy link
Copy Markdown
Contributor

@RicArch97 RicArch97 commented Jun 22, 2020

Proposed change

New integration for Crownstone!

Crownstone creates smart crown stones to build into a power outlet, and plugs that plug into regular outlets/power strips.
A standout feature is the ability to detect a user's presence on room level by using BLE.

The integration exists of 3 core external libraries.

  • crownstone-uart (Adds the ability to use the Crownstone USB with Home Assistant. This dongle hooks directly into the Bluetooth mesh.)
  • crownstone-cloud (Pulls all data from our Cloud Service and adds switching/dimming commands)
  • crownstone-sse (Awesome server-send-events client to listen for changes in presence, and more to come!)

All of these libraries are available on PyPi.

Integration is configurable via the UI.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

The full Crownstone integration (so far) is now available on HACS for testing.
See Crownstone HACS repository.

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

@homeassistant
Copy link
Copy Markdown
Contributor

Hi @RicArch97,

It seems you haven't yet signed a CLA. Please do so here.

Once you do that we will be able to review and accept this pull request.

Thanks!

Copy link
Copy Markdown
Member

@frenck frenck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a quick scan of the PR to point out some early things.
The main missing part is the tests for the configuration flow, which we do require.

Furthermore, please limit the PR to a single platform (e.g., light or sensor). Additional platforms can be added later.

Comment thread .coveragerc Outdated
Comment thread homeassistant/components/crownstone/__init__.py Outdated
Comment thread homeassistant/components/crownstone/hub.py Outdated
Comment thread homeassistant/components/crownstone/translations/nl.json Outdated
@RicArch97
Copy link
Copy Markdown
Contributor Author

Changes were made after review. To sum up the changes:

  • Test for configuration flow added with 100% code coverage
  • Removed the sensor platform for this first PR (only light platform remains)
  • Added ability change event (belongs to light, that way users can turn dimming on/off without requiring a restart)
  • Removed the hass loop parameter in CrownstoneCloud
  • Removed additional translations

Comment thread homeassistant/components/crownstone/light.py Outdated
@RicArch97
Copy link
Copy Markdown
Contributor Author

New commit includes:

  • Not dynamically changing supported features. Since the light entities DO NOT implement an update method, all the data that gets loaded into the memory in the entry setup, is static, and will never change. Whenever an update event is received that an ability has been changed through the app, the config entry is reloaded, causing the entities to be removed and re-added.
  • Added assumed_state property to light, because there is no polling for state updates. This is because the state of Crownstones isn't always known/correct in the cloud.

@timmo001
Copy link
Copy Markdown
Member

timmo001 commented Sep 5, 2020

@RicArch97 Black has recently updated. Do a pip install --upgrade black and reformat your code, then the checks should fully pass 👍

@RicArch97
Copy link
Copy Markdown
Contributor Author

Black seems to be fine, but now checks are failing (that passed before) in files that either aren't mine, or i haven't changed anything to.

Copy link
Copy Markdown
Contributor

@epenet epenet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I suggest that you add some extra common keys to the config_flow strings, as discussed on #40578?
already_configured, password, username, cannot_connect are all available.

@RicArch97
Copy link
Copy Markdown
Contributor Author

RicArch97 commented Oct 4, 2020

Rebased on the lastest dev branch, after some failed checks on my previous commit.

Common keys have been added to the config flow strings.

@github-actions

This comment has been minimized.

@github-actions github-actions Bot added the stale label Jan 25, 2021
@RicArch97
Copy link
Copy Markdown
Contributor Author

Not stale. Crownstone integration has been further developed and updated here (HACS custom)

Comment thread CODEOWNERS
homeassistant/components/counter/* @fabaff
homeassistant/components/cover/* @home-assistant/core
homeassistant/components/cpuspeed/* @fabaff
homeassistant/components/crownstone/* @Crownstone @RicArch97
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you affiliated with Crownstone?

Are they aware they are set as a code owner?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started developing this integration as part of an internship for Crownstone. After it ended I decided to keep developing and maintaining the integration, but not as official member. This means Crownstone is the official owner of it, and they will take over maintenance in case I am unavailable.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are fine with it. ~CEO Crownstone.

def __init__(self) -> None:
"""Init with new event loop and instance."""
self.loop = asyncio.new_event_loop()
self.uart_instance = CrownstoneUart(self.loop)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is an async package, why are you running it in a new event loop?

Copy link
Copy Markdown
Contributor Author

@RicArch97 RicArch97 Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Uart lib is created by the Crownstone core members, with the idea to have it block until a USB is connected, so it works in a recursive way instead of creating tasks in the event loop to allow for other coroutines to be executed. If no USB is connected, it just keeps on scanning for USB dongles until it finds one. This way it would mean the USB could be connected and removed at any time, and the async_turn_on function uses the the USB whenever it's available, and falls back to the cloud if it is not.

We've decided to have it run in a separate thread to stop it from blocking the event loop.

I can request changes, if you deem it necessary.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will actually try to connect to every serial port it can find, to see if do a successful Crowdstone handshake.

https://github.com/crownstone/crownstone-lib-python-uart/blob/master/crownstone_uart/core/uart/UartManager.py#L90

This is very dangerous and can interfere with other equipment in use. Since this type of communication is not uncommon, I would say this is not a good method to have in Home Assistant.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Holy moly. Yeah that is 100% not allowed in HA. USB port need to be passed in.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible for the lib to take a port parameter and skip the handshake section. How do we ensure to provide the correct port though? Let's say I have the Crownstone dongle at /dev/ttyUSB0, but then i add an other USB dongle of some kind and restart HA, and the Crownstone dongle is now at /dev/ttyUSB1

Copy link
Copy Markdown
Member

@frenck frenck Jan 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By not using those ports. We advise our users to use the linked by-id paths in general to avoid those issues. However, that is a configuration problem, not a thing that integration should solve.

The ZHA integration might be worth a look. That one is showing a selection of devices/serial ports to use in the UI when setting up the integration.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another approach is to use a udev rule. In my case, I have the following line in a file in the folder /etc/udev/rules.d/:

SUBSYSTEM=="tty", ATTRS{idVendor}=="10c4", ATTRS{idProduct}=="ea60", SYMLINK+="crownstone"

This makes the usb dongle available at port /dev/crownstone

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hillstub That is a modification done on the system administration level. From a Python application perspective, that is not an option or requirement we can take into account. Let's keep this PR review on the topic of reviewing code at hand. Thanks 👍

self.sse.start()

# subscribe to Crownstone ability updates
self.sse.add_event_listener(EVENT_ABILITY_CHANGE_DIMMING, self.update_ability)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is SSE calling event listeners from the event loop or a thread?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are called from a thread.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then don't decorate with @callback and don't call any async function.

Copy link
Copy Markdown
Contributor Author

@RicArch97 RicArch97 Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it safe to initialize the SSE thread with the mainthread loop like so

self.sse = CrownstoneSSE(
      customer_email, customer_password, asyncio.get_running_loop()
)

And then in the SSE thread eventbus, provide coroutines to the loop by calling

asyncio.run_coroutine_threadsafe(coroutine, loop)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is a thread, why do you have to pass in the loop?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SSE lib has it's own eventbus. Which means any callbacks or coroutines added to it are executed within that thread (and the event loop running there). Let's say i want to run

await hass.async_create_task(coroutine)

in a listener in that thread. Do i need to pass it to the mainthread loop specifically or am I not understanding it correctly?

Comment thread homeassistant/components/crownstone/hub.py
"""Update the ability information."""
# make sure the sphere matches current.
update_sphere = self.cloud.spheres.find_by_id(ability_event.sphere_id)
if update_sphere.cloud_id == self.sphere.cloud_id:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use a guard clause to simplify the code.

if update_sphere.cloud_id != self.sphere.cloud_id:
    return

update_crownstone = self.sphere.crownstones.find_by_uid(
ability_event.unique_id
)
if update_crownstone is not None:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

            if update_crownstone is None:
                return

ability_event.ability_type
].is_enabled = ability_event.ability_enabled
# reload the config entry to process the change in supported features
self.hass.async_create_task(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

async_create_task can only be called from a callback inside the Home Assistant event loop.

entities = []
for crownstone in crownstone_hub.sphere.crownstones:
# some don't support light features
if crownstone.type not in CROWNSTONE_EXCLUDE:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more future proof to use an INCLUDE list.

"""Turn off this device via dongle or cloud."""
if self.uart.is_ready():
# switch using crownstone usb dongle
self.uart.switch_crownstone(self.unique_id, on=False)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is doing I/O inside the event loop

"Dimming is not enabled for this crownstone. Go to the crownstone app to enable it"
)
elif self.uart.is_ready():
self.uart.switch_crownstone(self.unique_id, on=True)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I/O in event loop

Comment on lines +17 to +19
},
"description": "Enter your email and password for Crownstone.",
"title": "Crownstone"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Title is filled in automatically based on name in manifest. This description is not needed.

Suggested change
},
"description": "Enter your email and password for Crownstone.",
"title": "Crownstone"
}

"title": "Crownstone",
"config": {
"abort": {
"already_configured": "[%key:common::config_flow::abort::single_instance_allowed%]"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is only a single instance allowed? I didn't see any code that required this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has to do with the Crownstone architecture. A config entry represents a "Sphere". A Sphere could be seen a house, or office. Since a single HA instance represents a house as well, it would not be logical to allow for multiple Spheres per HA instance.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's totally reasonable to be able to control my office space before I arrive at work or vice versa. We should not make that decision as developers. We should only limit to a single instance if a second instance is not necessary, like with Cast, which scans the network and find all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some difficulties when trying to allow for multiple spheres in a HA instance.

Crownstone communicates through a BLE mesh network and uses the smartphone of the user as gateway to provide the data to a cloud server; This means, if you want to switch a Crownstone that is in a different sphere, your smartphone would have to be physically there to be able to provide the data from to cloud to the BLE mesh, or in the other direction, send sensor data from the BLE mesh to the cloud server. A hub device is currently in development to help resolve that issue, but it is not ready yet.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, so those entities can become unavailable, just like what happens if the phone leaves the "sphere" now. That is unrelated to the number of spheres that can be configured.

)

async def async_step_sphere(self, user_input=None):
"""Handle the step for selecting a sphere."""
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should just add all spheres. Users can always disable devices later.

Comment on lines +109 to +110
CONF_EMAIL: self.login_info[CONF_EMAIL],
CONF_PASSWORD: self.login_info[CONF_PASSWORD],
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you're involved with Crownstone, can you ask them to update their authentication to use something like OAuth2? It's highly prefered over storing email/password. (not for this PR)

@RicArch97
Copy link
Copy Markdown
Contributor Author

This pull request is from some time ago, and is not completely in sync anymore with what is currently available on our HACS repository here. Would you recommend merging it first and then fixing according to review? (still limiting to 1 platform of course)

@github-actions github-actions Bot removed the stale label Jan 28, 2021
@balloob
Copy link
Copy Markdown
Member

balloob commented Jan 29, 2021

Let's get this PR fixed and merged, and we can go from there.

@balloob
Copy link
Copy Markdown
Member

balloob commented Mar 10, 2021

This PR seems to have gone stale. Closing it.

@balloob balloob closed this Mar 10, 2021
@mrquincle
Copy link
Copy Markdown

Hi @balloob. Do you want us to open a new PR?

We've been working on this steadily, following up your comments. It requires us to update our python code over the ecosystem in tandem though. My latest comment is from two days ago. :-)

@balloob
Copy link
Copy Markdown
Member

balloob commented Mar 10, 2021

I did see your latest comment which actually made me realize this PR was stale. The developer hasn't been active on this in 1.5 months.

Feel free to open a new PR once all comments have been addressed 👍

@RicArch97
Copy link
Copy Markdown
Contributor Author

The changes are still a WIP indeed, but the process is going a bit slow right now, as i have decided to refactor some code in the Python libraries. This way we start off with a very stable PR we can build upon.

@RicArch97 RicArch97 mentioned this pull request May 15, 2021
21 tasks
hahn-th pushed a commit to hahn-th/core that referenced this pull request May 2, 2025
I find it confusing to read only "If any of them do not return true, the automation will stop executing" (it's a double negation plus an "any" in the begining"). I've added an additional statement before
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants