Logic for combining device_trackers for PERSON component

[arsaboo]

Since the person component is now ready, it is now a good time to think about how should multiple device_trackers entities be combined. Based on the multiple threads on the forums and my own experience playing with 5 different device_trackers, here is the approach that I use that gives me 100% reliable presence detection. The logic relies on two assumptions:

1. Use most recent information wherever possible (see exceptions below)
2. GPS trackers are generally more accurate than wifi or BT trackers

So, here's one approach to combine different trackers:

1. Any new information from a GPS tracker should be updated in the person component
2. Wifi and BT trackers don't scan reliably (e.g., Google Home tracker does not even work on iOS) or have other issues (e.g., wifi sleeping), we should not act upon them immediately to mark not_home. They are far more reliable for marking home.
3. So, for home to not_home, GPS trackers will typically update first and they should be used to update person. Wifi/BT trackers should be acted upon with some delay to avoid false alarms, i.e., they should be not_home for some time before we mark person as not_home. This can be taken care using a slightly larger value for consider_home (about 10 minutes seems to work fine). Thus, in most cases, not_home will be marked based on GPS based trackers. Worst case scenario, if none of the GPS trackers are working, person will be marked not_home after the consider_home interval.
4. For not_home to home, we can use any tracker that updates first to set the state. Since wifi/BT trackers are quite reliable in this case, we don't need to delay them.

We can probably provide a config option to reverse the logic if someone wants to prioritize Wifi/BT trackers over GPS trackers (not preferred IMHO).

[MartinHjelmare]

I'd like us to avoid config options if at all possible. Besides that, I think the above sounds good. But let's hear what others think.

[arsaboo]

I agree...we can leave the config options out as I don't think we should ever prioritize wifi/BT over updated GPS.

[gorynychzmey]

As I can understand, consider_home affects only change of state from hometo not_home. If it really so, I think, that in multi-tracker configuration consider_home must be set as low as possible. In this case home from stationary tracker is reliable and must be prioritized. When no any stationary tracker shows home, then GPS must be prioritized.

[arsaboo]

We don't have to prioritize a particular tracker, we are prioritizing based on what provides the most recent information. Presence is the fundamental aspect of home automation and I am opposed to the idea of prioritizing any particular tracker as it can mess things up if it misbehaves. That is the reason I have 5 trackers :)

In your example, for home, wifi trackers will automatically be used as we are not delaying them (due to consider_home). Thus, it handles your use case as well.

[jjlawren]

In my experience with the Unifi presence tracker, it's very accurate for determining when a device arrives at home. However, the device timeout on the Unifi controller (external to HA) is long and not obviously configurable, leaving devices marked as home far after other trackers have updated.

[gorynychzmey]

Situation with Unifi is almost alike to situation with consider_home. This makes stationary trackers reliable to detection of arriving home but not being home. Relying on GPS gives us another problem - false detections of not_home in case of fluctuations of GPS (my example with Google Maps). I don't see ideal solution without additional parameters to config of Person.

[Chris-V]

I can see two things people may want here: presets and advanced configuration. Presets are nice, but there will be people with very specific needs not covered by them. With the advanced configuration scenario, I can see a per person, per device tracker setting. This would require an alternate schema for the device_tracker list though. Advanced config and presets can even be mutually exclusive to help keep things simple.

When more likely configurations are figured out, then I think would be time to pre-configure things with presets.

ie (added some random keys for demonstration purposes):

person:
  - id: me
    name: Me
    device_trackers:
      - entity_id: device_tracker.track_me_1
        consider_not_home: 5
        consider_home: 30
        priority: 2
        force_if_not_home: true
      - entity_id: device_tracker.track_me_2

[arsaboo]

@Chris-V While it will be great to have such fine-grained control, we should weigh it against the complexities and move gradually towards such options. As a first step, we should try things out without any config options as @MartinHjelmare suggested.

[MartinHjelmare]

Something to consider, though, is that we are talking about moving or remaking the consider_home option from the device tracker component as part of that refactor. Maybe it has a place in the person component?

I still think we should be careful before adding config options, but it's good to consider the big picture and changes we are planning.

[arsaboo]

If we move all the "intelligence" to the person component such that the device_tracker just collects the information and passes it on to person, we can include options like consider_home and consider_not_home in person component. Given that person will be the primary presence detection (including room detection at some point), I am sure we will need config options here. In which case, it may not be a bad idea to move them to person during the refactor.

[rohankapoorcom]

I'm not sure that makes sense in all situations. It makes sense when all you are trying to keep track are people but what about object detection or animal detection?

For example, I could have multiple device trackers in my car that I may or may not want to combine. In that scenario I'd certainly want the ability to adjust consider_home and consider_not_home depending on the tracker but I wouldn't want to create a person entity for that.

The same with a dog for example, if I had a smart dog door and tracking collar, I'd want to customize consider_home and consider_not_home but it would feel odd to set up a person entity, especially since a dog would never login to Home Assistant.

[gorynychzmey]

Currently Person combines two functions - trackable entity and (potential) subject of authentication. Maybe it worth to split this functions and make Trackable as independent entity with Person as subclass? Or make metatracker, containing all logic of combining device trackers, as a platform for device_tracker?

[balloob]

I'm glad we are getting this discussion going.

The config example by @Chris-V is great, but out of scope for now, and we should not further discuss that in the current issue. Our config format is capable of expanding to it in the future, if necessary.

@rohankapoorcom I think that we should focus on tracking humans for now, and so we should not consider supporting alternative tracking approaches for cars/dogs. I think that we can consider expanding scope or add a new component to track objects/animals/etc in the future. Also note, you can create persons that are not connected to a user, so you could still track your dog without giving them login credentials.

Things like consider_home should be deprecated from device tracker and be added to the person component. The device tracker should, just like all our other integrations, present the data as-is as we receive from the source. An entity is either a raw data representation, or a synthetic/processed representation of various raw data entities. (video explaining some more info about the difference)

May I propose that when people suggest algorithms, let's describe how the algorithm should work in pseudo-code, and not sentences. That will make it easer to compare results.

Proposal based on what I have seen so far:

Person to adopt latest updated state for which any of these checks is true:

• Device Tracker is type wifi/bt and state is home
• Device Tracker is type wifi/bt, state is not_home and it has been this state for 5 minutes
• Device Tracker is type gps

Person to adopt battery level attribute of latest updated state that contains battery_level.

When a tracked device tracker state of the type wifi/bt changes to the state not_home, set an interval to update the person state again in 5 minutes.

[arsaboo]

That is an excellent summary. I am assuming 5 minutes should be sufficient time for wifi/bt devices to wakeup.

[philhawthorne]

Hi team,

So from my own personal use case point of view, the "Person" component is rather binary for my liking. I had the same issues with device_trackers in that a person could only be home, and not home.

Underneath the hood, I think that the current implementation of using device trackers for a person is fine. However, I would also like the ability to have this expanded, to perhaps binary sensors. After all, most of the status is just on/home, not home/off. For this reason, it would be good to allow a Bayesian sensor (or any other binary_sensor) to be able to mark a person as home or away. This way, people can use their own logic to dictate when they should be marked as home etc.

The reason I mention the bayesian sensor, is my current feeling towards the person entity. Right now, if my phone is connected to wifi and bluetooth, if I turn off my bluetooth, but my phone stays connected to the Wifi, I am marked as away, as that was the last entity to be updated. (I may be wrong here, if I am please accept my most humble apologies). For me, I would like the ability to "fine tune" the way in which the person component knows if I am home or away. Not based on a device tracker updating, which could change frequently if I move in and out of Bluetooth range.

Now what I would love to see in the person component, is an implementation of my Not so binary presence. I would love the person to have two kinds of states, home and not_home, which is just binary. Current logic. And also another state, where the person could be either Just Arrived, Just Left, or Extended Away. Bonus points if the rules and logic for this person could be defined against the person.

person:
  name: Phil
  states:
   Just Arrived: 
     state: home
     for:
       seconds: 0
   Home:
     state: home
     for:
       minutes: 15
    Just Left:
      state: not_home
      for:
        seconds: 0
    Away
      state: not_home
      for:
        minutes: 10
    extended_away:
      state: not_home
      for:
        hours: 24   

As mentioned I use this logic currently based on templates and dropdowns, which allows me to do some automations which trigger as a person gets home for the first time, and then move into another mode etc.

Just my 2c

[jjlawren]

Although 5 min seems like a sane timeout, some trackers update when leaving a zone much faster than that. Requiring a wifi/bt tracker to timeout first to not_home will limit the potential overall responsiveness.

Zone exits seem inherently unreliable for wifi/bt/ibeacon tracking methods. Perhaps they can be given top priority for arriving but relied on far less for remaining in or exiting the zone?

[helto4real]

My logic is quite simple.

If any non gps tracker is home, the person is home else set same state as gps tracker unless gps tracker did not report new data in an hour so we dont get stuck Home if we are away a long time.

This works most reliable of any logic I tried. I imolemented a appdaemon app to use it. In the Swedish community quite a few people use it and are happy with it.

Non gps trackers are normally reliable on home state but can be false positive in not_home state due to battery savings in mobiles. With this logic and use 3 trackers , gps, BT and WiFi(router) it is very acccurate.

Not so binary logic I also implement with just arrived and just left logic. I tend to do automations on when just arrived. I suggest we wait to implement the "not so binary" logic. Better to manage the device prio first. Let’s Make it stable before next step.

[gorynychzmey]

Currently state of device_tracker or person is spatial. In approach of 'not so binary logic' the state is spatial-temporal, what from my point of view makes things more complex. My proposal is to add attribute since to person, which will show time of last status change (opposite to last_updated, which shows last status check). With this attribute and Template Sensor it will be easy to implement any kind of 'not so binary logic'. Or we can make separate sensor type for this.

One more thought to tracker state combining: not only routers have time, when they doesn't update state after device is disconnected. GPS is also not immediate and makes update not every second. So we always have periods of time, when state of our sensors not reflect real things. I think that we should assume that we always have some delay between reality and it's representation in Home Assistant, but our task is simply to make this delay as low as possible. I believe that it can be done by manipulating with consider_home on device tracker level (because such kind of timeout mostly makes sense for device as a whole, not for every single entity) with possible addidtonal fine-tuning on the level of person's device tracker eitities). Something like:

device_tracker:
  - platform: traccar
  - platform: fritz
    consider_home: 60

person:
  - id: me
    name: Me
    device_trackers:
      - entity: device_tracker.track_me_traccar
        consider_home: 30
      - device_tracker.track_me_fritz

[DubhAd]

2. GPS trackers are generally more accurate than wifi or BT trackers

Except... I find the opposite. My GPS tracker will take 10 to 15 minutes (or so) to mark me as home or away.

That's why I'm using much the same approach as @helto4real (and also make use of much of @philhawthorne's awesome non-binary logic - except for my extended away logic being based on distance from home). As long as one of BT or WiFi shows me at home, they win over GPS. Indeed, GPS showing somebody at home, but WiFi and BT not probably means that they're at the neighbours.

[databoy2k]

As a non-coder, mere user, my feedback is simple: Please give us the ability to make our own decisions. We're using open-source software for a reason: flexibility. Let us establish our own rules, our own priorities, and then you guys guide us on the pros/cons of each option if we're getting it wrong. That's what the docs are supposed to be for.

What is the most intuitive way of doing that? IMHO, priority should be given in the order that the trackers or sensors are listed in the config. Don't second guess us. Doing so is most peoples' complaints about Google: your logic and AI don't always mesh with our expectations as users, and the decision that you (Devs) make in this thread have to be consistent for the rest of HA's life or until you have this debate again. A new device tracker is released that is killer accurate in 5 years from now, integrated immediately into HA, but it's not GPS-based so it falls to the end of the logic - now you guys have to update the logic and have this debate again. I set up my system to use Wifi with appropriate timeouts, notwithstanding that Google Maps integration is available? Let me override the GPS-heavy logic - I've built in the necessary delays in my automations already.

You guys are knocking it out of the park with this project. Don't over think it. Users can learn from tools that are too flexible. Users stop using tools that become inflexible.

[gorynychzmey]

@databoy2k, seems to be that your idea comes to simply iterating through device trackers in order, in which they're appearing in config, and searching for first tracker, which returns something different than 'not_home', right?

[jjlawren]

One more potential problem with prioritizing BT/router/etc tracker sources is when using non-home zones. Perhaps I have a GPS tracker that marks me at my Office zone. If something causes the router device_tracker entity to update (attribute change?) or HA is restarted, the person entity could be updated to not_home which will be technically correct but not the most accurate option.

[gorynychzmey]

@jjlawren, restart of HA is always the special case. After restart reliable status will be only atfer all (or almost all) device trackers got updates. GPS trackers usually make updates with bigger intervals to lower battery consumption and probably will be last.

[databoy2k]

@databoy2k, seems to be that your idea comes to simply iterating through device trackers in order, in which they're appearing in config, and searching for first tracker, which returns something different than 'not_home', right?

I think so, yes. Or prefer marked Zones to deal with @jjlawren's comment. I'm mostly responding to the suggestion that there needs to be an overarching logic that can't be configured; on the flip side, rather than creating some arcane set of rules that we might start cussing in 5 years time, I'm advocating for allocating complete manual control of priority to the users via config.

[gorynychzmey]

@databoy2k, I think that it quite good idea to make priority by config order. I want only to expand and evolve it.
So, I think, that it would be too complicated to config such order for each person. From my point of view it will be better to configure such things at component level and set order of platforms:

person:
  platform_priorities:
    - unifi_direct
    - buetooth_le_tracker   
    - google_maps
    - traccar

But this will require, that every device tracker entity should have attribute platform or maybe even every device tracker platform should have custom tag attribute, which will be propagated to entities on creation.

device_tracker:
  - platform: unifi_direct
    priority_tag: wifi
  - platform: buetooth_le_tracker
    priority_tag: ble
  - platform: google_maps
    priority_tag: gps
  - platform: traccar
    priority_tag: gps
person:
  priority:
    - ble
    - wifi   
    - gps

[databoy2k]

@databoy2k, I think that it quite good idea to make priority by config order. I want only to expand and evolve it.
So, I think, that it would be too complicated to config such order for each person. From my point of view it will be better to configure such things at component level and set order of platforms:

FWIW (and again, coming from strictly a user standpoint, and just one of the many at that) I don't think per-user is a bad idea or undesirable. I can think of lots of use cases where defining priorities based on the person, rather than globally, is a better alternative. Besides: you have to define trackers by person anyway, so the only use case for setting priorities by platforms is where you have multiple persons (perhaps significant numbers) and you're experimenting with the different orders. The cost of that is the loss of individuality; maybe GPS is the correct priority for my wife and my phones, but if I want the alarm system to shut off when the inlaws come in then Wifi is the best priority for them.

Again, I'm probably not the best guy to be engaging in this discussion too deeply as a non-dev, but when the decision gets made, I seriously hope that flexibility remains the guiding principle, even if it is at the expense of a few extra keystrokes. Better to have a truly flexible system that allows full user control, even if it means that the users have to think about the best priority order that works for their respective systems.

[balloob]

I just got triggered by a sentence in one of the above comments:

We're using open-source software for a reason: flexibility.

Our flexibility offering is that you can just copy our component into the custom component folder and make any changes that you want. By doing so, your flexibility does not come at the expense of offering a sane and usable defaults that will be enough for 90% of the users or at the expense of our developers time.

So right now, we should be discussing default behavior for person component. Going all in on per-person configuration is out of scope, as I mentioned in my initial response #163 (comment) . If we don't have sane defaults, we can't even start to offer config options.

As I haven't seen any other algorithm suggestions since mine at #163 (comment) , I suggest that we just implement that and consider this issue closed.

Small edit: this doesn't mean it's the final feature set, it just means that we have some default that works pretty ok out of the box.

[isabellaalstrom]

Flexibility doesn't just always come at the expense of "a few keystrokes". It can also cause a much more complex logic to make sure you keep undesired effects out and the worst side effect can be a less robust component with a lot of things that can go wrong. It can also make bug reporting and finding bugs a lot harder.

For me open source is not necessarily the same as a super flexible solution, and not even that everyone gets the thing they think is best, there are still developers that has to make the decisions of what's best for the project in terms of maintainability and robustness.

[isabellaalstrom]

@balloob I would suggest one addition to your logic above, that if you have multiple trackers of wifi or bt, that maybe at least two has to be not home for 5 min. Otherwise, and also without this suggestion, I think it sounds great and will be really useful.

[gorynychzmey]

As I haven't seen any other algorithm suggestions since mine at #163 (comment) , I suggest that we just implement that and consider this issue closed.

Small edit: this doesn't mean it's the final feature set, it just means that we have some default that works pretty ok out of the box.

I can only suggest to make check of wifi/ble is not_home for 5 minutes, only if no any gps in status zone. From my point of view all zones (except home) should have priority over not_home.
And maybe not constant 5 min but some consider_home parameter on component level of person.

[isabellaalstrom]

Yes, I agree that not_home probably should only be set by wifi or bt if the current state is home.

[helto4real]

@balloob I aggree with @isabellaalstrom, if we introduce a timeout it needs to be both or there will be some cases that could trigger false positives. I also like the @gorynychzmey with the zone logic. The Zone feature that @gorynychzmey suggested could be a different PR if we feel it needs to be implemented.

[Data-Monkey]

Person to adopt battery level attribute of latest updated state that contains battery_level.

I was a bit surprised by this statement. If we want to keep person a pure person thing, then why add a "random" battery level? A person does not have a battery level.
If we represent the latest updated tracker's battery, then it will just bounce around all the time.

What is the use-case for battery level on person?

[B-Kramer]

As above, can we allow binary sensors as data sources? It would allow the Bayesian sensor (and all that entails), as well as other sensors types (ultrasonic car sensors, in my case) to influence home/not_home. That might be a solution for those requiring more control, and avoid adding config options to this.

[tboyce021]

I thought the same thing as @Brianckramer. Why add all this logic to the person component? If we just had something like a "generic" or "template" device_tracker, we could implement whatever logic and combine whatever other sensors we want. Then there's really only need for having a single device_tracker attached to any person. Even with the current behavior, a "group" device_tracker could be the equivalent of what the person component does for presence.

Essentially, leave all the presence detection logic in the device_trackers and just tie one of them to a person for indicating presence of that person.

[balloob]

This can already be done today by using the device_tracker.see service and just linking a single device tracker to a person.

Instead of a binary sensor, we should add a device_tracker.template ?

That still should not stop us from adding sensible defaults to the person component that works for most people.

[Data-Monkey]

If we just had something like a "generic" or "template" device_tracker

I am using the composite tracker custom component (by @pnbruckner) to achieve exactly this.
https://github.com/pnbruckner/homeassistant-config/blob/master/docs/composite.md
"This platform creates a composite device tracker from one or more other device trackers and/or binary sensors."

My setup is not perfect yet, but I will just have 1 device tracker in person and manage all the other "fine tuning" elsewhere.

Basic defaults could work for some, and if the user needs more control a component like Composite might be for the more advanced user. Maybe we can get Composite into the HA code?

[tboyce021]

I'm not arguing against sensible defaults. I was just suggesting putting them in device_tracker instead of person, i.e. device_tracker.group similar to light.group. That would contain the sensible default behavior for people wanting to link more than one device_tracker. The person component doesn't need that logic and you would just link it to your device_tracker.group. For people wanting more control than that, yes, device_tracker.template would probably work.

[tboyce021]

Of course, that only works if the logic for device_tracker.group would match what the person component would expect, which may or may not be the case. Possibly a device_tracker.person_group or something could work in that case, but that may be a bit odd.

Either way, that was really just a suggestion. I would prefer to see something like the device_tracker.template to be able to use Bayesian sensor and what not as the source. The default behavior can live wherever makes sense.

[jjlawren]

As noted in home-assistant/core#22017, there's a current assumption here that 'stationary' trackers are always in the home zone. This is not true for (at least) stationary bluetooth beacons detected by a mobile device (ie, the iOS app).

The PR above is not meant to open the door for tracking mobile beacons. Perhaps documentation should explicitly reflect this.

[jjlawren]

To be more explicit about the last post and linked PR, the proposal should be updated to change:

Device Tracker is type wifi/bt and state is home

to

Device Tracker is type wifi/bt and state is not not_home

This should prioritize 'stationary' trackers if they're in any zone, not just home.

[gorynychzmey]

@jjlawren, in this case we must also add to stationary trackers an option to show specific zone as a place of presence instead of home. AFAIK currently no any stationary tracker does that.

[jjlawren]

AFAIK currently no any stationary tracker does that.

My example is for the HA iOS app and OwnTracks iBeacon detection methods with stationary beacons. Perhaps this concept only really applies to the bluetooth_le type. Or perhaps it would be useful to have a separate source_type attribute for mobile vs stationary beacons and a way to configure them as such.

[zeehio]

One of the problems of the device_trackers as they are is that to make accurate tracking estimations they require to make assumptions.

Example: "The device is not detected by nmap, therefore it is not at home"

Example: "The device is not detected by nmap because it is sleeping, so we add the consider_home option".

These assumptions are detrimental as they are because they add complexity to the tracker and more importantly because they make it more difficult to debug tracker issues (e.g. Is this tracker error caused by a physical malfunction or by a bug/missconfiguration in the assumption code?). Moving consider_home logic to the Person may be part of a solution to this.

Having a distinction between "physical" and "logical" trackers would make things easier. Physical trackers would not be allowed to make assumptions, and logical trackers would be built on top of other trackers to refine them.

• A tracker would provide at least: (device id, time when the device was last detected, last_location, time when the tracker last searched for that device).
• Physical trackers feed on hardware to provide outputs.
• Logical trackers can feed from other tracker outputs to provide their outputs.
• A Person is tied to the last_location of a single tracker.

Removing assumptions from physical trackers would render them less useful by themselves (e.g. the nmap tracker, operating on my home wifi, would never be able to tell that a device is not at home). But that is because the nmap tracker is not really able to do so without assumptions.

Having the assumptions separated from the physical tracker makes the output of the physical tracker more reliable, and gives more confidence to the user to rely on it when building his/her custom logic.

[jwelter1971]

Presence detection is tricky business! Lots of good info above. A few things to add:

1. With proper coverage beacons work very well. This is what they were designed for. They are not prone to sleeping effects, but coverage issues - you need a lot of beacons to reliably cover a typical house. For my usage beacons are by far the most reliable and lowest latency for home arrival/departure.

2. For those using iOS devices HomeKit adds a very reliable method to detect home arrivals and departures. The common way to get this info back into HomeAssistant is to use an input_boolean that gets toggled on arrival/departure. We should make sure we consider this in the sources above.

3. We can also use door sensors to give us hints and make device_trackers more reliable. For example if WiFi shows a device dropped from the network but no doors have opened/closed it's likely a false exit and due to coverage/sleeping device. Or you have teenagers climbing out windows ;)

4. It's also possible to use some active techniques - any iPhone or iPad can be actively queried if it's present if you have turned on iTunes Wifi synchronization. This you don't want to do at high frequency as it will drain your device battery quickly. But it is useful in some cases; for example if you sense a door trigger but not a device_Tracker state change you can actively query.

Lots to consider to really get this right.

[ties]

There is one small issue in the data model that I would like highlight - which is probably encountered by any device tracker using the see API call when you have multiple entities/observers detecting devices (including beacons).

Current situation:

• There is a single device (by mac address) shared by all observers
• The observed state depends on the ordering (ap1.disconnect -> ap2.connect -> online, ap2.connect, ap1.disconnect -> offline).

If i frame this as raw events and synthesized events (from tracker/physical tracker as coined above), I see these things:

• device: an entity with a unique identity, often with a mac address
• observer: an instance of a device tracker (possibly with a physical zone), with a type of signal it detects (mac/gps/...)
• observation: an observer detecting a certain state for a device, a raw event (possibly with a ttl), 1:N

A general device tracker can than track the state of the devices to create synthesized events that indicate the presence of an object.

I do not like that this is more complex, but this is the easiest way I see to fuse observations for a device from multiple observers (use cases: multiple access points, see and a device tracker via wifi for an iPhone). I do like that it separates the raw event more.

[frenck]

This architecture issue is old, stale, and possibly obsolete. Things changed a lot over the years. Additionally, we have been moving to discussions for these architectural discussions.

For that reason, I'm going to close this issue.

../Frenck