MySQL recorder stops recording after some time

[simonszu]

Home Assistant release (hass --version):
0.52.1

Python release (python3 --version):
Python 3.5.3

Component/platform:
Recorder

Description of problem:
The recorder stops recording new events after some time running. I have the recorder component with MySQL (mysqlclient pip module) and MariaDB as a backend. For example, this run, i started Home Assistant on 2017-08-30 11:32:39 according to the data in the schema_runs table, and the last entry is on 2017-08-30 17:11:33 (in the events table as well as in the states table). There is no associated entry in the log file, but when i stop Home Assistant and start it again, i get the following log entry:
2017-08-31 10:45:42 WARNING (Recorder) [homeassistant.components.recorder] Ended unfinished session (id=2 from 2017-08-30 11:32:39) Now the recording starts again until it hangs sometimes again, until i restart HA again...and so on.

Expected:
The recorder component recording stuff as long as Home Assistant is running.

Problem-relevant configuration.yaml entries and steps to reproduce:

recorder:
  purge_days: 30
  db_url: mysql://homeassistant:mysupersecretpassword@127.0.0.1/homeassistant

1. Install Home Assistant
2. Install MariaDB and create database and user
3. Install mysqlclient pip module
4. Config HA and start it
5. Wait...after some time there are no new log entries written any more

Additional info:

I am unsure if it has something to do with the recent database changes. I have completely dropped the database in between because i thought there could be some failed migration. But also with a fresh database (and even with a complete reinstall of MariaDB) this problem occurs as well. Haven't tested it with SQlite, though.

[cmsimike]

I have seen this before when the Amcrest component was breaking the recorder:

2017-08-31 10:45:42 WARNING (Recorder) [homeassistant.components.recorder] Ended unfinished session (id=2 from 2017-08-30 11:32:39)

What this means is that something threw an exception in the middle of a recorder run and that session/transaction was never completed or rolled back or something. Typically restarting hass will fix it.

I feel like this is more an issue of the recorder not being able to come back from a problem. Yes the root of this specific issue might be another component failing or otherwise taking the recorder down, but I feel like the recorder needs to better deal with errors and close/rollback sessions on error.

[simonszu]

I have tested it with SQlite as a backend, and it seems that this issue is backend-unaware, since the recording stops here after some hours as well.

@cmsimike Since you mentioned another component failing or something, i noticed that on both recorder stop events the last event was an automation which should automatically check me in into my flat on Foursquare when i arrive back home. This checkin was successful, but i remember Foursquare announcing some new API restrictions (since i am hitting their API once or twice a day i am not affected with this), but maybe there is a correlation.

Although i see no errors on the log file, i will casually mention @robbiet480 since he committed the current version of the Foursquare component...

[mericano1]

I am having the same problem. After a few days it just stops recording events. It seems to first stop some types (temperature) and after a bit it drops everything.

[mericano1]

I forgot to mention that this happened after upgrading to some version above 0.5, not sure exactly when.

[alpinewinter]

I have the same or similar issue in 0.54

2017-10-07 09:23:02 ERROR (Recorder) [homeassistant.components.recorder.util] Error executing query: Object of type 'function' is not JSON serializable

Interestingly, the timestamp coincides with a failed attempt to get Wunderground data:

2017-10-07 09:23:02 ERROR (SyncWorker_6) [homeassistant.components.sensor.wunderground] Check WUnderground API ('this key does not exist',)

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[balloobbot]

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem 👍