Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Recurrent error with 'sensor_history.get' MQ request #542

Open
vdomos opened this issue Oct 25, 2017 · 6 comments
Open

Recurrent error with 'sensor_history.get' MQ request #542

vdomos opened this issue Oct 25, 2017 · 6 comments
Labels
Need Info
Milestone
0.6.0 - technical...

Comments

@vdomos
Copy link
Contributor

vdomos commented Oct 25, 2017

In my plugin flowmeter, I use 'sensor_history.get' MQ request to calculate sum per interval every 5mn
I have a recurent error with MQ in core_admin.log

my call in plugin is like this:

    mq_client = MQSyncReq(self.zmq)
    msg = MQMessage()
    msg.set_action('sensor_history.get')
    msg.add_data('mode', 'filter')          # Like REST functions sensorHistory_from_filter and sensorHistory_from_to_filter
    msg.add_data('sensor_id', id) 
    msg.add_data('from', ts)                # 
    #msg.add_data('to', 1500847199)         # Default now
    msg.add_data('interval', interval)      # 'minute|hour|day|week|month|year'
    msg.add_data('selector', selector)      # 'min|max|avg|sum'

Same error with avg selector.

core_admin.log:

2017-10-25 04:25:44,008 domogik-admin DEBUG Send client status : alive
2017-10-25 04:25:56,989 domogik-admin DEBUG MQ Request received : <MQMessage(action=sensor_history.get, data='OrderedDict([(u'selector', u'sum'), (u'sensor_id', 107), (u'from', 1508896800), (u'mode', u'filter'), (u'interval', u'hour')])')>
2017-10-25 04:25:57,179 domogik-admin DEBUG ### mdp_reply_sensor_history: <MQMessage(action=sensor_history.result, data='{'status': True, 'reason': '', 'sensor_id': 107, 'values': {'values': [(2017, 10, 43, 25, 4, 0.0)], 'global_values': {'max': 0.0, 'sum': 0.0, 'avg': 0.0, 'min': 0.0}}, 'mode': 'filter'}')>
2017-10-25 04:25:57,279 domogik-admin DEBUG MQ Request received : <MQMessage(action=sensor_history.get, data='OrderedDict([(u'selector', u'sum'), (u'sensor_id', 107), (u'from', 1508882400), (u'mode', u'filter'), (u'interval', u'day')])')>
2017-10-25 04:25:57,482 domogik-admin DEBUG ### mdp_reply_sensor_history: <MQMessage(action=sensor_history.result, data='{'status': True, 'reason': '', 'sensor_id': 107, 'values': {'values': [(2017, 10, 43, 25, 11.0)], 'global_values': {'max': 4.5, 'sum': 11.0, 'avg': 0.3055555555555556, 'min': 0.0}}, 'mode': 'filter'}')>
2017-10-25 04:25:57,913 domogik-admin DEBUG MQ Request received : <MQMessage(action=sensor_history.get, data='OrderedDict([(u'selector', u'sum'), (u'sensor_id', 107), (u'from', 1506808800), (u'mode', u'filter'), (u'interval', u'month')])')>
2017-10-25 04:26:02,996 domogik-admin DEBUG ### mdp_reply_sensor_history: <MQMessage(action=sensor_history.result, data='{'status': True, 'reason': '', 'sensor_id': 107, 'values': {'values': [(2017, 10, 9844.5)], 'global_values': {'max': 125.0, 'sum': 9844.5, 'avg': 1.419129306616693, 'min': 0.0}}, 'mode': 'filter'}')>
2017-10-25 04:26:03,198 domogik-admin DEBUG MQ Request received : <MQMessage(action=sensor_history.get, data='OrderedDict([(u'selector', u'sum'), (u'sensor_id', 107), (u'from', 1483225200), (u'mode', u'filter'), (u'interval', u'year')])')>
2017-10-25 04:26:05,091 domogik-admin DEBUG ### mdp_reply_sensor_history: <MQMessage(action=sensor_history.result, data='{'status': True, 'reason': '', 'sensor_id': 107, 'values': {'values': [(2017, 96736.25)], 'global_values': {'max': 125.0, 'sum': 96736.25, 'avg': 0.302711333496054, 'min': 0.0}}, 'mode': 'filter'}')>
2017-10-25 04:26:05,092 domogik-admin ERROR Error while processing thread request. Message is : <MQMessage(action=sensor_history.get, data='OrderedDict([(u'selector', u'sum'), (u'sensor_id', 107), (u'from', 1483225200), (u'mode', u'filter'), (u'interval', u'year')])')>. Error is : Traceback (most recent call last):
  File "/home/dan/dmg/domogik/src/domogik/bin/admin.py", line 508, in __on_mdp_request
    self._mdp_reply_sensor_history(msg, db)
  File "/home/dan/dmg/domogik/src/domogik/bin/admin.py", line 1229, in _mdp_reply_sensor_history
    self.reply(msg.get())
  File "/usr/local/lib/python2.7/dist-packages/domogikmq/reqrep/worker.py", line 178, in reply
    to_send.extend(msg)
AttributeError: 'NoneType' object has no attribute 'extend'

In the plugin there are 3 devices * 4 call of this request. I catch this error in the plugin log.

Here are the errors reported from midnight to 4h20:

2017-10-25 00:01:25,665 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:07:10,949 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:12:31,274 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:18:01,545 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:23:48,699 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:24:03,717 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:34:49,511 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:40:15,700 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:51:18,713 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:56:54,255 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 00:57:09,277 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:02:26,729 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:02:50,094 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:03:05,102 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:08:36,990 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:14:05,133 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:14:22,639 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:19:51,537 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:20:09,754 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:25:29,737 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:25:56,876 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:31:36,753 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:37:07,761 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:37:24,179 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:37:39,212 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:43:06,771 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:48:26,246 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:48:41,260 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:49:02,117 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 01:49:17,139 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:00:11,796 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:05:45,309 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:11:01,812 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:11:27,203 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:11:42,212 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:16:58,593 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:22:38,811 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:22:54,369 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:23:15,716 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:28:36,825 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:39:31,831 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:39:54,587 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:45:27,186 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:45:42,205 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:56:29,846 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 02:56:44,865 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:02:01,111 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:02:26,717 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:07:54,758 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:08:12,052 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:13:47,857 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:24:35,346 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:24:50,361 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:30:23,633 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:30:38,651 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:35:57,878 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:46:45,885 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:52:15,802 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:57:47,682 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 03:58:13,882 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:03:29,895 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:03:55,783 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:04:10,803 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:09:27,155 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:09:50,881 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:10:05,899 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory
2017-10-25 04:20:41,673 domogik-flowmeter ERROR ### AttributeError for get FilterSensorHistory

this error crashes the other MQ requests if at the same time

I could not find the reason
@vdomos
Copy link
Contributor Author

vdomos commented Oct 28, 2017

The errors occur mostly with the query on the month and the year

@Nico0084
Copy link
Contributor

Nico0084 commented Nov 5, 2017

Hi
Possible issue from timing too short between two request.
Try #539 to optimise db
And increase timing request.

@vdomos
Copy link
Contributor Author

vdomos commented Nov 6, 2017

thank you, I will test these improvements:

ALTER TABLE core_sensor_history ADD INDEX (sensor_id, date);
OPTIMIZE TABLE core_sensor_history;

I deleted several devices and manually injected their data. This may be a cause of the issue.

@Cereal2nd
Copy link
Contributor

Does this solve your issue?

@vdomos
Copy link
Contributor Author

vdomos commented Nov 12, 2017

No, still have this error.
but It's a little better, the error appears especially for year query

What I modified:

  • ALTER TABLE core_sensor_history ADD INDEX (sensor_id, date);
  • OPTIMIZE TABLE core_sensor_history;
  • increase MQ get timeout to 30s !
  • add 2s delay between two request

for information, if I use REST for the same requests in my plugin, I have no problem
Not a database issue, no ?

@Cereal2nd
Copy link
Contributor

hmm, this is as expected

for these commands rest is going directly to the database, so there is no MQ in the background, so its normal that you don't see this in a rest request.

To be honest i don't think we can really fix this, its sql thats to slow to return the data.
The only solution i see is failing the mq request if it takes to long

@Nico0084 Nico0084 mentioned this issue May 23, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Need Info
Projects
None yet
Milestone
0.6.0 - technical improvments
Development

No branches or pull requests
4 participants
@vdomos @Cereal2nd @fritz-smh @Nico0084