New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MQTT in Node: Error stopping node: Close timed out #2934
Comments
Hi, I'm not able to reproduce this issue, so there may be something more specific with your environment that is contributing to it. I assume you have the broker running locally rather than using a remote broker? |
Are using mosquito 2.0? And is it on the same machine as node-red? |
The error is connected to 04a3c4b. It happens only with Node-RED v1.3.x and when a full deploy is performed, not an incremental one. |
Hi, I am the maintainer of node-red-contrib-aedes. This issue only happens with a broker as part of Node-RED. An external broker is fine.
As mentioned before, everything was fine until version 1.2.9. The issue only happens when Node-RED shuts down i.e. at close and at a full deploy. Is there anything I can do in order to make the broker close after the MQTT nodes? Or do I have to send a specific signal before close? No change for version 1.3.3. Any help is welcome. Martin |
Could the change #2880 be relevant this? |
Yes, this will be related to the change in order of stopping things. Previous, the flow/config nodes would be stopped in a somewhat random order - related to the order they were originally added to the workspace. Now, the flow nodes are stopped first and then the config nodes are stopped. This originally broke the 'on-close' message sending of the MQTT nodes as it was being handled by the close event of the config node - by which time all of the MQTT nodes would have already been stopped and the connection closed. So the sending of the message was moved to get triggered when the last MQTT node is closed. If the aedes broker is a flow node, it will getting stopped alongside the MQTT nodes. If the aedes broker node was added first, it will get stopped before all of the MQTT nodes - meaning the connection will be dropping in parallel to the MQTT nodes stopping and trying to send the final on-close message. To be honest, I'm not sure why you'd have an on-close message configured in the MQTT nodes if your broker is running inside Node-RED as well - the only time they are going to disconnect cleanly is when the flow is restarting or Node-RED is stopping, in which case the broker is going to be restarting/stopping as well. |
As far as I know, sending the on-close message is the standard behaviour of the MQTT config nodes. There is no additonal message configured. So, the problem is now that the MQTT config node expects an answer of a broker which is already down. Right now, I have no idea how to fix this issue on the broker side. |
Ah right. I thought this was related to the publishing of the This is just the node timing out when trying to disconnect.. which is indeed odd. I'll investigate. |
The message Nick is referring to in the last paragraph is the one configured in the "Messages" tab for the MQTT-in/out's config node that includes the broker connection details. These are the Birth/Close/LWT messages. It doesn't make much sense to have any (OK, may be the Birth message) of these messages if the broker is in the same process as the client as they will only fire when the broker is very likely to be in the process of restarting as well. |
having the same issue on RaspberryPi with: |
@emc02 Just to be clear, what broker are you using? |
I am using aedes broker and connecting to zigbee2mqtt |
I have this problem too, with an external broker on a separate VM. I use for this:
The MQTT broker node is configured with the MQTT protocol version 3.1.1 and there are no messages configured on connection, before disconnecting or unexpected disconnection. |
The issue still exists with Node-RED version 2. Is there any fix available? |
Same problem here. Any update? Thanks! |
@RGNagel just to be 100% clear you are using the mqtt broker nodes as well? If so the best workaround is to one of the following
The error is normally because the broker is getting stopped before the client nodes |
Hello Together
The broker runs on the same pi and is mosquitto (2.0.11). I have already completely deleted the flow and recreated by hand. The first 4 or 5 deploys were fast. After that the behavior happened again. |
@denonbw This issue is all about using node-red-contrib-aedes. If you have a problem when using an external broker (mosquitto) then it is very unlikely to be related to this as it is caused by the broker (aedes) being shutdown before the mqtt node connected to it tries to send it's last message. Given your flow doesn't appear to be using birth or close messages again not likely to be related to this issue. I suggest you open a new issue and include all the relevant data (including the Node-RED log output covering the period before and after the deploy and mosquitto logs for the same period) |
Same problem after update
mosquitto version 2.0.13 |
@sanyafifa can you supply Node-RED logs from before and after a deploy and the broker logs from the same period. Along with the type of deploy (full, changed flows, changed nodes). |
As well as you logs, can you describe you setup (OS, Hardware, direct connection to internet? behind proxy?) |
Same here since updating from v2.2.0 to v2.2.1. Raspberry Pi 3b
Install log:
|
Hi Paul, I see aedes in your screenshot but I don't say it listed in your nodes. What gives? Are you using aedes as a broker in your node-red? |
I should have made this clear, apologies. Aedes broker is running on a cloud server, which has not been edited, or even the server accessed during the past few weeks.
When node_RED restarts, If I search amongst my Cron+ log entries... 🔇
I am also running mosquitto version 1.5.7 on my local Pi which is working without issue, and handling most of my MQTT traffic. |
In fact, it's not confined to Aedes. Both Aedes & cloudmqtt have been happily working for a number of months, and these errors have only started after updating to node-RED v2.2.1, and worked fine under v2.2.0. |
We updated the underlying MQTT.js library in 2.2.1. It went from 4.3.4 to 4.3.5. The changelog doesn't reveal anything suspicious that could be causing this sudden influx of problems. But it is a change nonetheless. We have the option to rollback that update and release 2.2.2 - that would be a quick and pragmatic (and temporary) solution. The alternative is we try to diagnose the actual issue and see if there's a change needed in how the |
If you can't replicate, I'm around today, and will gladly help wherever I can. |
For me it is not an pressing problem, so I can wait until the problem is found. The deploy works, it just takes a long time and returns an error. Sorry for not opening a new issues and posting it here in the wrong one. |
I'm also encountering this issue since updating to the newest docker container. nodered 2.2.1 |
@Paul-Reed did you leave off the |
Yes. |
An update for those waiting a solution. I now understand the issue (race condition between in/out nodes closing vs broker disconnecting -AND- the MQTT lib change of when it calls the TBH: I rarely use full deploy so hadn't noticed this however I suspect the issue has been present since I added Dynamic subscriptions to MQTT 4 months ago 😱 Should have an update for you today or tomorrow 🤞 |
Thanks Steve, however I'd be surprised if it has been present 4 months, as I updated to node-RED v2.2.0 on 29th Jan, and this problem did not occur until I updated to v2.2.1 yesterday. |
Have you recently started using FULL DEPLOY? Or recently started changing MQTT nodes? The issue is related to when either a FULL DEPLOY or a PARTIAL/NODE DEPLOY causes all MQTT in/out nodes to be In other words, if you have been moving flows around and happened to affect all MQTT in/out nodes that are related to 1 broker, you can trigger this timing issue. No matter anyhow. Perhaps it is better to say, all the elements of the problem have been in place since dynamic MQTT however a number of other factors (MQTT client internal changes) have highlighted the issue. |
mosqutto log using FULL DEPLOY NR:
This issue was introduced in v2.2.1 . i always use full deploy |
As I said above...
So, instead of expending efforts on when / how, I am concentrating on a more stable solution. Shouldn't be too long. |
Update: I have a working fix but I will be spending a lot more time trying various scenarios, including the aedes issue mentioned earlier in this thread before I publish. Possibly tomorrow or Friday. In the meantime, please sit tight (or avoid full deploys if you can) |
Same problem. Thanks! |
2.2.2 Works great! Thanks for the quick fix |
I'm using the Home Assistant Node Red plugin. I get the error if I go from nodered_11.0.2 to 11.0.3. I'm using https://www.home-assistant.io/integrations/mqtt as my broker. |
@quadhammer those version numbers are home assistant versions not node-red versions. What version number of node-red do you see if you look in the nodes hamburger menu? |
when is v2.2.2 being released in home assistant? |
@jonascoenen005 that is not a question the Node-RED community can answer, we have no input into when Home Assistant release things |
Node-RED v2.2.0 |
@quadhammer hopefully now you can see you are not on 2.2.2 which contains this fix. The node-red community has no control or influence on when HA release updates. In the short term, until HA release an update, I suggest you stick to node deploys (instead of full deploys) to reduce the occurrence of this timeout. |
Yes, thanks. The latest one is: "Bump node-red from 2.2.0 to 2.2.1 in /node-red" so it shouldn't be too far behind. I'm sticking with 2.2.0 which doesn't seem to have the problem, even with full deploys. Cheers! |
I do have the same problem
some Tasmotadevices won't be found anymore. In Mosquitto the devices are normal visibe |
@Mark-Muc this is more likely an issue with that node & you should raise it on the tasmota nodes repository. |
Also, @Mark-Muc to minimise / reduce the problem, use "modified nodes" instead of "full deploy" where possible. |
@Steve-Mcl |
What are the steps to reproduce?
Import the flow in: https://cookbook.nodered.org/mqtt/connect-to-broker, start any MQTT broker (e.g. Mosquitto) and deploy
EDIT: The error is not evident with Mosquitto, as initially reported, but e.g. with https://github.com/martin-doyle/node-red-contrib-aedes.
What happens?
Every time the flow is deployed, the MQTT in node gets stuck for a while, then output: "Error stopping node: Close timed out"
What do you expect to happen?
The MQTT in node is immediately closed without errors and the flow is deployed without error and time wasting.
Please tell us about your environment:
The text was updated successfully, but these errors were encountered: