esp32 Modbus TCP master problem (IDFGH-8393)

[zhangxujun16]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Environment
Development Kit: [ESP32-Wrover-Kit]
Kit version (for WroverKit/PicoKit/DevKitC): [v4.4]
Module or chip used: [ESP32-WROVER-B]
IDF version: [v4.4]

Modbus TCP master can only works well for several minutes , and then show such error:

MB_CONTROLLER_MASTER: mbc_master_get_parameter(83): Master get parameter failure, error=(0x107) (ESP_ERR_TIMEOUT)

sometimes it can recover , and run several minutes , but then go to fail again.
sometimes it will also trigger watch dog like this:

_�[0;31mE (160376) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:�[0m
�[0;31mE (160376) task_wdt: - IDLE (CPU 0)�[0m
�[0;31mE (160376) task_wdt: Tasks currently running:�[0m
�[0;31mE (160376) task_wdt: CPU 0: tcp_master_task�[0m
�[0;31mE (160376) task_wdt: CPU 1: IDLE�[0m
�[0;31mE (160376) task_wdt: Print CPU 0 (current core) backtrace�[0m

Backtrace:0x400F4416:0x3FFB0B800x40082C59:0x3FFB0BA0 0x4000BFED:0x3FFCEAF0 0x4008D4EA:0x3FFCEB00 0x4008A39F:0x3FFCEB20 0x4010DBB2:0x3FFCEB60 0x400F8D24:0x3FFCEB80 0x400F8D3F:0x3FFCEBA0 0x400F903E:0x3FFCEBC0 0x400F905A:0x3FFCEBE0 0x400FAD66:0x3FFCEC00 0x400FAE56:0x3FFCEC60 0x400DDB09:0x3FFCEC80 0x400DDBE2:0x3FFCECB0 0x400DDFEA:0x3FFCECE0 0x4008D239:0x3FFCED30_

once it trigger watch dog , it will never recover again and it always trigger wdt in loop .
What's wrong with the tcp modbus mater please ?

[zhangxujun16]

I have try tcp modbus with the new patch : esp-modbus component,
replace folders: common, modbus, port, serial_master, serial_slave,tcp_master, tcp_slave with files in the patch.
but the result seems more worse, It go to panic directly.

[alisitsyn]

Hello @zhangxujun16,

Thank you for this issue. In order to reproduce the issue please send your whole TCP modbus project including the build folder. This will allow to get backtrace from your binary (only the *.elf, *.bin, *.map files are required in that folder) . Starting the project with idf.py -p PORT flash monitor will also allow to get backtrace automatically on your side. The third option is to create the core dump when the application crashes on WDT.

Thank you.

[zhangxujun16]

Thank you very much, If use the new modbus patch, mbc_master_set_descriptor must be called before mbc_master_start. Otherwise it will go to panic . The old doesn't require this . I will test for several days whether it is stable .

[zhangxujun16]

**After test modbus tcp , I find a problem .
If the wifi signal quality is bad for minutes, the modbus tcp master will fail to work for ever, even wifi signal quality recovery soon again.

[alisitsyn]

@zhangxujun16 ,

mbc_master_set_descriptor must be called before mbc_master_start.

In fact all your register areas should be defined calling mbc_master_set_descriptor() as described in the documentation before start. The master and slave can not work without this it does not matter which version of Modbus stack you are using.

If the wifi signal quality is bad for minutes, the modbus tcp master will fail to work for ever, even wifi signal quality recovery soon again.

If the master lost connection with certain modbus slave and it is not recovered after MB_TCP_RECONNECT_TIMEOUT = 5sec, the master starts reconnection cycle and as per my tests is able to connect to the slaves. However this behavior has some dependencies with your network environment and with dependent components such as esp-wifi, esp-mdns, lwip. Please send your log which would describe the situation and when slave was lost and connected again. I will try to check this and provide some recommendations.

Thanks.

[zhangxujun16]

@alisitsyn
we have two tcp modbus slaves A, B.
After starting mbc_master_start , it works well. Several minutes later, we close slave A ,
and left B active. mbc_master_get_cid_info will suspend and mbc_master is always waiting for connection of slave A . We expect mbc_master continue to get data from slave B and don't suspend because of slave A's
disconnection. Even we create a thread for each slave and it has no effective , if one slave close during work, mb master will not sample any slave. how to do it ?
Thanks!

[alisitsyn]

@zhangxujun16 ,

The stack is designed in such a way that if one node of the network is not alive the master can not normally proceed with control process and starts the re-connection stage after connection timeout. Usually the master is able to reconnect to slave and restart the polling cycle. If you need other behavior you need to perform some modification of the tcp port code as described below. The stack handles the communication fails in two places:

1. Handling of the case when current slave get data fail.
2. The handling of the read data failed due to critical error.

In these cases the master :

1. vMBTCPPortMasterStopPoll(); - stops polling cycle,
2. xMBTCPPortMasterCheckShutdown(); - checks if the master is waiting shutdown
3. xMBTCPPortMasterCheckConnState(&xConnSet); - checks connection states of each slave marked as connected on previous connection phase.
4. break - stops polling loop and restarts connection cycle for each slave;

If you need other behavior you can change the code in above branches to be able to continue the polling cycle even if the slave A is disconnected.
xTime = xMBTCPPortMasterGetRespTimeLeft(pxCurrInfo); - the processing time left for active transaction
xMBMasterPortFsmWaitConfirmation(MB_EVENT_REQ_DONE_MASK, pdMS_TO_TICKS(xTime)); - waits for completion of the active transaction in main Modbus FSM (synchronize the read polling cycle with the main FSM).

Using the above information you can modify the code above to perform your custom behavior of processing in case of slave errors. For example: you can check if the slave alive xMBTCPPortMasterCheckAlive(...) and mark it as bad or try to reconnect once. Let me know if you need more information.

[Alvin1Zhang]

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates. Thanks.