Improve docs and add examples

[brainelectronics]

Related to #8 and #9

Changes

• Add client examples for TCP and RTU
• Add scripts to read TCP and RTU Modbus client devices
• Fix documentation about usage

[beyonlo]

Hi @brainelectronics

I'm not sure, but I think I gave the feedback in the wrong place. Maybe it should be here and not as this reply on the #9. If here is the correct place to the feedback, please, let me know that I can to move from there to here.

Thank you so much.

[beyonlo]

Hi @brainelectronics

Here the all errors found in my tests using the PR #10:

• This test coverage all modes: the ModBus TCP (Master and Slave) and the ModBus RTU (Master and Slave).
• For all tests for the ModBus TCP Slave I used the new Modbus TCP Client example.
• For all tests for the ModBus RTU Slave I used the new Modbus RTU Client example.
• For all tests for the ModBus Master (TCP and RTU), working together with the Slaves above, I used the the new docs of the Master Implementation.
• For all tests I used MicroPython 1.19.1 in both sides: Slave and Master.

1. On the Slave implementation docs show "This requires the modules submodule to be cloned..", but that URL is broken.

2. On the Modbus TCP Client example I think that line 42 need to be inside the while loop (line 36), otherwise the loop print() will be very fast, without wait more time to connect to WiFi.

3. The Modbus TCP Client example do not accept more than 1 TCP connection. When I try to request from a second ModBus TCP Master, the first ModBus TCP Master is disconnected. I see that default max_connections in the ModBus implementation is 10 but that is not working.

4. coil_qty is not working. Same error for the ModBus TCP and RTU.

>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]   <--- "coil_qty=1" but showed 8 and not just 1.
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=20)
[True, False, False, False, False, False, False, False]   <--- "coil_qty=20" but still showing 8, and not 20.
>>>

However I tested from another Modbus TCP Master (the MODSCAN) connected to the same ModBus TCP Slave example and quantity works. Image below:

Ps 1: Take a look on the name Lenght on the MODSCAN image, that is the same of coil_qty
Ps 2: Address on the MODSCAN is 124 instead 123 because MODSCAN start count at 1.

5. COIL do not accept to write 1. It accept to write just 0. Same error for the ModBus TCP and RTU.

>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=0)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "umodbus/tcp.py", line 137, in write_single_coil
  File "umodbus/functions.py", line 48, in write_single_coil
ValueError: Illegal coil value
>>> 

However from the MODSCAN is possible to write 0 and 1

6. On the HREGS, is reading wrong value on the read_holding_registers. The default value is 19, but is showing (999,). Same error for the ModBus TCP and RTU.

register_definition -> 'EXAMPLE_HREG': {'register': 93, 'len': 1, 'val': 19}

>>> 
>>> host.read_holding_registers(slave_addr=10, starting_addr=93, register_qty=1, signed=False)
(999,)
>>> host.write_single_register(slave_addr=10, register_address=93, register_value=44, signed=False) <--- Writing a diferrent value to test
True
>>> host.read_holding_registers(slave_addr=10, starting_addr=93, register_qty=1, signed=False) <--reading again. Now works.
(44,)
>>> 

7. On the ISTS (read_discrete_inputs), the default value is 0, but ModBus Master is receiving 1 (True). Same error for the ModBus TCP and RTU.

register_definition -> 'EXAMPLE_ISTS': {'register': 67, 'len': 1, 'val': 0}

>>> host.read_discrete_inputs(slave_addr=10, starting_addr=67, input_qty=1)
[True, False, False, False, False, False, False, False]
>>> 

8. On the IREGS (read_input_registers), the default value is 60001, but ModBus Master is receiving (999, 999). Same error for the ModBus TCP and RTU.

register_definition -> 'EXAMPLE_IREG': {'register': 10, 'len': 2, 'val': 60001}

>>> host.read_input_registers(slave_addr=10, starting_addr=10, register_qty=2, signed=False)
(999, 999)
>>> 

Thank you very much!

[brainelectronics]

First and second mentioned topic is fixed without ticket

3. The Modbus TCP Client example do not accept more than 1 TCP connection. When I try to request from a second ModBus TCP Master, the first ModBus TCP Master is disconnected. I see that default max_connections in the ModBus implementation is 10 but that is not working.

That is true. According MicroPython Socket docs the listen method is about

... the number of unaccepted connections that the system will allow before refusing new connections

as I only had one connection so far, I did not see this issue. To be fixed in #11

4. coil_qty is not working.
   see Coil quantity specification does not work #12

5. COIL do not accept to write 1. It accept to write just 0. Same error for the ModBus TCP and RTU.
   After fixing Enable coil set value 0 and 1 instead of 0x0000 and 0xFF00 #14 True, False, 0xFF00, 0x000, 1 or 0 can be used, see

   micropython-modbus/umodbus/functions.py

   Line 47 in dd5b8e5

   if output_value not in [0x0000, 0xFF00]:

6. On the HREGS, is reading wrong value on the read_holding_registers. The default value is 19, but is showing (999,). Same error for the ModBus TCP and RTU.
7. On the ISTS (read_discrete_inputs), the default value is 0, but ModBus Master is receiving 1 (True). Same error for the ModBus TCP and RTU.
8. On the IREGS (read_input_registers), the default value is 60001, but ModBus Master is receiving (999, 999). Same error for the ModBus TCP and RTU.

Fixed by #13 the default value of parameter use_default_vals in the setup_registers function was True

[beyonlo]

Hello @brainelectronics

Thank you to fix the bugs and create new issue tickets to solve. I can confirm that bugs marked as solved/completed are working now!

I found more 3 problems using this PR.

1. host.write_single_coil() incorrect return when write 1/True.

I believe that return (True or False) on write methods is to check if data that will be write is correct - if is valid, right?
When I write COIL value as 1 or True, it works, I mean, that value is writing with success, but the valid return is False, differently when I write 0 or False, where the return is True.

>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=0)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=1)
False
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>> 

2. Error when trying to write multiple coils (host.write_multiple_coils())

I not found examples on the docs about how is the correct way to write multiple coils, so I get a example from here

>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>>
>>> host.write_multiple_coils(slave_addr=10, starting_address=123, output_values=[1,1,1,0,0,1,1,1,0,0,1,1,1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "umodbus/serial.py", line 274, in write_multiple_coils
  File "umodbus/serial.py", line 173, in _send_receive
  File "umodbus/serial.py", line 190, in _validate_resp_hdr
ValueError: slave returned exception code: 1
>>> 
>>> host.write_multiple_coils(slave_addr=10, starting_address=123, output_values=[1,1,1,0,0,1])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "umodbus/serial.py", line 274, in write_multiple_coils
  File "umodbus/serial.py", line 173, in _send_receive
  File "umodbus/serial.py", line 190, in _validate_resp_hdr
ValueError: slave returned exception code: 1
>>> 

3. Error when trying to write multiple registers (host.write_multiple_registers())

>>> host.write_single_register(slave_addr=10, register_address=93, register_value=44, signed=False)
True
>>> host.read_holding_registers(slave_addr=10, starting_addr=93, register_qty=1, signed=False)
(44,)
>>> 
>>> host.write_multiple_registers(slave_addr=10, starting_address=93, register_values=[2, -4, 6, -256, 1024], signed=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "umodbus/serial.py", line 291, in write_multiple_registers
  File "umodbus/serial.py", line 173, in _send_receive
  File "umodbus/serial.py", line 190, in _validate_resp_hdr
ValueError: slave returned exception code: 1
>>> 
>>> host.write_multiple_registers(slave_addr=10, starting_address=93, register_values=[2, 1024], signed=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "umodbus/serial.py", line 291, in write_multiple_registers
  File "umodbus/serial.py", line 173, in _send_receive
  File "umodbus/serial.py", line 190, in _validate_resp_hdr
ValueError: slave returned exception code: 1
>>> 

Thank you very much!

Edit: This problem happens in both: RTU and TCP.

[beyonlo]

Hello @brainelectronics

Could you to please take a look if is easy to fix that problems to write multiple coils and multiple registers? I would like so much to use that functions on ModBus :)

Thank you in advance!

[brainelectronics]

@beyonlo I'll have a look at it on the weekend again

[beyonlo]

Hey @brainelectronics did you have any success at the weekend? :)

Thank you so much!

[brainelectronics]

I had no time for this repo on the last weekend. The code in these repos is not created in my day job.

But I have a comment on your issue with reading a single coil.

The coils in the response message are packed as one coil per bit of the data field. Status is indicated as 1= ON and 0= OFF. The LSB of the first data byte contains the output addressed in the query. The other coils follow toward the high order end of this byte, and from low order to high order in subsequent bytes.

See https://modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf Page 11

So returning a list of 8 true or false elements is following the standard.

[beyonlo]

Hello @brainelectronics I'm sorry to bore you. Please, don't get me wrong. Take your time to fix that.

About the issue #12, reading a single coil you are right, it is following the standard. The return is always multiple of eight, and the next sentence of Page 11 confirm that:

If the returned output quantity is not a multiple of eight, the remaining bits in the final data
byte will be padded with zeros (toward the high order end of the byte). The Byte Count field
specifies the quantity of complete bytes of data.

Thank you and have a nice week.

[beyonlo]

Hi @brainelectronics Just a small error in the docs: on the Master implementation docs, on the TCP Master setup part:

host = ModbusTCPMaster(slave_ip=192.168.178.34, slave_port=180, timeout=5)

On the slave_ip, the quotes are missing: slave_ip= '192.168.178.34'

I saw that you created the issue #16 (Create test framework) to fix the problems and have this lib 100% compatible with ModBus - that's will be great, thank you very much for you effort! I still reporting new problems (like this) here. Please, let me know if you prefer on the another way/place.

[brainelectronics]

@beyonlo thank you really for your patience! I believe your comments and reports are really valuable for this lib! Keep reporting 😀

If it's a new thing to solve, directly create a new issue. Otherwise, if it is related to this PR, comment here so I can resolve it within this PR.

I'll extract code of the general helpers repo into this one to be able to really test it on my hardware.
Unfortunately I can only work on it one day per week, so outcome is not so high, but I'm on it!

[brainelectronics]

@beyonlo thank you really for your patience! I believe your comments and reports are really valuable for this lib! Keep reporting 😀

If it's a new thing to solve, directly create a new issue. Otherwise, if it is related to this PR, comment here so I can resolve it within this PR.

I'll extract code of the general helpers repo into this one to be able to really test it on my hardware.
Unfortunately I can only work on it one day per week, so outcome is not so high, but I'm on it!

[beyonlo]

@beyonlo thank you really for your patience! I believe your comments and reports are really valuable for this lib! Keep reporting grinning

@brainelectronics You are welcome! I'm very grateful to helping, at least, testing this lib.

If it's a new thing to solve, directly create a new issue. Otherwise, if it is related to this PR, comment here so I can resolve it within this PR.

I agree!

I'll extract code of the general helpers repo into this one to be able to really test it on my hardware.

I'm not a Python expert, but I did a take a look in that repo, and I think that you will adapt it to create/use as a new framework (#16) to work testing and fixing the bugs.

Unfortunately I can only work on it one day per week, so outcome is not so high, but I'm on it!

No problem. At least, even with few time, there is consistency - that is very important :)

When you have a new version/PR (even not finished) let me know that I can to do all tests again!

[beyonlo]

Hello @brainelectronics

How are you?

I trying to choose the right words here to you do not get me wrong. English is not my native language, so something can be not perfect :)

I would like to know if do you have plans to fix that all problems in this lib and have it 100% following the ModBus requirements until the end of this year (2022).

I'm developing a simple project with ESP32 and MicroPython that need to use ModBus TCP/RTU Slave/Master, and my deadline is until end of this year. Because that I'm asking if you have plans to finish still in this year. If not, I need to find another solution, but i don't know what :)

I'm trying to figure out how I can help you in this process and improve this amazing ModBus library. Unfortunately I have no acknowledge enough in development to help you with ModBus problems/issues, but I can to try to fix the #11 - I had worked a bit with sockets and maybe I can to research/study to fix that issue. Another way that I'm thinking is to donate something to this project. As this project is a amazing lib, people can to contribute donating what they can and than back to community as better lib. I have no much money, that is one reason that I use a cheap micro-controller (ESP32), but I can to donate something. Maybe to put here (github project) a link explaining how to donate. Another way that I can to help is testing everything that you develop/fix and reporting everything. Another thing that I can help, if you agree that is necessary, is to create a better place to the docs, like as the https://readthedocs.org or another utility that you think be better to build the docs, if necessary. If you have another idea/thoughts, please let me know!

Thank you very much for your attention!

[brainelectronics]

Hi @beyonlo
it's been a while since I had to use this package, but there is a new need for it. I've made some progress on the overall testing environment and automation. I plan to contribute further to this package in calender week 44. I might split your request and comments into several smaller PRs to finish them faster and provide smaller deliverables. I'm confident to make the package compliant with the Modbus spec by the end of this year!

I'm also thinking about a good way for documentation like RTD but I would need to invest some time to get familiar with it.

[beyonlo]

Hello @brainelectronics

This is a wonderful news! I'm so happy :)

Thank you very much for your effort!

[brainelectronics]

#19 provides possibility to test release candidates, see https://github.com/brainelectronics/micropython-package-template#test-version

import upip
# overwrite index_urls to only take artifacts from test.pypi.org
upip.index_urls = ['https://test.pypi.org/pypi']
upip.install('micropython-modbus')

[beyonlo]

Hello @brainelectronics

I did a simple test on the new release 1.1.0. I dowloaded it from the 1.1.0 tag instead to use upip.

I would like just to notice you that some bugs that was already fixed during the PR #10 are back in this new release. Maybe you already know that, or just started to works in the new version using the original 1.0.0 instead from that PR.

Ps: as I checked in the 1.1.0 changelog that you still not fixed other bugs, I didn't tried to test other things.

Thank you very much!

[brainelectronics]

@beyonlo I startet from the latest version on develop which did of course not contain the fixes and changes of this PR. I'll later pull the now updated develop to create a 1.1.1 release. I'll also finish my test setup within the next few hours to be sure everything is really backwards compatible.

[brainelectronics]

@beyonlo this PR is updated so far. I'll merge after your approval/review and continue with the next tickets and fixes

[beyonlo]

@beyonlo this PR is updated so far. I'll merge after your approval/review and continue with the next tickets and fixes

Hi @brainelectronics

My understand is that I need to test this PR (named as 1.1.1) just what are described on the changelog.md in the version 1.1.1, listed as Fixed. I will to test tomorrow, because I'm going to sleep now :)

I would like to notice you that the link 1.1.1 inside changelog.md is broken. I clicked in that 1.1.1 to download to test, but I understand that this PR already is the 1.1.1.

Sorry for me to confirming above somethings that maybe are obvious for you, but not for me! If I'm missing some understand, please, tell me.

Tomorrow I will post here the tests results :)

[brainelectronics]

Your of course absolutely right with the broken link in the changelog. The tag 1.1.1 will only be available after this PR is merged. I'll create a ticket to post a -rcX tag at the same time as a test deploy to Test PyPi has been made

Right now you could install the package like this

import upip
# overwrite index_urls to only take artifacts from test.pypi.org
upip.index_urls = ['https://test.pypi.org/pypi']
upip.install('micropython-modbus')

That would install the latest version micropython-modbus 1.1.1rc3.dev10 on your board.

[beyonlo]

Hi @brainelectronics

Tests done on the 1.1.1rc3.dev10 version. I installed via upip as you suggest me: Installing micropython-modbus 1.1.1rc3.dev10

Note: now the new version need the urequests, that I installed via upip aswell: Installing urequests 0.6

Tests results

Short asnwer: all passed

Long asnwer: details of tests below

For all tests was used this Slave TCP:
Used this Slave example: tcp_client_example.py

Serving on IP: 192.168.1.4 on port: 502
Using this register_definitions: {'ISTS': {'EXAMPLE_ISTS': {'register': 67, 'len': 1, 'val': 0}}, 'IREGS': {'EXAMPLE_IREG': {'register': 10, 'len': 2, 'val': 60001}}, 'HREGS': {'EXAMPLE_HREG': {'register': 93, 'len': 1, 'val': 19}}, 'COILS': {'EXAMPLE_COIL': {'register': 123, 'len': 1, 'val': 1}}}

1.1.1 - 2022-11-09, Fixed:

Bugfix number 1: Default value of setup_registers function parameter use_default_vals changed to False to avoid confusion behaviour if not explicitly defined, see #13

Master TCP:

>>> from umodbus.tcp import TCP as ModbusTCPMaster
>>> host = ModbusTCPMaster(slave_ip='192.168.1.4', slave_port=502, timeout=5)
>>> host.read_holding_registers(slave_addr=10, starting_addr=93, register_qty=1, signed=False)
(19,)
>>> host.write_single_register(slave_addr=10, register_address=93, register_value=44, signed=False)
True
>>> host.read_holding_registers(slave_addr=10, starting_addr=93, register_qty=1, signed=False)
(44,)
>>> 
>>> 
>>> host.read_discrete_inputs(slave_addr=10, starting_addr=67, input_qty=1)
[False, False, False, False, False, False, False, False]
>>> 
>>> 
>>> host.read_input_registers(slave_addr=10, starting_addr=10, register_qty=2, signed=False)
(60001,)
>>> host.read_input_registers(slave_addr=10, starting_addr=10, register_qty=1, signed=False)
(60001,)
>>> 

Bugfix number 2: Missing function docstring added to setup_registers function

Nothing to test.

Bugfix number 3: write_single_coil allows 0, 1, False, True, 0x0 or 0xFF00 instead of 0x0 and 0xFF00 only as set value, see #14

Master TCP:

>>> from umodbus.tcp import TCP as ModbusTCPMaster
>>> host = ModbusTCPMaster(slave_ip='192.168.1.4', slave_port=502, timeout=5)
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=0)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=1)
False
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]

[beyonlo]

Test report, part 2:

Important note: just to remember you, there are many bugs reported in this PR that was not fixed yet, and was not created separated tickets for them. If you prefer, let me know that I can to open issues for them.

On the Bugfix number 3 I forgot to test the False, True, 0x0 and 0xFF00

Short answer: all passed

Long answer: details below

>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=True)
False
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>> 
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=False)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> 
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=0xFF00)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[True, False, False, False, False, False, False, False]
>>> host.write_single_coil(slave_addr=10, output_address=123, output_value=0x0)
True
>>> host.read_coils(slave_addr=10, starting_addr=123, coil_qty=1)
[False, False, False, False, False, False, False, False]
>>> 

[brainelectronics]

@beyonlo thank you very much for your extensive test reports! I would be happy if you could open new issues for each individual bug reported in this PR but not yet fixed.
If you agree I would after the issues are created merge this PR and continue with #19 to fix the "set multiple coils" bug

[beyonlo]

Hi @brainelectronics

@beyonlo thank you very much for your extensive test reports! I would be happy if you could open new issues for each individual bug reported in this PR but not yet fixed. If you agree I would after the issues are created merge this PR and continue with #19 to fix the "set multiple coils" bug

Excellent, I agree! :)

Tomorrow I will open an individual issue for each bug reported in this PR that is not fixed yet.

Thank you very much!

[beyonlo]

Hi @brainelectronics

Tomorrow I will open an individual issue for each bug reported in this PR that is not fixed yet.

Done, all issues created.

When you have new versions, just let me know that I can to do all tests.

Note 1: The last test I did just using ModBus TCP (Master and Slave) from this lib. I forgot to test the RTU mode. I would like to say you that from the next test, I will always test TCP and RTU, like as I did everytime in other tests.

Note 2: My mode to test this lib is using this way:
ModBus TCP: ESP32_1 (Slave) <------ WiFi ----> ESP32_2 (Master)

ModBus RTU: ESP32_1 (Slave) <------ UART ----> ESP32_2 (Master)
Ps: On the RTU tests, for now, I'm using just UART, but in the production will he RS485, but I think that is transparent for this lib, just I will need to have a enable/disable a pin to read/write, right?

Thank you so much! :)