Add support for acting as Modbus server

[JeroenVanOort]

What does this implement/fix?

This adds the ability to let ESPHome act as a Modbus RTU server, i.e. it can handle requests from a modbus client instead of being the client itself.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Other

Related issue or feature (if applicable): fixes esphome/feature-requests#708

Pull request in esphome-docs with documentation (if applicable): esphome/esphome-docs#3332

Test Environment

• ESP32
• ESP32 IDF
• ESP8266
• RP2040

Example entry for config.yaml:

# For testing, I hooked up two UARTS together (RX -> TX, TX -> RX) 
uart:
  - id: uart_modbus_client
    tx_pin: 32
    rx_pin: 34
    baud_rate: 115200
  - id: uart_modbus_server
    tx_pin: 25
    rx_pin: 35
    baud_rate: 115200

sensor:
  - platform: modbus_controller
    modbus_controller_id: modbus_evse
    name: "Laadpaal: L1-N"
    register_type: holding
    address: 0x0000
    device_class: voltage
    entity_category: diagnostic
    value_type: S_DWORD_R
    accuracy_decimals: 1
    unit_of_measurement: V
    filters:
      - multiply: 0.1

modbus:
  - uart_id: uart_modbus_client
    id: modbus_client
  - uart_id: uart_modbus_server
    id: modbus_server
    role: server

modbus_controller:
  - id: modbus_evse
    modbus_id: modbus_client
    address: 0x2
    update_interval: 5s
  - id: modbus_evse_server
    modbus_id: modbus_server
    address: 0x2
    server_registers:
      - address: 0x0000
        value_type: S_DWORD_R
        read_lambda: |-
          return 42.3;

Checklist:

• The code change is tested and works locally.
• Tests have been added to verify that the new code works (under tests/ folder).

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated in esphome-docs.

[jpeletier]

@JeroenVanOort This is great and exactly what I am talking about.

I gave it a good look and pointed to a very few things but nothing important. A net of +126 lines of code and it is backwards compatible, which is very important. From reading the code, I understand writing registers is TBD.

I hope to be able to give it a good testing next week.

Cheers

[probot-esphome]

Hey there @martgras, mind taking a look at this pull request as it has been labeled with an integration (modbus_controller) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[JeroenVanOort]

The code itself is ready for review: only need to make a PR for the docs.

[playya]

From my point of view we should not serverregister and instead maintain server registers in sensorset_, too.

This allows to inherit modbus::ModbusDevice e.g. in sdm_meter and set sensors for registers and use the role for server and client implementation.

[JeroenVanOort]

I think I understand what you're trying to say, but I think SensorItem has some properties that are not needed for using it as a server register model and SensorItem doesn't have a lamba property for returning the register value.

[playya]

Yes that's the overhead if you implement both roles in one class. That's the reason for me to add a new component.
Your solution allows to inherit ModbusDevice e.g. in sdm_meter and add SensorItems and let use user choose the role.

My biggest objection on the current implementation is that it's only providing lambda to get the value, but from my point of view it should be possible to create a device in pure yaml without knowing any c++.

I see many people asking for virtual/fake modbus devices to create an adapter between devices in to solar area and that's how I currently use my modbus_device.

[Call-Me-G-Now]

hey guys, any update on this? Really interested in this.

[tjsaunders]

Hi @JeroenVanOort,

Thanks for your work on this!

Please forgive me if I'm missing something, but after implementing your pull request, I think there's a bug in your on_modbus_read_registers function in the esphome/components/modbus_controller/modbus_controller.cpp file.

I believe there's a mistake in the iteration logic? Again please correct me if I'm missing something, but it appears that the outer for loop will continue iterating through all the set registers even if the first one matches the request, and with "found" set to false on every iteration, each subsequent register will throw an error.

Does the "found" flag only need to be set once, before the initial for loop? I think that would work in my use-case, but I don't know if it would break others that I don't understand.

Thanks

T

Some context:

modbus_controller:
  - id: dataread1
    command_throttle: 100ms
    modbus_id: modbus_server
    address: 0x1
    server_registers:
      - start_address: 0x0C
        value_type: FP32
        lambda: |-
          return 11.3;
#  - id: dataread2
#    command_throttle: 100ms
#    modbus_id: modbus_server
#    address: 0x1
#    server_registers:
#      - start_address: 0x0E
#        value_type: FP32
#        lambda: |-
#          return 0.0;
#  - id: dataread3
#    command_throttle: 100ms
#    modbus_id: modbus_server
#    address: 0x1
#    server_registers:
#      - start_address: 0x10
#        value_type: FP32
#        lambda: |-
#          return 0.0;

With this yaml, a request 0104000c0002b1c8 (2 registers starting at 0c, works fine). The same yaml without the comments gives this response:

[D][modbus_controller:082]: Received read holding/input registers. FC: 0x4. Start address: 0xC. Number of registers: 0x2.
[D][modbus_controller:093]: Matched register. Start address: 0x0C. Value type: 12. Register count: 2. Value: 11.3.
[D][uart_debug:114]: <<< 01:04:00:0C:00:02:B1:C8
[D][modbus_controller:082]: Received read holding/input registers. FC: 0x4. Start address: 0xC. Number of registers: 0x2.
[W][modbus_controller:101]: Could not match any register to address 0C. Sending exception response.
[D][modbus_controller:082]: Received read holding/input registers. FC: 0x4. Start address: 0xC. Number of registers: 0x2.
[W][modbus_controller:101]: Could not match any register to address 0C. Sending exception response.
[D][uart_debug:114]: >>> 01:04:04:00:00:00:0B:BA:43:01:81:02:C1:91:01:81:02:C1:91

[JeroenVanOort]

What you probably want to do, is combine the controllers with the same address into one, like this:

modbus_controller:
  - id: dataread1
    command_throttle: 100ms
    modbus_id: modbus_server
    address: 0x1
    server_registers:
      - start_address: 0x0C
        value_type: FP32
        lambda: |-
          return 11.3;
      - start_address: 0x0E
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 0x10
        value_type: FP32
        lambda: |-
          return 0.0;

The way you've done it, multiple devices are matched in modbus.cpp. Possibly, the matching should stop after the first and the configuration should not allow controllers with the same address for a given modbus_id. I suppose that's for a different PR.

[FloMeyer]

hi @JeroenVanOort, i also implemented your pr and i am a little bit confused.

My config:

modbus_controller:
  - id: modbus_server_controller
    command_throttle: 100ms
    modbus_id: modbus_server
    address: 0x2
    server_registers:
      - start_address: 0 # Phase 1 line to neutral volts
        value_type: FP32
        lambda: |-
          return id(hass_p1_voltage).state;
      - start_address: 2 # Phase 2 line to neutral volts
        value_type: FP32
        lambda: |-
          return id(hass_p2_voltage).state;
      - start_address: 4 # Phase 3 line to neutral volts
        value_type: FP32
        lambda: |-
          return id(hass_p3_voltage).state;
      - start_address: 6 # Phase 1 current
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 8 # Phase 2 current
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 10 # Phase 3 current
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 12 # Phase 1 power
        value_type: FP32
        lambda: |-
          return id(hass_p1_power).state;
      - start_address: 14 # Phase 2 power
        value_type: FP32
        lambda: |-
          return id(hass_p2_power).state;
      - start_address: 16 # Phase 3 power
        value_type: FP32
        lambda: |-
          return id(hass_p3_power).state;

      - start_address: 18 # Phase 1 volt amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 20 # Phase 2 volt amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 22 # Phase 3 volt amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 24 # Phase 1 volt reactive amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 26 # Phase 2 volt reactive amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 28 # Phase 3 volt reactive amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 30 # Phase 1 power factor
        value_type: FP32
        lambda: |-
          return id(hass_p1_pf).state;
      - start_address: 32 # Phase 2 power factor
        value_type: FP32
        lambda: |-
          return id(hass_p2_pf).state;
      - start_address: 34 # Phase 3 power factor
        value_type: FP32
        lambda: |-
          return id(hass_p3_pf).state;

      - start_address: 36 # Phase 1 phase angle
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 38 # Phase 2 phase angle
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 40 # Phase 3 phase angle
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 42 # Average line to neutral volts
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 46 # Average line current
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 48 # Sum of line currents
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 52 # Total system power
        value_type: FP32
        lambda: |-
          return id(hass_total_system_power).state;
      - start_address: 56 # Total system volt amps
        value_type: FP32
        lambda: |-
          return 0.0;
      - start_address: 60 # Total system VAr
        value_type: FP32
        lambda: |-
          return 0.0;

Log output:

[21:35:10][D][modbus_controller:082]: Received read holding/input registers. FC: 0x4. Start address: 0x0. Number of registers: 0x12.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x00. Value type: 12. Register count: 2. Value: 237.5.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x02. Value type: 12. Register count: 2. Value: 237.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x04. Value type: 12. Register count: 2. Value: 237.4.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x06. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x08. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x0A. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x0C. Value type: 12. Register count: 2. Value: 130.6.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x0E. Value type: 12. Register count: 2. Value: 236.7.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x10. Value type: 12. Register count: 2. Value: 135.2.

[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x12. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x14. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x16. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x18. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x1A. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x1C. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x1E. Value type: 12. Register count: 2. Value: 0.7.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x20. Value type: 12. Register count: 2. Value: 0.9.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x22. Value type: 12. Register count: 2. Value: 0.9.

[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x24. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x26. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x28. Value type: 12. Register count: 2. Value: 0.0.
[21:35:10][D][modbus_controller:093]: Matched register. Start address: 0x2A. Value type: 12. Register count: 2. Value: 0.0.

[21:35:10][D][uart_debug:114]: <<< 02 04 00 00 00 12 70 34
[21:35:11][D][uart_debug:114]: >>> 02 04 58 00 00 00 ED 00 00 00 ED 00 00 00 ED 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 82 00 00 00 EC 00 00 00 87 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 3E 25

[21:35:11][W][component:204]: Component modbus took a long time for an operation (0.38 s).
[21:35:11][W][component:205]: Components should block for at most 20-30ms.

So it says

Start address: 0x0. Number of registers: 0x12.

but it responds much more registers..

And also the float to hex conversion does not work as i need it.
E. g. for Value: 237.5 the response is. 0x0000 (0), 0x00ED (237)
But as you can convert it for yourself (https://resource.heltec.cn/utils/hf) is has to be 0x436d8000 -> so 0x436d and 0x8000 in the two registers.
Or does it work as expected and i need to send raw hex values and convert the fp32 values to hex via lambda?

Thanks so far for the great work!

[FloMeyer]

Hi @JeroenVanOort, i fixed an issue in JeroenVanOort#1. Now only the registers are returned for what was asked in the request.

[JeroenVanOort]

May I ask why you used start_address: instead of address: like the modbus_controller? Just looking for a little bit of consistency, but if the purpose of the field is different, an explanation would be great.

To be honest, I don't even know. It indeed makes way more sense to use address.

value_type: might be useful, to define how many bytes the read_lambda: should be packed into. U_DWORD is 32 bits, so a read to 0x9001 for 1 register should return only the high word, while a read to 0x9002 for 1 register would return the low word. Kind of works with the start_address: idea, although there is no corresponding end_address:. At least address: and value_type: together define the extent of the value. One thing that the value_type: field could possibly also assist with is conversion of the read_lambda: result to the provided value_type:. In your example above, how should the floating point 42.3 be encoded as a S_DWORD_R, since that is defined as a 32-bit integer? Maybe we need a way to specify that too?

The other thought I had was with regards to atomicity. It's not ideal to repeat the read_lambda: to get the high register and the low register of a U_DWORD, in case the result changes from one execution to the next (possibly via a side effect?). Not 100% sure how feasible that is. Obviously, if the reads are in separate commands, all bets are off, getting non-atomic results should not be a surprise. But if I request two registers starting at 0x9001, does it make sense to execute the read_lambda: twice? Feels almost like a bit of lookahead might be useful to see how many registers were requested, and whether more than one register can be satisfied from a single read_lambda: based on the value_type:. This can probably wait for a future enhancement, but might be worth calling out as explicitly not considered yet, so that a future contributor might implement it.

The way this is currently implemented, requesting 1 register at 0x9001 would get you both words (so 2 registers). This might not be great, but I personally don't care about it too much. Requesting 0x9002 would get you nothing, about which I don't care too much either, because I don't see a use case for requesting high and low words of the same value separately.

Please understand I am not trying to make more work for you, I really appreciate what you have already done, which does work! Just trying to make sure that the design makes sense, before it gets merged, and there are breaking changes that need to be made.

Thank you for acknowledging this! From time to time it feels like everyone is just nitpicking at my code while I'm just trying to contribute something that I made for myself. It takes conscious thought to realize that we are all just trying to make good things.

[nagyrobi]

Thank you for acknowledging this! From time to time it feels like everyone is just nitpicking at my code while I'm just trying to contribute something that I made for myself.

We're all trying to team up our efforts to make it better for everyone. I'm sorry if I made you feel uncomfortable, it's also probably because English is not my native language.

[RoganDawes]

May I ask why you used start_address: instead of address: like the modbus_controller? Just looking for a little bit of consistency, but if the purpose of the field is different, an explanation would be great.

To be honest, I don't even know. It indeed makes way more sense to use address.

😎

value_type: might be useful, to define how many bytes the read_lambda: should be packed into. U_DWORD is 32 bits, so a read to 0x9001 for 1 register should return only the high word, while a read to 0x9002 for 1 register would return the low word. Kind of works with the start_address: idea, although there is no corresponding end_address:. At least address: and value_type: together define the extent of the value. One thing that the value_type: field could possibly also assist with is conversion of the read_lambda: result to the provided value_type:. In your example above, how should the floating point 42.3 be encoded as a S_DWORD_R, since that is defined as a 32-bit integer? Maybe we need a way to specify that too?

The other thought I had was with regards to atomicity. It's not ideal to repeat the read_lambda: to get the high register and the low register of a U_DWORD, in case the result changes from one execution to the next (possibly via a side effect?). Not 100% sure how feasible that is. Obviously, if the reads are in separate commands, all bets are off, getting non-atomic results should not be a surprise. But if I request two registers starting at 0x9001, does it make sense to execute the read_lambda: twice? Feels almost like a bit of lookahead might be useful to see how many registers were requested, and whether more than one register can be satisfied from a single read_lambda: based on the value_type:. This can probably wait for a future enhancement, but might be worth calling out as explicitly not considered yet, so that a future contributor might implement it.

The way this is currently implemented, requesting 1 register at 0x9001 would get you both words (so 2 registers). This might not be great, but I personally don't care about it too much. Requesting 0x9002 would get you nothing, about which I don't care too much either, because I don't see a use case for requesting high and low words of the same value separately.

Ok, I think that's fine. Perhaps responding with an error might be better than nothing, but that can wait for another PR, I'm sure, as could responding with the requested number of registers. Not sure how compliant it is to respond with more than requested, but it's probably not a big deal.

Please understand I am not trying to make more work for you, I really appreciate what you have already done, which does work! Just trying to make sure that the design makes sense, before it gets merged, and there are breaking changes that need to be made.

Thank you for acknowledging this! From time to time it feels like everyone is just nitpicking at my code while I'm just trying to contribute something that I made for myself. It takes conscious thought to realize that we are all just trying to make good things.

I know that having people make too many comments can feel like they are never satisfied. I'm sorry if I made you feel that way. Thanks again for your contribution!

[yutani42]

Hope this gets merged into a Release soon. Been using this feature for a few months now, works great!

[bubble07]

Hope this gets merged into a Release soon. Been using this feature for a few months now, works great!

I'm really keen to try it but don't know how to, I've installed ESPhome (dev), but I can't get this pull to work.
In the log I see:
INFO: Installing ESPHome from fork 'JeroenVanOort:modbus-server' (https://github.com/JeroenVanOort/esphome/archive/modbus-server.tar.gz)...
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install
...
And then a little later in the log in red:

FATAL: Failed installing ESPHome from fork.
cont-init: info: /etc/cont-init.d/30-esphome-fork.sh exited 1
cont-init: warning: some scripts exited nonzero
s6-rc: warning: unable to start service legacy-cont-init: command exited 1

Any ideas what I'm doing wrong?

Many thanks

[pedrovanzella]

For what it's worth, I've tested this between two Kincony boards (one KC868-AI acting as a server, exporting the state of the dry contact switches, and one KC868-A16 acting as a client reading those states every second) and it works wonderfully! Thanks for this!

Hope to see this merged soon.

[pedrovanzella]

Tested with 2024.04.0, working great.

[esphome]

Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍

Learn more about our pull request process.