I break my head on how to get mqtt reading from HA into evcc also installed in HA

[maskedfighter]

Hi there!
Just yesterday i started getting into evcc since we got our solar panel installation and BEV all ready. So far, evcc is installed, inverters (we got 2 of them) are running and evcc is showing the correct amount of produced solar energy. Now i wanted to integrate my energy meter (i have a KAIFA smartmeter from my energy provider, and do readout over user-interface, data provided via mqtt). Data in HA energy dashboard, etc. is all perfectly fine, mqtt connection is rockstable and running perfectly fine.
So now i wanted to get the specific data readout into evcc.yaml, but i struggle hard, not exactly knowing how to get those mqtt readings into it.
This is my evcc.yaml right now:

network:
  # schema is the HTTP schema
  # setting to `https` does not enable https, it only changes the way URLs are generated
  schema: http
  # host is the hostname or IP address
  # if the host name contains a `.local` suffix, the name will be announced on MDNS
  # docker: MDNS announcements don't work. host must be set to the docker host's name.
  host: homeassistant.local
  # port is the listening port for UI and api
  # evcc will listen on all available interfaces
  port: 7070

interval: 10s # control cycle interval

# database configuration for persisting charge sessions and settings
# database:
#   type: sqlite
#   dsn: <path-to-db-file>

# sponsor token enables optional features (request at https://sponsor.evcc.io)
sponsortoken: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# telemetry enables aggregated statistics
#
# Telemetry allows collecting usage data (grid and green energy, charge power).
# Data is aggregated, no individual charging sessions are tracked. The collected,
# anonymous data can be retrieved using https://api.evcc.io.
#
# See https://github.com/evcc-io/evcc/pull/4343 or details.
#
# For time being, this is only available to sponsors, hence data is associated with
# the sponsor token's identity.
#
# telemetry: true

# log settings
log: info
levels:
  site: debug
  lp-1: debug
  lp-2: debug
  cache: error
  db: error

# modbus proxy for allowing external programs to reuse the evcc modbus connection
# each entry will start a proxy instance at the given port speaking Modbus TCP and
# relaying to the given modbus downstream device (either TCP or RTU, RS485 or TCP)
modbusproxy:
  #  - port: 5200
  #    uri: solar-edge:502
  #    # rtu: true
  #    # readonly: true

# meter definitions
# name can be freely chosen and is used as reference when assigning meters to site and loadpoints
# for documentation see https://docs.evcc.io/docs/devices/meters



meters:
- name: my_meter
  mqtt_sensor: sensor.smartmeter_netzlast
  type: custom
  power: # power (W)
    source: sensor.smartmeter_netzlast
  
  
  
- type: template
  template: fronius-solarapi-v1
  usage: pv
  host: 192.168.x.x
  name: pv2
- type: template
  template: fronius-solarapi-v1
  usage: pv
  host: 192.168.x.x
  name: pv3


# charger definitions
# name can be freely chosen and is used as reference when assigning charger to vehicle
# for documentation see https://docs.evcc.io/docs/devices/chargers
chargers:
- type: template
  template: go-e-v3
  host: 192.168.x.x
  name: wallbox5


# vehicle definitions
# name can be freely chosen and is used as reference when assigning vehicle to loadpoint
# for documentation see https://docs.evcc.io/docs/devices/vehicles
vehicles:
- type: template
  template: cupra
  title: Cupra Born
  user: xxx
  password: xxx
  capacity: 58
  phases: 3
  icon: car
  cache: 2m
  mode: minpv
  minSoc: 20
  targetSoc: 80
  minCurrent: 6
  maxCurrent: 16
  name: ev4


# site describes the EVU connection, PV and home battery
site:
  title: Zuhause
  meters:
    grid: grid1
    pv:
    - pv2
    - pv3


loadpoints:
- title: Garage
  charger: wallbox5
  mode: minpv
  phases: 3
  mincurrent: 6
  maxcurrent: 16
  resetOnDisconnect: true


# tariffs are the fixed or variable tariffs
tariffs:
  currency: EUR # three letter ISO-4217 currency code (default EUR)
  grid:
    # either static grid price (or price zones)

    type: awattar
    region: at # optional, choose at for Austria
    charges: 0.07284 # optional, additional charges per kWh
    tax: 0.23 # optional, additional tax (0.1 for 10%)

  feedin:
    # rate for feeding excess (pv) energy to the grid
    type: fixed
    price: 0.14457 # EUR/kWh
  co2:
    # co2 tariff provides co2 intensity forecast and is for co2-optimized target charging if no variable grid tariff is specified
    # type: grünstromindex # GrünStromIndex (Germany only)
    # zip: <zip>

    # type: electricitymaps # https://app.electricitymaps.com/map
    # uri: <uri>
    # token: <token>
    # zone: DE

# mqtt message broker
mqtt:
  # broker: localhost:1883
  # topic: evcc # root topic for publishing, set empty to disable
  # user:
  # password:

# influx database
influx:
  # url: http://localhost:8086
  # database: evcc
  # user:
  # password:

# eebus credentials
eebus:
  # uri: # :4712
  # interfaces: # limit eebus to specific network interfaces
  # - en0
  # certificate: # local signed certificate, required, can be generated via `evcc eebus-cert`
  #   public: # public key
  #   private: # private key

# push messages
messaging:
  events:
    start: # charge start event
      title: Charge started
      msg: Started charging in "${mode}" mode
    stop: # charge stop event
      title: Charge finished
      msg: Finished charging ${chargedEnergy:%.1fk}kWh in ${chargeDuration}.
    connect: # vehicle connect event
      title: Car connected
      msg: "Car connected at ${pvPower:%.1fk}kW PV"
    disconnect: # vehicle connected event
      title: Car disconnected
      msg: Car disconnected after ${connectedDuration}
    soc: # vehicle soc update event
      title: Soc updated
      msg: Battery charged to ${vehicleSoc:%.0f}%
    guest: # vehicle could not be identified
      title: Unknown vehicle
      msg: Unknown vehicle, guest connected?
  services:
  # - type: pushover
  #   app: # app id
  #   recipients:
  #   - # list of recipient ids
  # - type: telegram
  #   token: # bot id
  #   chats:
  #   - # list of chat ids
  # - type: email
  #   uri: smtp://<user>:<password>@<host>:<port>/?fromAddress=<from>&toAddresses=<to>
  # - type: ntfy
  #   uri: https://<host>/<topics>
  #   priority: <priority>
  #   tags: <tags>

MQTT-Sensor "Smartmeter_Netzlast" is defined in config.yaml, entity is showing the correct value.

I just have no clue anymore what to do, maybe i should sleep a night over it since today was already extremely busy maybe not having the brain for troubleshooting today... :/

[VolkerK62]

Have a look here https://docs.evcc.io/docs/reference/plugins#mqtt-lesenschreiben
it should be like that

- name: my_meter
  type: custom
  power: # power (W)
    source: mqtt
    topic: sensor.smartmeter_netzlast      

you must check, if the topic ist okay

and (i am not sure) the mqtt-broker must be configured in the yaml

[VolkerK62]

.... and, due to your site, the name must be "grid1" instead of "my_meter"

[maskedfighter]

Ok i think i am a step closer, but i just can´t make up my mind about how i implement the right mqtt topic as i am completely new to mqtt (never worked with it before). "Sensor.smartmeter_netzlast" gives me W momentarely consumed or returned to grid as a wether positive or negative number, but evcc log says its outdate (10sec).
I think my main problem is that i don´t exactly know how to listen to that specific topic.

[maskedfighter]

I think i found the problem. Energy meter is grabbing all data from my KAIFA smartmeter and publishing all data (1.7.0, 2.7.0, 1.8.0, 2.8.0, 16.7.0 and so on) combined into one topic and i don´t know how to extract/isolate that one power measurement value. Is there no way that i just can pick the already working sensor in HA and just configure it in evcc? That´s not how it works right?

[andig]

Is the topic json? The use the jq setting to select the relevant part.

[maskedfighter]

it is json yes. i just published the value isolated via mqtt (configured in config.yaml). It worked just fine, but it may be a simpler way to archive the same. I just didn´t look any further into this.

[IamMattM]

@andig @maskedfighter

I am also trying to figure out how to best parse mqtt json messages using "jq":

For example, the following evcc.yaml extract does not seem to pick up the 10s-published meter power value:

`meters:

• name: grid
  type: custom
  power:
  source: mqtt
  topic: glow/4417----60B4/SENSOR/electricitymeter
  jq: .power.value
  timeout: 30s # don't accept values older than timeout
  scale: 1.0
  `

Do you see any issues with my syntax in the evcc.yaml versus the way the json message is formatted whjen consulting MQTT explorer ?

Note: evcc mqtt client is correctly attached to mosquitto mqtt broker and is publishing its data.
The above mqtt meter power.value is correctrly displayed through HomeAssistant mqtt service.

The evcc reported errors:

[IamMattM]

I have just figured out what was wrong : the jq query needed to change to ".electricity.power.value"
Was missing the ".electricity"

EVCC now showing grid power in kw correctly:

[IamMattM]

I changed the scale factor from 1.0 to 1000 as the U.I seems to display in Watts and the json value was in kwH
Still working things out...the EVCC gridPower info seems to only be published every 1-1/2 to 2mins but the MQTT data is fed to the mqtt brtoker every 10seconds.

[maskedfighter]

i figured it out, what i did is just to publish the sensor value itself at my chosen topic (smartmeter/power). Snippet from configuration.yaml:

automation:
  - alias: Publish Sensor Value
    trigger:
      platform: state
      entity_id: sensor.16_7_0
    action:
      - service: mqtt.publish
        data_template:
          topic: "smartmeter/power"
          payload: "{{ states('sensor.16_7_0') }}"

Furthermore i just implemented it into evcc.yaml as mentioned in the evcc documentation, snippet from evcc.yaml:

meters:
- name: grid1
  type: custom
  power: # power (W)
    source: mqtt
    topic: smartmeter/power

[maskedfighter]

This is the device i bought which works great and rockstable: https://cms.shrdzm.com/produkt/smartmeter-modul/
You can also diy like shown in the tutorial given in your link, but unless u have a spare raspberryPi lying around, it would be cheaper and less work to buy the module. Are you planning to include evcc in home assistant or do you want it to run standalone/docker?

[movieaddicted86]

That´s neat :D less cabeling in the power distribution box and easier to set it up - thx. Well good question - i would run it as a container, as i have most of my apps in containers. I would also configure grafana and a compatible DB. But again, i am open for some insperation. Most important i would like to test if i could load my honda e with the "lade-ziegel"/"loading-brik" over my pv, when i generates excess. And second got a overview of the power consumption/production.

What´s your setup?

[maskedfighter]

It´s just plug and play basically, yes. One thing that i struggled is that this module pushes all given data from the smartmeter at once, so i was not able to "isolate" just the value i needed for evcc, which i was able to bypass bc i run my evcc instance in home assistant, where i can simply push the needed, isolated sensor value via mqtt to get it working with evcc. If you plan on using evcc outside of HA there is another way to implement data provided by "Volkszaehler", a addon implemented on the smartmeter module. Since evcc supports "volkszaehler" this should work fine.
In short, what i want to say is that you will most likely get the module and evcc running and communicating even without HA installed. Just be sure that all your hardware (WB, inverter, smartmeter) is supported by evcc then it should work.
I do have two Fronius inverters, a "go-e Charger" WB and the mentioned smartmeter readout-module, all running on a RpPi4 with home assistant, where i also retrieve all the data from all kinds of sensors to have a nice looking overview about our energy flow. The "energy" tab in HA is pretty neat since it collects all data from inverter, smartmeter, etc. to give you pretty nice looking and informing diagrams about your energy-household, produced energy and so on. It´s maybe not as advanced, or better say, sophisticated as Grafana, but then there is always the question to everyone how much data you really need and what you wanna do with it and why you want to grab and collect it... As ever with databases.. :)

Some screenshots of my recent setup in HA.

[movieaddicted86]

Hi,

i am pretty amazed about your setup. After a few hours of reading and enjoying the youtube-academy ;) i finally got all the hardware.
Deye Micro inverters, Go-e Gemini 11kwh wallbox and the smartmeter moduel to read out the data you've breen recommended.

I installed HA on docker, but this type of isntallation does not supports addons, so no integreated evcc. I will now try to install all the needed components seperated and hoping i will get all working together.

Regarding your HA setup, what addons etc. have you installed? And can you share your evcc yaml so i can get an idea of how to configure all?

Thanks in advance.
movie

[maskedfighter]

Sent you a mail :)

[maskedfighter]

Another question, is there a way to show SOC of my car when no charger is connected?

[maskedfighter]

Thx, lets see, if i just update SOC every 15 or even 30min it would be quite enough, shouldnt be big of a deal i guess.

[VolkerK62]

Always? I think it is too much. I have it at 4 hours

[maskedfighter]

have to see, maybe it´s enough every hour or less, i just dont want to look at the (really bad) my-cupra app to see SOC..

[StefanSchoof]

If the Cupra does it the same way as VW (the vehicle send the data on change to the cloud and a request does not reach the vehicle) you should be fine. But there are other, where the request is routed to the car and the car need to answer it by starting some kind of process and this sometimes affects the 12 V or car battery

[maskedfighter]

Cupra Born is basically a prettier ID.3, so i would guess it´s the same :)