Skip to content

CLI tool to upload historical photovoltaics data to InfluxDB

License

Notifications You must be signed in to change notification settings

loebse/csv-importer

 
 

Repository files navigation

Continuous integration wakatime

CSV importer

Import CSV with photovoltaic data and push it to InfluxDB for use with SOLECTRUS.

Requirements

  • SOLECTRUS installed and running
  • CSV files in one of the following supported formats:
    • SENEC (downloaded from mein-senec.de)
    • Sungrow (downloaded from portaleu.isolarcloud.com)

Usage

  • Login to your host machine where SOLECTRUS is running
  • CD into the folder where the .env of SOLECTRUS file is located
  • Create a folder csv and put the CSV files into it (subfolders allowed)
  • Run the following command:
docker run -it --rm \
           --env-file .env \
           --mount type=bind,source="$PWD/csv",target=/data,readonly \
           --network=solectrus_default \
           ghcr.io/solectrus/csv-importer

(Name of the network may vary, see docker network ls)

This imports all CSV files from the folder ./csv (it uses $PWD because Docker requires an absolute path here) and pushes them to your InfluxDB. The process is idempotent, so you can run it multiple times without any harm.

Beware of caching issues

If the import is performed after SOLECTRUS has already been used, caching issues may occur, meaning that older periods will not be displayed. In this case, the Redis cache must be flushed once after the import:

docker exec -it solectrus-redis-1 redis-cli FLUSHALL

(Name of the Redis container may vary, see docker ps)

Check the .env variable INSTALLATION_DATE. This must be set to the day your PV system was installed.

SENEC: Dealing with missing wallbox measurements

The CSV data from mein-senec.de is not complete, there are no measurements for the wallbox. To get around this, wallbox charges are estimated using the following formula:

wallbox_charge_power =   inverter_power (Stromerzeugung)
                       + grid_power_plus (Netzbezug)
                       + bat_power_minus (Akkuentnahme)
                       - grid_power_minus (Netzeinspeisung)
                       - house_power (Stromverbrauch)
                       - bat_power_plus (Akkubeladung)

Please note that this method appears to be ineffective for processing CSV files that were created from July 2022 onwards. This is because wallbox charges are now being included in the overall house consumption since that time. Therefore, it seems that there is currently no way to import wallbox measurements.

The senec-collector does not have this problem, as it obtains the wallbox measurements directly.

Configuration

The following environment variables can be used to configure the importer:

Variable Description Default
INFLUX_HOST Hostname of InfluxDB
INFLUX_SCHEMA Schema (http/https) of InfluxDB http
INFLUX_PORT Port of InfluxDB 8086
INFLUX_TOKEN_WRITE or INFLUX_TOKEN Token for InfluxDB (requires write permissions)
INFLUX_ORG Organization for InfluxDB
INFLUX_BUCKET Bucket for InfluxDB
INFLUX_MEASUREMENT Measurement for InfluxDB
INFLUX_OPEN_TIMEOUT Timeout for InfluxDB connection (in seconds) 30
INFLUX_READ_TIMEOUT Timeout for InfluxDB read (in seconds) 30
INFLUX_WRITE_TIMEOUT Timeout for InfluxDB write (in seconds) 30
IMPORT_FOLDER Folder where CSV files are located /data
IMPORT_PAUSE Pause after each imported file (in seconds) 0
TZ Time zone to use when parsing times CET

License

Copyright (c) 2020-2023 Georg Ledermann, released under the MIT License

Many thanks to these incredible people for improving this project:

About

CLI tool to upload historical photovoltaics data to InfluxDB

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Ruby 96.9%
  • Dockerfile 3.1%