Merge remote-tracking branch 'tbnobody/OpenDTU/master'

helgeerbe · Jan 20, 2023 · 9a7a0d2 · 9a7a0d2
2 parents ddb6346 + 1d3acb4
commit 9a7a0d2
Show file tree

Hide file tree

Showing 55 changed files with 1,191 additions and 226 deletions.
diff --git a/README.md b/README.md
@@ -66,7 +66,7 @@ Sends text raw data as difined in VE.Direct spec.
 **TSUN compatibility remark:**
 Compatibility with OpenDTU seems to be related to serial numbers. Current findings indicate that TSUN inverters with a serial number starting with "11" are supported, whereby inverters with a serial number starting with "10" are not.
 Firmware version seems to play not a significant role and cannot be read from the stickers. For completeness, the following firmware version have been reported to work with OpenDTU:
-* v1.0.10 TSOL-M800 (DE)
+* v1.0.8, v1.0.10 TSOL-M800 (DE)
 * v1.0.12 TSOL-M1600
 
 ## Features for end users
@@ -86,7 +86,7 @@ Firmware version seems to play not a significant role and cannot be read from th
 * Ve.Direct interface (via web-interface, REST-api, or MQTT)
 * Ethernet support
 * Prometheus API endpoint (/api/prometheus/metrics)
-* English and german web interface
+* English, german and french web interface
 
 ## Features for developers
 * The microcontroller part
@@ -151,6 +151,7 @@ This can be achieved by copying one of the [env:....] sections from 'platformio.
 -DVICTRON_PIN_RX=22 
 ```
 It is recommended to make all changes only in the  'platformio_override.ini', this is your personal copy.
+You can also change  the pins by creating a custom [device profile](docs/DeviceProfiles.md).
 
 ## Flashing and starting up
 ### with Visual Studio Code
@@ -216,6 +217,9 @@ esptool.py --port /dev/ttyUSB0 --chip esp32 --before default_reset --after hard_
 #### Flash with ESP_Flasher (Windows)
 Users report that [ESP_Flasher](https://github.com/Jason2866/ESP_Flasher/releases/) is suitable for flashing OpenDTU on Windows.
 
+#### Flash with [ESP_Flasher](https://espressif.github.io/esptool-js/) - web version
+It is also possible to flash it via the web tools which might be more convenient and is platformindependent.
+
 ## First configuration
 * After the initial flashing of the microcontroller, an Access Point called "OpenDTU-*" is opened. The default password is "openDTU42".
 * Use a web browser to open the address [http://192.168.4.1](http://192.168.4.1)

diff --git a/docs/DeviceProfiles.md b/docs/DeviceProfiles.md
@@ -0,0 +1,56 @@
+# Device Profiles
+
+It is possible to change hardware settings like pin assignments or ethernet support using a json file. The json file can be uploaded using the configuration management in the web interface. Just select "Pin Mapping (pin_mapping.json)" in the recovery section.
+
+When the file is uploaded the ESP performs a reboot. This is required as the pin settings could have changed within the file. By default all the pin assignments are used as compiled into the firmware.
+
+To change the device profile, navigate to the "Device Manager" and selected the appropriate profile. You can see the current (Active) and the new (Selected) in assignment in the table below the combobox.
+
+## Structure of the json file
+
+```json
+[
+    {
+        "name": "Generic NodeMCU 38 pin",
+        "nrf24": {
+            "miso": 19,
+            "mosi": 23,
+            "clk": 18,
+            "irq": 16,
+            "en": 4,
+            "cs": 5
+        },
+        "eth": {
+            "enabled": false,
+            "phy_addr": -1,
+            "power": -1,
+            "mdc": -1,
+            "mdio": -1,
+            "type": -1,
+            "clk_mode": -1
+        }
+    },
+    {
+        "name": "Olimex ESP32-POE",
+        "nrf24": {
+            "miso": 15,
+            "mosi": 2,
+            "clk": 14,
+            "irq": 13,
+            "en": 16,
+            "cs": 5
+        },
+        "eth": {
+            "enabled": true,
+            "phy_addr": 0,
+            "power": 12,
+            "mdc": 23,
+            "mdio": 18,
+            "type": 0,
+            "clk_mode": 3
+        }
+    }
+]
+```
+
+The json file can contain multiple profiles. Each profile requires a name and different parameters. If one parameter is not set, the default value, as compiled into the firmware is used. The example above shows all the currently supported values. Others may follow. A sample file with a lot of boards can be found [here](pin_mapping.json). This means you can just flash the generic bin file and upload the pin_mapping.json file. Then you select your board and everything works hopyfully as expected.
diff --git a/docs/README.md b/docs/README.md
@@ -4,4 +4,5 @@ More detailed descriptions for some topics can be found here.
 
 ## [MQTT Topic Documentation](MQTT_Topics.md)
 ## [Web API Documentation](Web-API.md)
+## [Device Profile Documentation](DeviceProfiles.md)
 ## [Builds](builds/README.md)
diff --git a/docs/pin_mapping.json b/docs/pin_mapping.json
@@ -0,0 +1,122 @@
+[
+    {
+        "name": "Generic NodeMCU 38 pin",
+        "nrf24": {
+            "miso": 19,
+            "mosi": 23,
+            "clk": 18,
+            "irq": 16,
+            "en": 4,
+            "cs": 5
+        },
+        "eth": {
+            "enabled": false,
+            "phy_addr": -1,
+            "power": -1,
+            "mdc": -1,
+            "mdio": -1,
+            "type": -1,
+            "clk_mode": -1
+        }
+    },
+    {
+        "name": "Olimex ESP32-POE",
+        "nrf24": {
+            "miso": 15,
+            "mosi": 2,
+            "clk": 14,
+            "irq": 13,
+            "en": 16,
+            "cs": 5
+        },
+        "eth": {
+            "enabled": true,
+            "phy_addr": 0,
+            "power": 12,
+            "mdc": 23,
+            "mdio": 18,
+            "type": 0,
+            "clk_mode": 3
+        }
+    },
+    {
+        "name": "Olimex ESP32-EVB",
+        "nrf24": {
+            "miso": 15,
+            "mosi": 2,
+            "clk": 14,
+            "irq": 13,
+            "en": 16,
+            "cs": 17
+        },
+        "eth": {
+            "enabled": true,
+            "phy_addr": 0,
+            "power": 12,
+            "mdc": 23,
+            "mdio": 18,
+            "type": 0,
+            "clk_mode": 0
+        }
+    },
+    {
+        "name": "Generic NodeMCU 30 pin",
+        "nrf24": {
+            "miso": 19,
+            "mosi": 23,
+            "clk": 18,
+            "irq": 16,
+            "en": 17,
+            "cs": 5
+        },
+        "eth": {
+            "enabled": false,
+            "phy_addr": -1,
+            "power": -1,
+            "mdc": -1,
+            "mdio": -1,
+            "type": -1,
+            "clk_mode": -1
+        }
+    },
+    {
+        "name": "WT32-ETH01",
+        "nrf24": {
+            "miso": 4,
+            "mosi": 2,
+            "clk": 32,
+            "irq": 33,
+            "en": 14,
+            "cs": 15
+        },
+        "eth": {
+            "enabled": true,
+            "phy_addr": 1,
+            "power": 16,
+            "mdc": 23,
+            "mdio": 18,
+            "type": 0,
+            "clk_mode": 3
+        }
+    },
+    {
+        "name": "LILYGO TTGO T-Internet-POE",
+        "nrf24": {
+            "miso": 2,
+            "mosi": 15,
+            "clk": 14,
+            "irq": 34,
+            "en": 12,
+            "cs": 4
+        },
+        "eth": {
+            "enabled": true,
+            "phy_addr": 0,
+            "power": -1,
+            "mdc": 23,
+            "mdio": 18,
+            "type": 0,
+            "clk_mode": 3
+        }
+    }
+]
diff --git a/include/Configuration.h b/include/Configuration.h
@@ -19,14 +19,16 @@
 #define MQTT_MAX_PASSWORD_STRLEN 64
 #define MQTT_MAX_TOPIC_STRLEN 32
 #define MQTT_MAX_LWTVALUE_STRLEN 20
-#define MQTT_MAX_ROOT_CA_CERT_STRLEN 2048 
+#define MQTT_MAX_ROOT_CA_CERT_STRLEN 2560
 
 #define INV_MAX_NAME_STRLEN 31
 #define INV_MAX_COUNT 10
 #define INV_MAX_CHAN_COUNT 4
 
 #define CHAN_MAX_NAME_STRLEN 31
 
+#define DEV_MAX_MAPPING_NAME_STRLEN 31
+
 #define JSON_BUFFER_SIZE 6144
 
 struct CHANNEL_CONFIG_T {
@@ -92,6 +94,8 @@ struct CONFIG_T {
 
     char Security_Password[WIFI_MAX_PASSWORD_STRLEN + 1];
     bool Security_AllowReadonly;
+
+    char Dev_PinMapping[DEV_MAX_MAPPING_NAME_STRLEN + 1];
 };
 
 class ConfigurationClass {

diff --git a/include/PinMapping.h b/include/PinMapping.h
@@ -0,0 +1,42 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#pragma once
+
+#include <Arduino.h>
+#include <stdint.h>
+#include <ETH.h>
+
+#define PINMAPPING_FILENAME "/pin_mapping.json"
+
+#define MAPPING_NAME_STRLEN 31
+
+struct PinMapping_t {
+    char name[MAPPING_NAME_STRLEN + 1];
+    int8_t nrf24_miso;
+    int8_t nrf24_mosi;
+    int8_t nrf24_clk;
+    int8_t nrf24_irq;
+    int8_t nrf24_en;
+    int8_t nrf24_cs;
+    int8_t eth_phy_addr;
+    bool eth_enabled;
+    int eth_power;
+    int eth_mdc;
+    int eth_mdio;
+    eth_phy_type_t eth_type;
+    eth_clock_mode_t eth_clk_mode;
+};
+
+class PinMappingClass {
+public:
+    PinMappingClass();
+    bool init(const String& deviceMapping);
+    PinMapping_t& get();
+
+    bool isValidNrf24Config();
+    bool isValidEthConfig();
+
+private:
+    PinMapping_t _pinMapping;
+};
+
+extern PinMappingClass PinMapping;
diff --git a/include/WebApi.h b/include/WebApi.h
@@ -6,6 +6,7 @@
 #include "WebApi_dtu.h"
 #include "WebApi_eventlog.h"
 #include "WebApi_firmware.h"
+#include "WebApi_device.h"
 #include "WebApi_inverter.h"
 #include "WebApi_limit.h"
 #include "WebApi_maintenance.h"
@@ -37,6 +38,7 @@ class WebApiClass {
     AsyncEventSource _events;
 
     WebApiConfigClass _webApiConfig;
+    WebApiDeviceClass _webApiDevice;
     WebApiDevInfoClass _webApiDevInfo;
     WebApiDtuClass _webApiDtu;
     WebApiEventlogClass _webApiEventlog;

diff --git a/include/WebApi_config.h b/include/WebApi_config.h
@@ -11,6 +11,7 @@ class WebApiConfigClass {
 private:
     void onConfigGet(AsyncWebServerRequest* request);
     void onConfigDelete(AsyncWebServerRequest* request);
+    void onConfigListGet(AsyncWebServerRequest* request);
     void onConfigUploadFinish(AsyncWebServerRequest* request);
     void onConfigUpload(AsyncWebServerRequest* request, String filename, size_t index, uint8_t* data, size_t len, bool final);
 

diff --git a/include/WebApi_device.h b/include/WebApi_device.h
@@ -0,0 +1,16 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#pragma once
+
+#include <ESPAsyncWebServer.h>
+
+class WebApiDeviceClass {
+public:
+    void init(AsyncWebServer* server);
+    void loop();
+
+private:
+    void onDeviceAdminGet(AsyncWebServerRequest* request);
+    void onDeviceAdminPost(AsyncWebServerRequest* request);
+
+    AsyncWebServer* _server;
+};
diff --git a/include/WebApi_errors.h b/include/WebApi_errors.h
@@ -81,4 +81,7 @@ enum WebApiError {
     PowerBase = 11000,
     PowerSerialZero,
     PowerInvalidInverter,
+
+    HardwareBase = 12000,
+    HardwarePinMappingLength,
 };
diff --git a/include/defaults.h b/include/defaults.h
@@ -79,6 +79,8 @@
 #define MQTT_HASS_TOPIC "homeassistant/"
 #define MQTT_HASS_INDIVIDUALPANELS false
 
+#define DEV_PINMAPPING ""
+
 #define VEDIRECT_ENABLED false
 #define VEDIRECT_UPDATESONLY true
 #define VEDIRECT_POLL_INTERVAL 5
diff --git a/lib/Hoymiles/src/inverters/InverterAbstract.cpp b/lib/Hoymiles/src/inverters/InverterAbstract.cpp
@@ -108,12 +108,12 @@ void InverterAbstract::clearRxFragmentBuffer()
 void InverterAbstract::addRxFragment(uint8_t fragment[], uint8_t len)
 {
     if (len < 11) {
-        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) fragment too short\n", __FILE__, __LINE__);
+        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) fragment too short\r\n", __FILE__, __LINE__);
         return;
     }
 
     if (len - 11 > MAX_RF_PAYLOAD_SIZE) {
-        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) fragment too large\n", __FILE__, __LINE__);
+        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) fragment too large\r\n", __FILE__, __LINE__);
         return;
     }
 

diff --git a/lib/Hoymiles/src/parser/AlarmLogParser.cpp b/lib/Hoymiles/src/parser/AlarmLogParser.cpp
@@ -15,7 +15,7 @@ void AlarmLogParser::clearBuffer()
 void AlarmLogParser::appendFragment(uint8_t offset, uint8_t* payload, uint8_t len)
 {
     if (offset + len > ALARM_LOG_PAYLOAD_SIZE) {
-        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) stats packet too large for buffer (%d > %d)\n", __FILE__, __LINE__, offset + len, ALARM_LOG_PAYLOAD_SIZE);
+        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) stats packet too large for buffer (%d > %d)\r\n", __FILE__, __LINE__, offset + len, ALARM_LOG_PAYLOAD_SIZE);
         return;
     }
     memcpy(&_payloadAlarmLog[offset], payload, len);

diff --git a/lib/Hoymiles/src/parser/DevInfoParser.cpp b/lib/Hoymiles/src/parser/DevInfoParser.cpp
@@ -37,7 +37,7 @@ void DevInfoParser::clearBufferAll()
 void DevInfoParser::appendFragmentAll(uint8_t offset, uint8_t* payload, uint8_t len)
 {
     if (offset + len > DEV_INFO_SIZE) {
-        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) dev info all packet too large for buffer\n", __FILE__, __LINE__);
+        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) dev info all packet too large for buffer\r\n", __FILE__, __LINE__);
         return;
     }
     memcpy(&_payloadDevInfoAll[offset], payload, len);
@@ -53,7 +53,7 @@ void DevInfoParser::clearBufferSimple()
 void DevInfoParser::appendFragmentSimple(uint8_t offset, uint8_t* payload, uint8_t len)
 {
     if (offset + len > DEV_INFO_SIZE) {
-        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) dev info Simple packet too large for buffer\n", __FILE__, __LINE__);
+        Hoymiles.getMessageOutput()->printf("FATAL: (%s, %d) dev info Simple packet too large for buffer\r\n", __FILE__, __LINE__);
         return;
     }
     memcpy(&_payloadDevInfoSimple[offset], payload, len);