Latest version of controller code only supports V2 of influxdb

[jbuszkie]

There are a lot of us out there that don't want to/ can't upgrade to V2 of influxdb.

It would be really nice if we had the option of staying with the 1.x implementation.

[stuartpittaway]

Hello, if you are running v1.8 or newer of Influx, it should support the forward compatibility interface.

https://docs.influxdata.com/influxdb/v1.8/tools/api/#apiv2write-http-endpoint

In InfluxDB 2.0 uses API Tokens to access the platform and all its capabilities. InfluxDB v1.x uses a username and password combination when accessing the HTTP APIs. Use the Token schema to provide your InfluxDB 1.x username and password separated by a colon (:).

For example: username:password.

[jbuszkie]

When I get my esp32 version of the controller up and running, I'll have to try this.

[jbuszkie]

If someone can test this and verify, then we can close this (hopefully) non-issue

[stuartpittaway]

Has anyone tested this?

[jbuszkie]

Has anyone tested this?

Not me.. I don't have an ESP32 controller yet..

[virtuvas]

hello,

just tested it with influxdb 1.8 on a windows10 machine.
Logs each cell b,e,i,v values
to my understanding:
e is external Temp
i is internal Temp
v is voltage
b is empty and haven't got a clue at what it's meant to measure!

So assuming that's what it's meant to log, it's working :-)
good job!

BTW, BMS is connected via CAN to Victron (raspberry pi 3B+ running VenusOS 2.80Large) and through that I also get bank voltage/current/watts and all warnings from the system as sent to Victron. Wonder if they should also be sent to be logged directly from the bms influxDB interface?
Is there any documentation on what is actually being sent to influx?

cheers
V.

[virtuvas]

still works without crashes in diyBMS 10h later. Excuse the regular jumping about it's 4 lifepo4 modules connected to one half dead 18560 cell each with some tiny cables, when it starts balancing all hell breaks loose.
I'll come back on this on what would be nice to be sending to influxdb as I think v and temps are not really enough.

[virtuvas]

hm, influxDB works fine, but now controller started rebooting. Was away from the pc, will try to note down how often/when it reboots. Sadly cannot easily pick it up/record it from influx...
Uptime was in days before starting influx, now it's in hours, guess it's related.
One thing I'd like to see is being able to set the interval for sending data to influx. Now it is like every few secs, don't see any reason (other than debugging) to have such dataflow. Would be happy with 60 secs as I do for all my NMEA2000 data logged via Victron.

V.

[virtuvas]

looks like circa 2h between controller reboots.
b is inBypass which I guess is either false or true.

[stuartpittaway]

Hi, there was a bug on the influxdb which was fixed a few months. Are you running the latest version?

I really could do with the latest version being applied and then the serial port log files being captured when the reset happens.

[virtuvas]

yes Stuart, I'm aware of that, running the latest version (loaded it 4-5days ago):

Host name:DIYBMS-004CE8AC

Processor: ESP32

Version: 1dde4670aecf5ea29f773a964928492e5fb5d035

Compiled: 2021-12-06T15:10:39.269Z

Language: language

SDK Version:v3.3.5-1-g85c43024c

Min free Heap:86360

Free heap:102404

Heap size:296744

for now I'm trying to log reboots (not easy when not working on my desk though...), changed to a powerbrick for the ESP32 (plan to get it running from the pcb power input tomorrow if I have time)
If you can explain how to do a serial port log, I'll be happy to help.

since yesterday I had (at least recorded maybe a few more I missed):

19:15
20:03
22:15
03:50
09:38
11:15
13:03
mostly doing bugger all, did try to charge the 4 18650cells a bit, now just let it run and see

I'd also be happier if we could sent to influx on set intervals (like 30s or 1m) - don't want all this traffic from the RUT240 onboard nor the data charges... Shall I put up a new issue for that?

cheers

V.

[stuartpittaway]

Yes please raise a new issue for the influx timing.

[virtuvas]

OK, will do, now how do I log serial port log? any pointers?

[stuartpittaway]

Probably just connecting a laptop to the USB port on the ESP32 - then use a serial terminal like PUTTY to log everything to a file.

Then just wait until something goes wrong!

More info about it here...
https://arduino-esp8266.readthedocs.io/en/latest/faq/a02-my-esp-crashes.html

[virtuvas]

lol, was hoping for something more advanced than waiting looking at the putty screen tbh 😁

anyway, only took 50mins before hearing the relay clicking (ie rebooting!)
wifi working, canbus connected via PCAN-USB to raspberry. Board fully populated bar RS chip (just ordered that)
Hope it helps (XXXXXXXed the webaddress of my influxdb server just in case):

[I][main.cpp:1459] influxdb_task(): Influx task
[D][influxdb.cpp:140] influx_task_action(): Create new client
[I][influxdb.cpp:177] influx_task_action(): Influx host XXXXXXXXXXXXXX port 8086
[D][influxdb.cpp:73] influxdb_onConnect(): Influx connected
[D][influxdb.cpp:126] influxdb_onConnect(): Influx data sent
[D][influxdb.cpp:47] influxdb_onData(): Influx data received
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x40151c36  PS      : 0x00060930  A0      : 0x8014a426  A1      : 0x3ffd9420
A2      : 0x3ffec4b4  A3      : 0x00450008  A4      : 0x0000004e  A5      : 0x00000000
A6      : 0x3ffd8934  A7      : 0x00000000  A8      : 0x00740ee8  A9      : 0x00741a20
A10     : 0x00000000  A11     : 0x0000010b  A12     : 0x00741915  A13     : 0x00000b38
A14     : 0x00060b20  A15     : 0x00000000  SAR     : 0x00000019  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00450018  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff

ELF file SHA256: 0000000000000000

Backtrace: 0x40151c36:0x3ffd9420 0x4014a423:0x3ffd9450 0x400e62ca:0x3ffd9470 0x40149564:0x3ffd9490 0x40092ce2:0x3ffd94c0

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8



                _          __
    _|  o      |_)  |\/|  (_
   (_|  |  \/  |_)  |  |  __)
           /
[I][main.cpp:3286] setup(): CONTROLLER - ver:1dde4670aecf5ea29f773a964928492e5fb5d035 compiled 2021-12-06T15:10:39.269Z
[I][main.cpp:3291] setup(): ESP32 Chip model = 1, Rev 1, Cores=2, Features=50

[stuartpittaway]

Thanks

python decoder.py -p ESP32 -e diybms_controller_firmware_espressif32_esp32-devkitc.elf stack.txt -t ~/.platformio/packages/toolchain-xtensa32

outputs...

stack:
0x40151c36: tcp_write at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:663
0x4014a423: tcp_bind at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:615 (discriminator 6)
0x400e62ca: victron_message_374_375_376_377() at /home/runner/work/diyBMSv4ESP32/diyBMSv4ESP32/ESPController/src/victron_canbus.cpp:107
0x40149564: vfs_fat_stat at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/fatfs/src/vfs_fat.c:457
0x40092ce2: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:143

[virtuvas]

way above my paygrade Stuart, if you want anything else let me know.
Found the 8000msec in main.cpp, will change it to 60000 for now anyway.
Good luck!

[virtuvas]

not sure how helpful (definitely not conclusive!) changed the 8sec to 60, system survived 3h before rebooting but was also playing with overcharging the cells to check the behaviour so not quite sure.

[virtuvas]

after a 4h session it rebooted at 04:18 and is up and running till now which is a record 9:30h (for me with influx running)
that is with 60sec intervals between posting to influx.
Another observation is that at around 5h running (circa 9:00am) victron can comms ceased... Raspberry doesn't see it at all, no data posted on venus.
Not sure if these two are related. Will at some point tonight disable influx and leave victron can running so that I can check relevance.
Posting a pic just in case

cheers

V.

[jbuszkie]

Dumb question... Did you try a different ESP32 yet? I have had an instance (on some other project) where one module behaved poorly.. I was scratching my head for a while and I tried a different ESP32 and it worked fine. So I gave the bad module an early, forced retirement! :-)

[virtuvas]

not dumb at all 😀

luckily I have a second ESP32 board ready and programmed on my desk - plan is to have one board running on the boat with the rpi3 and a second one running at home for testing with the second PEAK CAN on a r4
Will swap over and test tonight if it crashes!
However, it's passed 6h (no influxDB only CANBUS to Victron) without crashing so still testing.
max I've managed with influxDB running as 13h (but CANBUS comms died halfway through) So few things to test and eliminate!

V.

[virtuvas]

@stuartpittaway

issues threads re influx are getting slightly messy and mixed up (imho) not sure how to report so listing as concise as I can here my last three days observations:

• influxdb works with v1.8
• when influxdb is enabled (with stock 8sec interval) system will reboot within 2-3h on my 4S idle setup
• interval edited to 60sec system survives for 9-11h
• HOWEVER, venus can bus connection dies within 4-5h (influx still posts normally - raspberrypi looses diyBMS data)
• influxDB disabled, venus can bus connection fine for 24h and counting.

so influxdb activation affects venus can bus as well as causing system reboots.
Hope that helps.

cheers

V.

[virtuvas]

typical...
half an hour later I get this and canbus comms killed:

                _          __
    _|  o      |_)  |\/|  (_
   (_|  |  \/  |_)  |  |  __)
           /
[I][main.cpp:3294] setup(): CONTROLLER - ver:LocalCompile compiled 2021-12-18T19:47:13.230Z
[I][main.cpp:3299] setup(): ESP32 Chip model = 1, Rev 1, Cores=2, Features=50
[I][HAL_ESP32.cpp:233] ConfigureI2C(): Configure I2C
[D][HAL_ESP32.cpp:253] ConfigureI2C(): i2c conf value: 0
[D][HAL_ESP32.cpp:254] ConfigureI2C(): i2c conf.mode value: 1
[I][HAL_ESP32.cpp:294] ConfigureI2C(): Found TCA9534A
[I][HAL_ESP32.cpp:325] ConfigureI2C(): Found TCA6408
[D][HAL_ESP32.h:289] TouchScreenUpdate(): Touch = touch=0 pressure=128 x=0 y=3170
[I][main.cpp:3314] setup(): TFT screen is INSTALLED
[I][main.cpp:942] SetControllerState(): ** Controller changed state from Unknown to PowerUp **
[I][main.cpp:3336] setup(): LITTLEFS mounted, totalBytes=589824, usedBytes=98304
[I][main.cpp:241] mountSDCard(): Mounting SD card
[W][sd_diskio.cpp:101] sdWait(): Wait Failed
[E][sd_diskio.cpp:123] sdSelectCard(): Select Failed
[W][sd_diskio.cpp:505] ff_sd_initialize(): GO_IDLE_STATE failed
[E][sd_diskio.cpp:775] sdcard_mount(): f_mount failed: (3) The physical drive cannot work
[W][sd_diskio.cpp:101] sdWait(): Wait Failed
[E][sd_diskio.cpp:123] sdSelectCard(): Select Failed
[E][main.cpp:260] mountSDCard(): Card Mount Failed
[I][HAL_ESP32.cpp:141] ConfigureCAN(): CAN driver installed.  Filter=1621098496 Mask=6291455
[I][HAL_ESP32.cpp:151] ConfigureCAN(): CAN driver started
[D][settings.cpp:20] ReadConfig(): ReadConfig diybms
[D][settings.cpp:42] ReadConfig(): checksum verified
[I][main.cpp:3360] setup():
[D][main.cpp:200] SetupRS485(): Setup RS485
[D][main.cpp:187] ConfigureRS485(): Configure RS485
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 0=153
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 1=153
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 2=153
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 3=153
Press SPACE BAR to enter terminal based configuration........[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
.....[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
...skipped
[D][settings.cpp:20] ReadConfig(): ReadConfig diybmswifi
[D][settings.cpp:42] ReadConfig(): checksum verified
[I][main.cpp:942] SetControllerState(): ** Controller changed state from PowerUp to Stabilizing **
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][main.cpp:759] tca6408_isr_task(): tca6408_isr
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 2 - STA_START
[I][main.cpp:1430] connectToWifi(): Hostname: DIYBMS-004CE8AC, current state 255
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 4 - STA_CONNECTED
[D][WiFiGeneric.cpp:374] _eventCallback(): Event: 7 - STA_GOT_IP
[D][WiFiGeneric.cpp:419] _eventCallback(): STA IP: 192.168.178.121, MASK: 255.255.255.0, GW: 192.168.178.1
[I][main.cpp:1517] onWifiConnect(): Wi-Fi status=3
[I][main.cpp:1519] onWifiConnect(): Request NTP from time.google.com
[I][DIYBMSServer.cpp:2710] StartServer(): Start Web Server complete
[I][ArduinoOTA.cpp:141] begin(): OTA server at: DIYBMS-004CE8AC.local:3232
[I][main.cpp:1553] onWifiConnect(): mDNS responder started
[I][main.cpp:1558] onWifiConnect(): You can access DIYBMS interface at http://DIYBMS-004CE8AC.local or http://192.168.178.121
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[I][Rules.cpp:142] SetError(): Set error 4
[I][main.cpp:1312] rules_task(): Set Relay 1=1
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 1=255
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[I][Rules.cpp:142] SetError(): Set error 4
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[W][AsyncTCP.cpp:969] _poll(): rx timeout 4
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[W][victron_canbus.cpp:188] victron_message_351(): active_errors=0
[I][main.cpp:942] SetControllerState(): ** Controller changed state from Stabilizing to Running **
[I][main.cpp:1312] rules_task(): Set Relay 1=0
[D][HAL_ESP32.cpp:87] SetOutputState(): SetOutputState 1=153
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch 
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch 
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch 
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[W][AsyncTCP.cpp:969] _poll(): rx timeout 4
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[W][AsyncTCP.cpp:969] _poll(): rx timeout 4
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[D][tft.cpp:59] TFTScreenTouchInterrupt(): Touch
[I][tft.cpp:388] tftwakeup_task(): Wake up screen
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout

ignore all the touchscreen events, was just checking thing was still working...

following a raspberry pi reboot I get lots of :

[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][main.cpp:2569] victron_canbus_rx(): CANBUS timeout
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message
[E][victron_canbus.cpp:30] send_canbus_message(): Fail to queue message

Only solution seems to be to reboot ESP32.
more testing will follow!

[stuartpittaway]

I'm wondering if this is an issue with the raspberry pi version of cerbo? With a real device I've left it running for days without issue. I generally don't use the influxdb interface though, only the MQTT one.

[virtuvas]

@stuartpittaway an update:

was only using CANL and CANH from the P-CAN to the diyBMS. Added GND to pin3 (of the P-CAN). Now I can even unplug the USB and replug it (have to go to the rpi settings and re-edit can(2) settings) or unplug and replug on the diyBMS side with rpi picking it up straight away. So seems that CANBUS issues were mine. Influx issues most likely remain will continue testing and will revert to sending every 10secs to get to the problems faster... Will also try without the CANBUS connection for good measure.

Waiting for a GX device sometime next week from a friend for further testing, but looks like there is something going on in influxDB code/comms. So updated list of comments:

• influxdb works with v1.8
• when influxdb is enabled (with stock 8sec interval) system will reboot within 2-3h on my 4S idle setup
• interval edited to 60sec system survives for 9-11h

Hope that helps.

cheers

[virtuvas]

that was quick rebooting again:

[D][influxdb.cpp:73] influxdb_onConnect(): Influx connected
[D][influxdb.cpp:129] influxdb_onConnect(): Influx data sent
[D][influxdb.cpp:47] influxdb_onData(): Influx data received
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC      : 0x4016373e  PS      : 0x00060f30  A0      : 0x8015bf2e  A1      : 0x3ffd9430
A2      : 0x3ffec46c  A3      : 0x00450008  A4      : 0x0000004e  A5      : 0x00000000  
A6      : 0x3ffd88fc  A7      : 0x00000000  A8      : 0x00700d29  A9      : 0x00701861
A10     : 0x00000000  A11     : 0x0000024a  A12     : 0x00701617  A13     : 0x00000b38  
A14     : 0x00060f20  A15     : 0x00000000  SAR     : 0x00000019  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00450018  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

ELF file SHA256: 0000000000000000

Backtrace: 0x4016373e:0x3ffd9430 0x4015bf2b:0x3ffd9460 0x400f168a:0x3ffd9480 0x4015b06c:0x3ffd94a0 0x40092cce:0x3ffd94d0
  #0  0x4016373e:0x3ffd9430 in tcp_output at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:1027
  #1  0x4015bf2b:0x3ffd9460 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:1765
  #2  0x400f168a:0x3ffd9480 in _tcp_recved_api(tcpip_api_call_data*) at .pio\libdeps\esp32-devkitc-debug\AsyncTCP\src/AsyncTCP.cpp:1036
  #3  0x4015b06c:0x3ffd94a0 in tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
  #4  0x40092cce:0x3ffd94d0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...
ets Jun  8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DOUT, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8



                _          __
    _|  o      |_)  |\/|  (_
   (_|  |  \/  |_)  |  |  __)
           /
[I][main.cpp:3294] setup(): CONTROLLER - ver:LocalCompile compiled 2021-12-21T10:16:42.460Z
[I][main.cpp:3299] setup(): ESP32 Chip model = 1, Rev 1, Cores=2, Features=50

[atanisoft]

[D][influxdb.cpp:47] influxdb_onData(): Influx data received
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x4016373e PS : 0x00060f30 A0 : 0x8015bf2e A1 : 0x3ffd9430
A2 : 0x3ffec46c A3 : 0x00450008 A4 : 0x0000004e A5 : 0x00000000
A6 : 0x3ffd88fc A7 : 0x00000000 A8 : 0x00700d29 A9 : 0x00701861
A10 : 0x00000000 A11 : 0x0000024a A12 : 0x00701617 A13 : 0x00000b38
A14 : 0x00060f20 A15 : 0x00000000 SAR : 0x00000019 EXCCAUSE: 0x0000001c
EXCVADDR: 0x00450018 LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0xffffffff

ELF file SHA256: 0000000000000000

Backtrace: 0x4016373e:0x3ffd9430 0x4015bf2b:0x3ffd9460 0x400f168a:0x3ffd9480 0x4015b06c:0x3ffd94a0 0x40092cce:0x3ffd94d0
#0 0x4016373e:0x3ffd9430 in tcp_output at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:1027
#1 0x4015bf2b:0x3ffd9460 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:1765
#2 0x400f168a:0x3ffd9480 in _tcp_recved_api(tcpip_api_call_data*) at .pio\libdeps\esp32-devkitc-debug\AsyncTCP\src/AsyncTCP.cpp:1036
#3 0x4015b06c:0x3ffd94a0 in tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
#4 0x40092cce:0x3ffd94d0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

This is a sign of the LwIP stack running out of buffers for TX/RX activities. Increasing the data publish rate from the default 8sec to 60sec should reduce the occurrence of this condition. There are a few other possibilities but it would need more in-depth testing.

[virtuvas]

thanks, well done that and went a bit further.
now complied the code in PlatformiO at a debug level 2 from the 5 it was (no idea if it will make any difference)
on the down side I'm posting a few more data now (voltagemVMax, voltagemVMin & PWMValue)
also shouldn't forget that I'm running a small test bank with 4S I guess a 16S will be 4times the data and with my extra 3 records almost 8times.

the only warnings I get now are these:

[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:962] _poll(): ack timeout 4
[W][AsyncTCP.cpp:969] _poll(): rx timeout 4
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL
[W][AsyncTCP.cpp:949] _poll(): pcb is NULL

See if it makes it overnight at 60sec intervals 🤞

V.

[virtuvas]

happy new year to all and an update 😁

redid all my test setup at 3S2P (3new and 3s/h 18650cells that is and a .7A engineroom light 12V), better wiring all around and a 2.7A powersupply to the terminals of the board.
influxDB Intervals at 8sec, system survives between 1-3h charging or discharging before rebooting. Not v.consistent though.
system generally survives 12h+ even at 8sec intervals when doing nothing (as in cells NOT charged nor discharged!)
Even tried without the usb cable connected so no logging via VSC, same thing.
Note that during the 6-8h charging or discharging cycle, I'll only get ONE reboot, rest of cycle goes 90% of the times without an extra one and system stays on generally without issues in idle state.

I'm posting the backtrace from 3-4 of such reboots if it's going to be of any help:

after 1h charging:

Backtrace: 0x40152312:0x3ffe0f40 0x4014aaff:0x3ffe0f70 0x400e69a6:0x3ffe0f90 0x40149c40:0x3ffe0fb0 0x40092ce2:0x3ffe0fe0
  #0  0x40152312:0x3ffe0f40 in tcp_output at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:1027
  #1  0x4014aaff:0x3ffe0f70 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:1765
  #2  0x400e69a6:0x3ffe0f90 in _tcp_recved_api(tcpip_api_call_data*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
  #3  0x40149c40:0x3ffe0fb0 in tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
  #4  0x40092ce2:0x3ffe0fe0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

after 90min charging:

Backtrace: 0x40152312:0x3ffe0f40 0x4014aaff:0x3ffe0f70 0x400e69a6:0x3ffe0f90 0x40149c40:0x3ffe0fb0 0x40092ce2:0x3ffe0fe0
  #0  0x40152312:0x3ffe0f40 in tcp_output at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:1027
  #1  0x4014aaff:0x3ffe0f70 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:1765
  #2  0x400e69a6:0x3ffe0f90 in _tcp_recved_api(tcpip_api_call_data*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
  #3  0x40149c40:0x3ffe0fb0 in tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
  #4  0x40092ce2:0x3ffe0fe0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

after 2:15h discharging:

Backtrace: 0x40152312:0x3ffe0f40 0x4014aaff:0x3ffe0f70 0x400e69a6:0x3ffe0f90 0x40149c40:0x3ffe0fb0 0x40092ce2:0x3ffe0fe0
  #0  0x40152312:0x3ffe0f40 in tcp_output at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp_out.c:1027
  #1  0x4014aaff:0x3ffe0f70 in tcp_recved at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/tcp.c:1765
  #2  0x400e69a6:0x3ffe0f90 in _tcp_recved_api(tcpip_api_call_data*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
  #3  0x40149c40:0x3ffe0fb0 in tcpip_thread at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/tcpip.c:483
  #4  0x40092ce2:0x3ffe0fe0 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

after 1h charging:

Backtrace: 0x1564010b:0x3fff0b00 0x400e79f9:0x3fff0b20 0x400eedad:0x3fff0b60 0x400f0bc6:0x3fff0bd0 0x400e72e2:0x3fff0bf0 0x400e7465:0x3fff0c20 0x400e7475:0x3fff0c40 0x400e77a7:0x3fff0c60 0x40092ce2:0x3fff0c90
  #0  0x1564010b:0x3fff0b00 in ?? ??:0
  #1  0x400e79f9:0x3fff0b20 in std::function<void (void*, AsyncClient*, signed char)>::~function() at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
      (inlined by) std::function<void (void*, AsyncClient*, signed char)>::operator=(std::function<void (void*, AsyncClient*, signed char)> const&) at c:\users\vassilis\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:2075
      (inlined by) AsyncClient::onError(std::function<void (void*, AsyncClient*, signed char)>, void*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:649
  #2  0x400eedad:0x3fff0b60 in AsyncWebServerRequest::AsyncWebServerRequest(AsyncWebServer*, AsyncClient*) at .pio\libdeps\esp32-devkitc\ESP Async WebServer\src/WebRequest.cpp:797
  #3  0x400f0bc6:0x3fff0bd0 in std::_Function_handler<void (void*, AsyncClient*), AsyncWebServer::AsyncWebServer(unsigned short)::{lambda(void*, AsyncClient*)#3}>::_M_invoke(std::_Any_data const&, void*&&, AsyncClient*&&) at .pio\libdeps\esp32-devkitc\ESP 
Async WebServer\src/StringArray.h:73
      (inlined by) _M_invoke at c:\users\vassilis\.platformio\packages\toolchain-xtensa32\xtensa-esp32-elf\include\c++\5.2.0/functional:1871
  #4  0x400e72e2:0x3fff0bf0 in std::function<void (void*, AsyncClient*)>::operator()(void*, AsyncClient*) const at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
  #5  0x400e7465:0x3fff0c20 in AsyncServer::_accepted(AsyncClient*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036  #6  0x400e7475:0x3fff0c40 in AsyncServer::_s_accepted(void*, AsyncClient*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
  #7  0x400e77a7:0x3fff0c60 in _async_service_task(void*) at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:1036
      (inlined by) _async_service_task at .pio\libdeps\esp32-devkitc\AsyncTCP\src/AsyncTCP.cpp:197
  #8  0x40092ce2:0x3fff0c90 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

tcp_out, tcpip, tcp and AsyncTCP feature on most of them...

tbh I'm not too concerned about system rebooting (OK, obvs would be nice if it didn't!) since it's a minimal disruption to the operation of the whole BMS and as long as it cleanly reboots and all systems are fine afterwards (seems to be the case in the 2weeks of testing here)

cheers

V.

[virtuvas]

fwiw,

60sec interval in influxDB v1.8, venus CANBUS connection working fine, latest s/w, uptime 6days and counting, charging/discharging 3S all the time.
closed afaik.

V.

[virtuvas]

@stuartpittaway ,

this issue is also completely solved as influxDB 1.8 is supported, and with the NOV2021 code I've yet to see a reboot.
maybe close it?

V.

[stuartpittaway]

Will close, I only really left it open as instructions on how to use it with 1.8.