Tried it on the ShineWifi-X, works to some extend.

[ZillionPrey]

Hey.

Just got my solar system setup today and I found out that the Growatt system sucks... Since I already have a smarthome with Home Assistant Your project looks very very promising.

I got the ShineWifi-X flashed and the pinouts are all the same like in the S. But it is not connecting to the Inverter. I'm using a Growatt MIC 1000TL-X. Any chance you could help me get this to work?

[otti]

Hi DerpyDoom,

I am running the ShineWiFi-S with an "Growatt 1000S" inverter.
Can you verify, that the schematic of the ShineWiFi-X is the same as the ShineWiFi-S? (see Images in the Repo)
Did you find any information on the protocol which is used by the MIC 1000TL-X? Does it also use Modbus? Are the used registers in both inverters the same?
There is a define named "SIMULATE" in the code. Setting this to "1" will just simulate the connection to the inverter. This is helpful to test the project on your desk without connecting the stick to the inverter.
Does the rest of the project work on the ShineWiFi-X? (WiFi, MQTT, JSON, ...)

Regards,
Otti

[ZillionPrey]

Hey Otti,

Sadly I have not found any information on a different procotol so I assume it should be the same. To flash the Stick I also had to Mod the GPIO0 Pin with a 1k resistor, that worked exactly like the ShineWifi-S.

LEDs, Wi-Fi, MQTT is all working without a problem. If I plug the stick in the Inverter the only return I get from the stick is Disconnected, green light blinking, blue light on.

I've attached some pictures for You. I'll test the Simulation and will come back to you with the results.

Greetings André

[ZillionPrey]

Simulating values returned without a problem to MQTT, still no luck on connecting to the inverter.

However I have found that the Stick is indeed using Modbus on a German datasheet.

Further Investigation, took a look in the manual and it seems like there are 4 connections to talk to the inverter.

[otti]

Hi André,

the XR21V1410 chip on the ShineWiFi-X is just a UART to USB converter. So it is possible, that it is otherwies fully compatible to the ShineWiFi-S and also with the code in this repository. But this is only an assumption and I can not prove it, because I don't own the stick and the inverter.

The Growatt documentation is very bad. But I think that Pin 3 and 4 of the connector will speak Modbus RTU.
Protocol
With a RS485 Tranceiver (e.g. MAX485) and some hardware hacking it could be possible to connect the ShineWiFi-X to this port.

Do you have an oscilloscope or logic analyser and some hardware hacking skills?

Regards,
Otti

[ZillionPrey]

Hey, sorry for the really really late reply! Sadly I am currently not able to look at signals because I don't have an oscilloscope.
I'll google some more and try if it is possible to use the esp as a wireless serial monitor maybe I can find things out this way.

It would be really awesome to get the data into MQTT for Home Assistant to use, I'm quite the data hoarder :D

[Adme84]

Hello Andre and Otti,

do you have any new to this topic? I also have a ShineWifi-X with MIC 1000TL-X and the same issues. I also attached a Wemos D1 and a ESP-07 with USB UART/TTL to verify my observations, but no success util now. I have also attached the USB UART/TTL with 3.3V mode directly to the COM PIN 3 and 4 (ref. 6.5 instruction) and I could see some dataflow with hterm (all data zero). Next step is to attach an RS485 to TTL conveter to COM. But it was not possible to receive any data from the USB interface of the inverter.

Best Regards

Jannis

[ZillionPrey]

Hi Jannis. Nope my stick is still sitting in my inverter doing practically nothing 😄

I currently do not have the funds to get myself a decent oscilloscope. (eBay Kleinanzeigen in Germany) also has nothing interesting listed. To continue experimenting/measure some stuff like Olli suggested.

Going the WeMos route should result the same goal as mine since the stick is basically just a WeMos. (You probably figured that out already seeing my attached images :P)

Please keep us updated if you find anything new we don't know yet 👍🏻

Greetings Andre

[Adme84]

Hi Andre,

If have attached a RS485 to TTL module to PIN 3 and PIN 4 without any success until now. I've tried to receive data with an TTL/USB device directle connected (by hterm) to the RS485 module [RO to RX, RI, DI to GND). With a Webmos D1 also no success. I've test my seput by sending data from one PC (USB/TTL) to RX of Webmos D1 -> USB on another PC. That worked.
I have ordered more RS485 modules to setup an USB/TTL -> RS485 -> RS485 -> USB/TTL data chain for excluding some general hardware/setup issues by myself.
I have also contacted growatt support for getting more information/specs about the USB port on inverter and some other issues with my ShineWifi-X. Unbearable conversation with sensless, contradictory information. I'm not quite sure if it was a translation problem or simply stupidity on growatt side. My question starts in english but I receive the answers in german from growatt. I think it was answered in china with google translator...
Reason for my request was an strange behavoir of the ShineWifi-X. In AP-mode it is not possible to access the webinterface, because there is no open port 80. Seems that it was flashed by growatt (or the stick vendor. I don't think the stick is produced by growatt) with a buggy firmware.
Another question to you: Did you anything special to flash your ShineWifi-X. I tried it with the 1k resistor and Arduino IDE (same config like for my ESP07) several times but without success.

Best regards,

Jannis

[BeoQ]

Hi all,

I can actually see two topics wthin this discussion thread:
A) readout using the ShineWifi-X hardware, with the data going thru the USB-Interface of the Growatt.
B) readout using wires connected to the Communication Interface on the 8-Pin Signal Connector.

Topic A):
I have the exact same behaviour that DerpyDoom describes:
The Growatt does not reply to anything the ShineWifi-X sends to it.
Unfortunately, i had overwritten the ESP's Firmware before capturing any communication of the stock firmware.

To debug, I was sniffing the serial connection at the RX0 and TX0 pins of the ShineWifi-X, using a serial Terminal a PC (like gtkterm/minicom/putty).
I could only observe the requests from the ESP8266 but could not see any reply from the Growatt at all.
Bad.

Note, that on the RX0 and TX0 pins on the PCB, you have standard RS232-signals with 3.3V Level.
Although it likely uses the Modbus-RTU protocol, Electrically this is NOT using RS485 voltage levels!

It might be as simple as just using the wrong Baudrate, but I did not get to test this (yet).
To test, I'd need to connect a standard USB-RS232-3,3V converter to RX0/TX0 and have a Firmware on the ESP that disables (HighZ or Input) its Pins for Serial Comms, to not interfere electrically. Then try to talk to the Growatt by sending the Modbus requests manually, with the right baudrate.

Does anybody have a copy of the stock firmware of the ShineWifi-X, to restore/flash ist?
Or somebody is willing and able to record the serial communication with the original firmware of the ShineWifi-X, when it is connected to the Growatt cloud?

@Jannis,
Do you still have the stock firmware on the ShineWifi-X ?
Could you sniff the serial pins?
Could you share the reply from Growatt (by PM)?

Topic B):
As an alternative to the ShineWifi-X, I have successfully hooked up an USB-to-RS485 Converter to the Communication interface (See DerpyDoom's Picture "6.5 Signal Cable", Pin 3 and Pin 4).
I can confirm it talks at 9600By,8N1 and listens to Modbus Addr 1.

I have successfully used the mbmd from Volkszähler (see here ) as a simple Modbus Client on the commandline. (I use this already for my energy meter. Implementing the Growatt protocol in mbmd for readout using RS485 with wires should not be so hard).

The Register Description perfectly matches the "Modbus RTU Protocol v1.05 2018.pdf" (found somewhere at forum.iobroker.net), as far as my box supports it.
Note: The somewhat static Registers are read using Read Holding Registers (modbus function code 0x03); whereas the dynamic registers are read using Read Input Registers (Modbus function code 0x04). The latter contains the current voltage and energy readings.

I am mainly interested in getting the ShineWifi-X to work.
I might fall back to implementing in mbmd, but that is not the purpose of this discussion thread here.

Happy hacking,
Florian

[BeoQ]

Hi again,

reading my own post again after publishing it, I simply changed the baudrate for the ShineWifi-X for talking across the USB interface in setup():
After a few tries, I found the Growatt responds to Modbus Queries when talking at 115200 Baud.
Looking at the Json-data, only the Reading for DC Voltage makes sense; all others read '0'.

My conclusion for now:
When using the ShineWifi-X on USB, we need to set Baudrate to 115200.
The Growatt also talks Modbus at Addr=1,
The Modbus Registers seem to be diffrerent,
but at least Register 0x03 (DC Voltage) seems to read ok.

[edit:]
It seems Otti implemented the Protocol Version 3.5 from 2013 for his Growatt.
I have just changed the Registers to match the Protocol 1.05 from 2018.
This works for me in a useable way, except the Temp reading.

Here's the code snippet from growatt.cpp that works for me:

  res = Modbus.readInputRegisters(0, 100); // was: (0, 33)
  
  if (res == Modbus.ku8MBSuccess)
  {
    _Data.InverterStatus     =  (eGrowattStatus)Modbus.getResponseBuffer(0);
    _Data.u32DcPower         = (Modbus.getResponseBuffer(5) << 16) + Modbus.getResponseBuffer(6); // Ppv1
    _Data.u16DcVoltage       =  Modbus.getResponseBuffer(3);  // Vpv1
    _Data.u16DcInputCurrent  =  Modbus.getResponseBuffer(4);  // Ipv1
    _Data.u16AcFrequency     =  Modbus.getResponseBuffer(37); // Fac
    _Data.u16AcVoltage       =  Modbus.getResponseBuffer(38); // Vac1
    _Data.u16AcOutputCurrent =  Modbus.getResponseBuffer(39); // Iac1
    _Data.u32AcPower         = (Modbus.getResponseBuffer(40) << 16) + Modbus.getResponseBuffer(41); // Pac1
    _Data.u32EnergyToday     = (Modbus.getResponseBuffer(53) << 16) + Modbus.getResponseBuffer(54); // Eac_today
    _Data.u32EnergyTotal     = (Modbus.getResponseBuffer(55) << 16) + Modbus.getResponseBuffer(56); // Eac_total
    _Data.u32OperatingTime   = (Modbus.getResponseBuffer(57) << 16) + Modbus.getResponseBuffer(58); // Time_total
    _Data.u16Temperaure      =  Modbus.getResponseBuffer(93); // Temp1 (not working)

I think, using #ifdef would be sensible here.

@André: this may help you.

Also, extending the amount of used registers for those haveing a 3-phase Growatt system seems not too hard.
The Protocol 1.05 seems to be prepared for up to 8 DC input strings, 3-phased output, battery pack, and such nice things.

Happy hacking,
Florian

[Adme84]

Hi Florian,

thousand thanks for your intuition!!! It works perfect on my USB/TTL -> ESP07 setup. Thus I will skip all my further RS485 on COM investigations...

On my ShineWifi-X I have (the not correct working) growatt stock firmware, because unfortunately I am not able to flash the stick.
I think the sniffing and e-mail request is obsolete now, but if you want I can share it to you.

Next step for me is to find out why the stick refused my flashing.

Regards,

Jannis

[BeoQ]

Hi Jannis,

glad to hear, it also works for you.
Now that sniffing the serial communication is not that important any more, it would still be helpful to have a backup of the original firmware of the stick.
Could you do a backup of the Firmware from the stick?
Are you familiar with esptool or similar tools to read back firmware from the esp?

Personally, i use esptool.py on Linux, which is supplied along with esp8266 libraries for the Arduino IDE, but to read back you need to run it on the commandline.

Bringing the stick to Flash Mode:
Remember, to get the esp8266 into flash mode (also needed for backup) you need to connect the GPIO0 to GND at reset (or powerup). Since the ShineWifi-X stick already has an USB-Serial converter onboard, you can simply use that for flashing.
Personally, i did not do the 1k-resistor-mod; but I use a Dupont-wire held by hand between the Pins marked GND and GPIO0 on the PCB. No need to hold that wire in place all the time, it is sufficient at the time of plugging it into the USB port of my PC, and a few seconds later.
The ESP will stay in flash mode until unplugged or reset; ready to backup firmware or flash new firmware. Worked for me also in Arduino-IDE after selecting the right Com port. Thankfully, Otti implemented OTA firmware upgrade, so you only need to manually flash it once. Big thumbs UP to Otti!

Out of curiosity: What chipset does your USB-TTL have?
The Chips from Exar are not so wide spread like the ones from FTDI, Silabs or the CH340. I suspect, that the Growatt Firmware supports multiple vendors of USB-serial chips. Which is good know for people that want to build their own replacement of the ShineWifi Stick.

Happy hacking,
Flo

[Adme84]

Hi Flo,

thanks for your reply and hints. I think i am not able to create a firmware backup for you for the same reason why I am not able to flash the stick. I think the stick has some malfunctions.

I tried to ask connected boards with esptool.py flash_id for some infos. My Wemos D1 and ESP07 answer this quite well, but the stick does not answer. [Yes: I connected GPIO0 to GND on powerup]

Getting firmware from ESP07 by "esptool.py --baud 115200 --port COM10 read_flash 0x0 0x100000 fw-backup-1M.bin" was successful. With stick only exceptions / no connection. To say it in german "Das Scheißding fliegt jetzt in die Tonne..." ;-)

The USB-TTL has a CH340G

[BeoQ]

Hi Jannis,

you could give the Stick a last try:

Check if the USB-Serial chip works and is visible an working (Drivers from Exar / MaxLinear ?) in your OS:

• See if it gets displayed in Windows Device Manager or Linux lsusb or dmesg, and on which port.

• If it is found, try hardware echo by shorting the Pins "RX0" and TX0" in the header of the stick by holding Dupont Cables to them; using hterm at any baudrate is fine. It should echo what you type; and stop echoing if you remove the short.
  I'm aware this shorts the Tx of the ESP to the Tx of the USB-Serial, but if it is really dead you might not care anyway. In in doubt, remove R30, and attach to one of its pads.

• If you really want to double-check, you could use your USB-TTL's rx to listen on the "RX0" pin, and watch what you type on the other terminal. Or listen on the TX0 pin and observe what the original firmware says (at 115200Bd)

You might as well disable the onboard USB-Serial by removing (un-soldering) R30, and connect your USB-TTL to RX0 and TX0.
Then bring the stick to flash mode when powering, and see if you can access it with esptool. You might want to remove the 1k-resistor-mod if you have implemented that.

For reference, here's some reverse Engineering of the board; showing what is attached to the pins of the ESP-07

ESP-07S Module (16Pin) on ShineWifi-X:
 1 RST			R19 	Pullup
 2 ADC			R1/R8 	Switch to GND
 3 EN			R16 	Pullup
 4 IO16			R11  	LED3
 5 IO14		SPI_CLK	U5_6	(to Flash) 
 6 IO12		SPI_DO	U5_2	(to Flash)
 7 IO13		SPI_DI	U5_5	(from Flash)
 8 VCC		+3.3V
 9 GND		Ground
10 IO15		SPI_CS
11 IO02		TX1	R9	LED2
12 IO00		GPIO0	R6	LED1
13 IO04		I2C_SCL	R7	U1_6
14 IO05		I2C_SDA	R10	U1_5
15 RXD0		RX0	R30	UART_TxD
16 TXD0		TX0	R29	UART_RxD

Ansonsten: hau wech das Teil. Oder schick es mir ;-)

Cheers,
Flo

[Adme84]

Do it right the first time and...I'm so so sorry. You first point was the solution. Driver was not installed correctly. I loaded a new version from MaxLinar (https://www.maxlinear.com/support/design-tools/software-drivers) and everything works fine now.

How shall I send you the firmware? Do you want to contact me by email? I don't find PM at github.

Best Regards,

Jannis

[ZillionPrey]

Hi again,

reading my own post again after publishing it, I simply changed the baudrate for the ShineWifi-X for talking across the USB interface in setup():
After a few tries, I found the Growatt responds to Modbus Queries when talking at 115200 Baud.
Looking at the Json-data, only the Reading for DC Voltage makes sense; all others read '0'.

My conclusion for now:
When using the ShineWifi-X on USB, we need to set Baudrate to 115200.
The Growatt also talks Modbus at Addr=1,
The Modbus Registers seem to be diffrerent,
but at least Register 0x03 (DC Voltage) seems to read ok.

[edit:]
It seems Otti implemented the Protocol Version 3.5 from 2013 for his Growatt.
I have just changed the Registers to match the Protocol 1.05 from 2018.
This works for me in a useable way, except the Temp reading.

Here's the code snippet from growatt.cpp that works for me:

  res = Modbus.readInputRegisters(0, 100); // was: (0, 33)
  
  if (res == Modbus.ku8MBSuccess)
  {
    _Data.InverterStatus     =  (eGrowattStatus)Modbus.getResponseBuffer(0);
    _Data.u32DcPower         = (Modbus.getResponseBuffer(5) << 16) + Modbus.getResponseBuffer(6); // Ppv1
    _Data.u16DcVoltage       =  Modbus.getResponseBuffer(3);  // Vpv1
    _Data.u16DcInputCurrent  =  Modbus.getResponseBuffer(4);  // Ipv1
    _Data.u16AcFrequency     =  Modbus.getResponseBuffer(37); // Fac
    _Data.u16AcVoltage       =  Modbus.getResponseBuffer(38); // Vac1
    _Data.u16AcOutputCurrent =  Modbus.getResponseBuffer(39); // Iac1
    _Data.u32AcPower         = (Modbus.getResponseBuffer(40) << 16) + Modbus.getResponseBuffer(41); // Pac1
    _Data.u32EnergyToday     = (Modbus.getResponseBuffer(53) << 16) + Modbus.getResponseBuffer(54); // Eac_today
    _Data.u32EnergyTotal     = (Modbus.getResponseBuffer(55) << 16) + Modbus.getResponseBuffer(56); // Eac_total
    _Data.u32OperatingTime   = (Modbus.getResponseBuffer(57) << 16) + Modbus.getResponseBuffer(58); // Time_total
    _Data.u16Temperaure      =  Modbus.getResponseBuffer(93); // Temp1 (not working)

I think, using #ifdef would be sensible here.

@André: this may help you.

Also, extending the amount of used registers for those haveing a 3-phase Growatt system seems not too hard.
The Protocol 1.05 seems to be prepared for up to 8 DC input strings, 3-phased output, battery pack, and such nice things.

Happy hacking,
Florian

Hey Beo,

Great findings, I'll try and see if they also work on mine. I also flashed my Stick before dumping the firmware... Tho the program that was on it didn't really seem to work. I got a code from my dealer and tried to register but nope no success.
I also rather prefer to keep things on my local network without talking to china first :D

Schön Abend noch :)

[ZillionPrey]

@BeoQ I cannot love you enough for your research. ❤️

Changeing the Baudrate to 115200 and using the snippet you provided worked like a charm!
I am going to close the Issue @Adme84 I'd suggest opening a second one so (we) can help you too 👍

[otti]

Hi all,

i have pushed a new firmware.
It will automatically detect the stick (USB or serial) and will use the correct baudrate and register set. The temperature is also working. It can be found in register 3093. :-)

Other new features:

• Added counter for accumulated energy
• Interface for register read/write over web interface added
• MQTT can be turned off by compiler switch
• WiFi connection can be reset if a known IP can not be pinged (optional)

@BeoQ Thanks for your investigations

Gruß,
Otti

[tux-o-matic]

Would an ESP8266 board with its own USB-Serial chip (tested with CP2102 so far) work as is?
So far the code fails for when trying to detect which Shine module is used, I get "undef"

_eDevice = Undef_stick;

  Serial.begin(9600);
  Modbus.begin(1, serial);
  res = Modbus.readInputRegisters(28, 2); // dummy read total energy
  if( res == Modbus.ku8MBSuccess )
  {
    _eDevice = ShineWiFi_S; // Serial
  }
  else
  {
    delay(1000);
    Serial.begin(115200);
    Modbus.begin(1, serial);
    res = Modbus.readInputRegisters(55, 2); // dummy read total energy
    if( res == Modbus.ku8MBSuccess)
    {
      _eDevice = ShineWiFi_X; // USB
    }
  }  

[tux-o-matic]

Answering myself, made it work on an ESP8266 board by just removing the initial attempt at 9600 baud meant to detect the Serial Shine stick connection.
Noticed the issue with baud speed when enabling ENABLE_DEBUG_OUTPUT printed out gibberish on my Serial console unless I explicitly set Serial.begin(115200) before printing any debug messages.

[hugocrank]

has anyone modified the registers for a SPH Storage inverter? adding the registers for discharge / charge / State + Export / Import to grid?
ok, see the registertable and will try to extend the configuration of registers. if someone do this befor feel free to share the code.

[hugocrank]

Florian

can you please share your code with me? i only get "update.Data not sucessfull", manual reding of the register input / Soc / DC / AC /Power is working fine. i dont see my problem and need help.

[BeoQ]

hugocrank:
I had forked otti's work and did some extensions to it:
BeoQ's fork

On the web interface, this adds access not only to holding registers (r/w) but also to input registers (ro);
This way you can check if the content of some registers match your expectation.
Prerequisite: the Baud rate (and Modbus address) must be right.

Works for me, YMMV

For converters with batteries, the holding registers around 1000 may be of interest,
and even more so, some of the input registers between 1014 to 1040.

I had also prepared to access many more registers that are described in the protocol definition, but only got to list them in the header file.
I do plan to do a pull request for otti eventually, but I think it's not worthwhile yet,
I don't have good coding skill, nor much time for it, so it may take an eternity for me to implement it. Unless someone feels like helping out here.

Cheers,
Florian

[hugocrank]

Hallo Florian,
danke für deine Antwort ich werde mir den Inhalt ansehen und schauen ob das bei mir läuft. Mir ist noch immer nicht verständlich warum ich immer die Meldung UpdateData not Sucessfull erhalte obwohl ich ja bereits Holding und Inputregister manuell mit der Webseite lesen kann. Die Speicherinformationen hatte ich ja auch schon gefunden. nur die Automatik läuft ums "verrecken" nicht. Wenn ich es mal am laufen habe werde ich meinen code hier teilen!

Hi Florian,
thanks for your reply, i will look @ your work. dont know why i can read holding and input registers over the Interface but job says updatee not succesfull, i think the Modbusadress and Baudrate is ok -- RW Modbus is still runing.

I found the SOCregisters starting @ 1000 / last week. ;-)

[BeoQ]

Servus Hugo,

while German is fine for me, let's stick to English for everybody else to read.

The webpage's register read/write introduced by otti only accesses the holding registers.
Note that Input registers cannot be written (makes sense, and there is no library function for this).

Make sure that the holding register you want to write to is actually writable, and that the value is withing acceptable range.
E.g. writing a value of more than 100(%) in output active power percentage (Holding 03) will get rejected.

If you struggle with the firmware's auto-detection of the Stick (actually, which baud-rate to use), you can hardcode it in Growatt::begin().

While you#re at it: could you read out some holding registers on your converter for me?
01 (SafetyFunctions)
28/29 (Module H/L)
43 (Device Type Code)
44 (TP, Tracker and Phases)
Purpose is, to see if we can auto-detect the converter.

Thanks and good luck,
Florian

[hugocrank]

Hey together, i am on a good way. it is running now but i have to created every register for 3 phase now, but Battery state is functional!

[hugocrank]

2 days using without any problems on the SPH TL3 with battery

:

[Solpowersun]

Hi everybody, i would like to learn the way for reading my growatt min 6000tl-xh without the delay every 5min by app shinephone, currently i use shinewifi-x firmware version 3.1.0.5, before update firmware mod what is the way to do backup original firmware? Let me know. Thanks a lot

[Solpowersun]

Hola juntos, estoy en un buen camino. se está ejecutando ahora, pero tengo que crear cada registro para la fase 3 ahora, ¡pero el estado de la batería es funcional!

Woow you can see the information every 10 sec. Real time, i also want to get this and so i can configure routines with google home assitant...

[fkirita]

Hello Adme84.
If you still have shine-wifi-x original firmware, you can help me with a copy?
Thanks

[aTommyC]

Still need it? Got a copy.

[fkirita]

Yes, please. Thank you.

…

On 2022-12-19 21:11, aTommyC wrote: > Still need it? Got a copy. -- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you commented.Message ID: ***@***.***> Links: ------ [1] https://github.com/otti/Growatt_ShineWiFi-S/issues/1#issuecomment-1358133491 [2] https://github.com/notifications/unsubscribe-auth/AFAE2EE2LU2HUKL7XAHV6GDWOCXO5ANCNFSM453ETMTQ