Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Found a new endpoint, but it returns a binary file #51

Closed
darryllee opened this issue Aug 6, 2021 · 32 comments
Closed

Found a new endpoint, but it returns a binary file #51

darryllee opened this issue Aug 6, 2021 · 32 comments
Assignees

Comments

@darryllee
Copy link
Contributor

darryllee commented Aug 6, 2021

https://teg/api/devices/vitals

Good old strings returns some interesting tidbits:

THC_STATE_AUTONOMOUSCONTROL
THC_AmbientTemp!

I'd love to get temperature info from my gateway/inverter/battery since I was having problems with my brand-new Tesla 7.6kW inverter because it was installed with low coolant, and until it got topped off last week, was throwing "Inverter Over Temperature" alerts (which after a recent update, were only visible after logging on as Installers instead of Customers. Super-shady.)

@vloschiavo
Copy link
Owner

Nice find! (And nice work on your project https://github.com/darryllee/teslog).

Feel free to issue a pull request with what you know so far, I'm sure the rest of the community will be interested and can help learn more.

Thanks!

@darryllee
Copy link
Contributor Author

DOH, I just saw you already published a bash script that's far more elegant than mine. I will create a PR for the vitals endpoint I discovered. :-}

@vloschiavo vloschiavo reopened this Aug 6, 2021
@vloschiavo
Copy link
Owner

I'm taking a look at the binary file output on my powerwall now. It looks like the various serial numbers of the gateway and the battery units are plastered all over this file. So I suggest you scrub those out before posting an example.

I don't know if there is some special encoding or not yet. But I suspect that these are key-value pairs and probably some standard format from Redis or something similar. I'll put into a hex editor and see what I can reverse engineer.

example: I'm looking at the output in VIM:

Line 24:
^OTHC_AmbientTemp!^@^@^@^@^@^@=@

THOC_AmbientTemp is likely the key, and the value is encoded in the hexidecimal value here: ^@^@^@^@^@^@=@ ; I'll need to see the hex values vs the ASCII translation to see if the numbers make sense.

These would be interesting to have the values too:
37 ^XPOD_nom_energy_remaining!^@^@^@^@^@~Á@^R#
38 ^XPOD_nom_full_pack_energy!^@^@^@^@^@×Ê@^R%
39 ^ZPOD_available_charge_power!^@^@^@^@^@x»@^R%
40 ^ZPOD_available_dischg_power!^@^@^@^@^@x»@^R%
41 ^ZPOD_curtailed_charge_power!^@^@^@^@^@^@^@^@^R%
42 ^ZPOD_curtailed_dischg_power!^@^@^@^@^@^@^@^@^R^_

Also, setting up a "charge completed" notification would be useful in my automation to then enable charging for the car on those days when I'm making more solar than I can use/store in the powerwall. But it looks like these are broken out for each powerwall (if you have multiples on the same gatway).

Powerwall 1:
Line 50:
50 ^RPOD_ChargeComplete0^@^R^Y

Others:
51 ^UPOD_DischargeComplete0^@^R^[
52 ^WPOD_PersistentlyFaulted0^@^R^Z
53 ^VPOD_PermanentlyFaulted0^@^R^U
54 ^QPOD_ChargeRequest0^@^R^U
55 ^QPOD_ActiveHeating0^@^R^O
56 ^KPOD_Resting0^@^R^O
57 ^KPOD_CCVhold0^@^R^Z
58 ^VPOD_HeatZeroCurrentReg0^@^R^
59 ^XPOD_ExtendedCapacityMode0^@^R^Y
60 ^UPOD_IpcRequestedBusUp0^@^R^Y
61 ^UPOD_SelfTestRequested0^A

Powerwall 2:
134 ^RPOD_ChargeComplete0^@^R^Y
135 ^UPOD_DischargeComplete0^@^R^[
136 ^WPOD_PersistentlyFaulted0^@^R^Z
137 ^VPOD_PermanentlyFaulted0^@^R^U
138 ^QPOD_ChargeRequest0^@^R^U
139 ^QPOD_ActiveHeating0^@^R^O
140 ^KPOD_Resting0^@^R^O
141 ^KPOD_CCVhold0^@^R^Z
142 ^VPOD_HeatZeroCurrentReg0^@^R^
143 ^XPOD_ExtendedCapacityMode0^@^R^Y
144 ^UPOD_IpcRequestedBusUp0^@^R^Y
145 ^UPOD_SelfTestRequested0^A

This is also interesting:
172 ^SPINV_MainRelayState*^LRELAY_CLOSED^R&
173 ^VPINV_NeutralRelayState*^LRELAY_CLOSED^R^V

@darryllee
Copy link
Contributor Author

Ha yes, I just created a PR without including my vitals file because of it contained my serial numbers, and I didn't feel like loading up a hex editor and scrubbing that info.

But yes, tons of very interesting info that I've not seen in other endpoints.

@vloschiavo
Copy link
Owner

vloschiavo commented Aug 6, 2021

Not sure if I'm going the right direction but I'm looking at this:
THC_AmbientTemp!......<@

0x0A (Hex value 0A) plus another hex byte always precedes each Value. (maybe 0x0A indicates the end of the line?)

The ! = 0x21 - This looks like a the delimiter for the end of each of the keys.
The @ = 0x40 - This also looks like a delimiter as well.

So the hex in between those delimiters should be the value.
In my case I see the following HEX as the value:
00 00 00 00 00 80 3C

0x80 = 128
0x3C = 60

Does this mean 128.60 °F ?

Seems a bit hot. My ambient outside of the chassis is 80.24 °F.

Maybe 60 ° C?

What are your hex values there?

@vloschiavo
Copy link
Owner

Ha yes, I just created a PR without including my vitals file because of it contained my serial numbers, and I didn't feel like loading up a hex editor and scrubbing that info.

But yes, tons of very interesting info that I've not seen in other endpoints.

Merged.

@darryllee
Copy link
Contributor Author

Here's a bit of data between THC_AmbientTemp! up to " TEPOD:

At 2:36PM (ambient temp 88ºF)
00 00 00 00 40 42 40 0A EF 06 0A 97 01 0A 94 01 0A

At 6:06PM (82ºF)
CE CC CC CC CC 4C 42 40 0A 80 07 0A 97 01 0A 94 01 0A

At 6:26PM (80ºF)
9A 99 99 99 99 19 42 40 0A 80 07 0A 97 01 0A 94 01 0A

So this seems consistent for end of data:
0A 97 01 0A 94 01 0A

@darryllee
Copy link
Contributor Author

From user power.saver on the teslamotorclub.com forums:

The binary data after the text strings is the value, in 64-bit IEEE-754 Floating Point notation. There is a ! character separating the text from the number, and the 8 bytes are in little-endian format.

For example, in the vitals file is the text:
THC_AmbientTemp!

This is followed by 8 bytes (shown here as hexadecimal digits):
CE CC CC CC CC CC 40 40

Here they are in big-endian format (again shown as hexadecimal digits):
40 40 CC CC CC CC CC CE

Those 8 bytes comprise the 64-bits of the IEEE-754 Floating Point number.

The decimal equivalent of that is 33.6 which is the temperature in Celsius. (92.5°F)

Text fields that are followed by a * character have a text value after them (enumeration).

The object:
THC_State*
has the value:
THC_STATE_AUTONOMOUSCONTROL

An interesting one in this file is:
POD_DC_Bus_Voltage!
which has a value of 428.1 volts.

@sean-user
Copy link

sean-user commented Aug 8, 2021

I'm power.saver on the TMC forums. Thanks to @darryllee for bringing my post over here.

Some additional info on the objects with * after them (text enumerations). The first byte after the * character is the length of the text string, followed by the actual text. That wasn't explained in my post on the TMC forums.

@sean-user
Copy link

With a little work, I was able to parse out most of the vitals file. Here is the text result. The first column is the number of bytes into the file where the parser found the start of an object. This will probably vary since differences in the text string lengths affects the file size.

A new delimiter was found: if a 0 (ascii zero) appears after the object then the value is TRUE or FALSE, and is represented by a binary 1 or 0 in the next byte. I show TRUE or FALSE when this occurs.

0031 : 1118431-00-J = 
00b4 : 1092170-03-G = 
0128 : THC_State = THC_STATE_AUTONOMOUSCONTROL
0152 : THC_AmbientTemp = 33.600000
019b : 1081100-79-J = 
020b : POD_nom_energy_to_be_charged = 2756.000000
0234 : POD_nom_energy_remaining = 11115.000000
0259 : POD_nom_full_pack_energy = 13707.000000
027e : POD_available_charge_power = 7000.000000
02a5 : POD_available_dischg_power = 7000.000000
02cc : POD_curtailed_charge_power = 0.000000
02f3 : POD_curtailed_dischg_power = 0.000000
031a : POD_est_DC_Bus_Power = -2320.000000
033b : POD_Ibatt = -50.600000
0351 : POD_Vbatt = 46.600000
0367 : POD_DC_Bus_Voltage = 428.100000
0386 : POD_state = POD_ACTIVE
039f : POD_enable_line = TRUE
03b4 : POD_ChargeComplete = FALSE
03cc : POD_DischargeComplete = FALSE
03e7 : POD_PersistentlyFaulted = FALSE
0404 : POD_PermanentlyFaulted = FALSE
0420 : POD_ChargeRequest = FALSE
0437 : POD_ActiveHeating = FALSE
044e : POD_Resting = FALSE
045f : POD_CCVhold = FALSE
0470 : POD_HeatZeroCurrentReg = FALSE
048c : POD_ExtendedCapacityMode = FALSE
04aa : POD_IpcRequestedBusUp = FALSE
04c5 : POD_SelfTestRequested = TRUE
050e : 1081100-79-J = 
057e : PINV_EnergyDischarged = 4173130.000000
05a0 : PINV_EnergyCharged = 4827370.000000
05bf : PINV_VSplit1 = 123.300000
05d8 : PINV_VSplit2 = 123.200000
05f1 : PINV_PllFrequency = 59.995000
060f : PINV_PllLocked = TRUE
0623 : PINV_Pout = 2.320000
0639 : PINV_Qout = -0.020000
064f : PINV_Vout = 246.500000
0665 : PINV_Fout = 60.011000
067b : PINV_ReadyForGridForming = TRUE
0699 : PINV_State = PINV_GridFollowing
06bb : PINV_GridState = Grid_Compliant
06dd : PINV_MainRelayState = RELAY_CLOSED
0702 : PINV_NeutralRelayState = RELAY_CLOSED
072a : PINV_Iavail = 32.400000
0742 : PINV_Savail = 7.990000
075a : PINV_Vdc = 428.100000
076f : PINV_HardwareEnableLine = TRUE
078c : PINV_PowerLimiter = PWRLIM_No_Power_Limit
080d : 1092170-03-G = 
0881 : THC_State = THC_STATE_AUTONOMOUSCONTROL
08ab : THC_AmbientTemp = 29.100000
08f4 : 1081100-59-J = 
0964 : POD_nom_energy_to_be_charged = 2758.000000
098d : POD_nom_energy_remaining = 11207.000000
09b2 : POD_nom_full_pack_energy = 13810.000000
09d7 : POD_available_charge_power = 7000.000000
09fe : POD_available_dischg_power = 7000.000000
0a25 : POD_curtailed_charge_power = 0.000000
0a4c : POD_curtailed_dischg_power = 0.000000
0a73 : POD_est_DC_Bus_Power = -2350.000000
0a94 : POD_Ibatt = -51.200000
0aaa : POD_Vbatt = 46.600000
0ac0 : POD_DC_Bus_Voltage = 428.000000
0adf : POD_state = POD_ACTIVE
0af8 : POD_enable_line = TRUE
0b0d : POD_ChargeComplete = FALSE
0b25 : POD_DischargeComplete = FALSE
0b40 : POD_PersistentlyFaulted = FALSE
0b5d : POD_PermanentlyFaulted = FALSE
0b79 : POD_ChargeRequest = FALSE
0b90 : POD_ActiveHeating = FALSE
0ba7 : POD_Resting = FALSE
0bb8 : POD_CCVhold = FALSE
0bc9 : POD_HeatZeroCurrentReg = FALSE
0be5 : POD_ExtendedCapacityMode = FALSE
0c03 : POD_IpcRequestedBusUp = FALSE
0c1e : POD_SelfTestRequested = TRUE
0c67 : 1081100-59-J = 
0cd7 : PINV_EnergyDischarged = 4179150.000000
0cf9 : PINV_EnergyCharged = 4857010.000000
0d18 : PINV_VSplit1 = 123.400000
0d31 : PINV_VSplit2 = 123.100000
0d4a : PINV_PllFrequency = 60.007000
0d68 : PINV_PllLocked = TRUE
0d7c : PINV_Pout = 2.330000
0d92 : PINV_Qout = -0.030000
0da8 : PINV_Vout = 246.600000
0dbe : PINV_Fout = 60.013000
0dd4 : PINV_ReadyForGridForming = TRUE
0df2 : PINV_State = PINV_GridFollowing
0e14 : PINV_GridState = Grid_Compliant
0e36 : PINV_MainRelayState = RELAY_CLOSED
0e5b : PINV_NeutralRelayState = RELAY_CLOSED
0e83 : PINV_Iavail = 32.400000
0e9b : PINV_Savail = 7.980000
0eb3 : PINV_Vdc = 428.100000
0ec8 : PINV_HardwareEnableLine = TRUE
0ee5 : PINV_PowerLimiter = PWRLIM_No_Power_Limit
0f67 : 1118431-00-J = 
0fdb : ISLAND_VL1N_Main = 122.500000
0ff8 : ISLAND_FreqL1_Main = 60.010000
1017 : ISLAND_VL1N_Load = 122.500000
1034 : ISLAND_FreqL1_Load = 60.010000
1053 : ISLAND_PhaseL1_Main_Load = 0.000000
1078 : ISLAND_VL2N_Main = 121.000000
1095 : ISLAND_FreqL2_Main = 60.010000
10b4 : ISLAND_VL2N_Load = 121.000000
10d1 : ISLAND_FreqL2_Load = 60.010000
10f0 : ISLAND_PhaseL2_Main_Load = 0.000000
1115 : ISLAND_VL3N_Main = 0.000000
1132 : ISLAND_FreqL3_Main = 0.000000
1151 : ISLAND_VL3N_Load = 0.000000
116e : ISLAND_FreqL3_Load = 0.000000
118d : ISLAND_PhaseL3_Main_Load = -256.000000
11b2 : ISLAND_L1L2PhaseDelta = -256.000000
11d4 : ISLAND_L1L3PhaseDelta = -256.000000
11f6 : ISLAND_L2L3PhaseDelta = -256.000000
1218 : ISLAND_GridState = ISLAND_GridState_Grid_Compliant
124d : ISLAND_L1MicrogridOk = TRUE
1267 : ISLAND_L2MicrogridOk = TRUE
1281 : ISLAND_L3MicrogridOk = FALSE
129b : ISLAND_ReadyForSynchronization = TRUE
12bf : ISLAND_GridConnected = TRUE
12d9 : SYNC_ExternallyPowered = FALSE
12f5 : SYNC_SiteSwitchEnabled = TRUE

@fkhera
Copy link

fkhera commented Aug 8, 2021 via email

@vloschiavo
Copy link
Owner

Nice work everyone! Thanks!

Is anyone up for creating a sample decoder script?

@darryllee
Copy link
Contributor Author

darryllee commented Aug 25, 2021

Got new firmware last night (21.31.0), and noticed that the vitals files that I've just been dumping (every 5 min) into a directory was consistently slightly bigger.

I did a diff -a and eyeballing it, this is the only thing that I could see that is "new":

PVS_a031_Mci3PvVoltage
PVS_a032_Mci4PvVoltage

That's "below"(?) a previously undocumented PVS section.

Full dump. Before:

        PVS_State*
PVS_Active^R$
^QPVS_SelfTestState*^OPVS_SelfTestOff^R^T
^PPVS_EnableOutput0^A^R^Y
^UPVS_StringA_Connected0^A^R^Y
^UPVS_StringB_Connected0^A^R^Y
^UPVS_StringC_Connected0^@^R^Y
^UPVS_StringD_Connected0^@^Z^SPVS_a019_MciStringC^Z^SPVS_a020_MciStringD

After:

        PVS_State*
PVS_Active^R$
^QPVS_SelfTestState*^OPVS_SelfTestOff^R^T
^PPVS_EnableOutput0^A^R^Y
^UPVS_StringA_Connected0^A^R^Y
^UPVS_StringB_Connected0^A^R^Y
^UPVS_StringC_Connected0^@^R^Y
^UPVS_StringD_Connected0^@^Z^SPVS_a019_MciStringC^Z^SPVS_a020_MciStringD^Z^VPVS_a031_Mci3PvVoltage^Z^VPVS_a032_Mci4PvVoltage

Yeah... a decoder would be nice. :-}

@sean-user
Copy link

I've attached the parser I wrote in C. Sample output file included as well.

vitals.zip

@darryllee
Copy link
Contributor Author

Awesome! To get it to build on my Raspberry Pi I just had to remove:

#include <conio.h>

And flip the slashes here:

#include <sys/types.h>
#include <sys/stat.h>

Then I ran it (so the input file has to be named vitals. Got it.)

Do you think those are new delimiters for the PVS... MciString values that your code isn't detecting? As I mentioned, I only started seeing them in this latest 21.31.0 firmware.

@sean-user
Copy link

I don't have 21.31.0 yet, so can't check the file, but all others were preceded by 0x12 0x?? 0x0a before the key. 0x?? is any value. So if it follows that format, it should get parsed out. I read all the way to the end of the file. Yes, the input file is called vitals since that's what the PW downloaded, but you could change that in the code if you like.

@darryllee
Copy link
Contributor Author

Would a hexdump help?

00000da0: 0a15 5056 535f 5374 7269 6e67 435f 436f  ..PVS_StringC_Co
00000db0: 6e6e 6563 7465 6430 0012 190a 1550 5653  nnected0.....PVS
00000dc0: 5f53 7472 696e 6744 5f43 6f6e 6e65 6374  _StringD_Connect
00000dd0: 6564 3000 1a13 5056 535f 6130 3139 5f4d  ed0...PVS_a019_M
00000de0: 6369 5374 7269 6e67 431a 1350 5653 5f61  ciStringC..PVS_a
00000df0: 3032 305f 4d63 6953 7472 696e 6744 1a16  020_MciStringD..
00000e00: 5056 535f 6130 3331 5f4d 6369 3350 7656  PVS_a031_Mci3PvV
00000e10: 6f6c 7461 6765 1a16 5056 535f 6130 3332  oltage..PVS_a032
00000e20: 5f4d 6369 3450 7656 6f6c 7461 6765 0aa5  _Mci4PvVoltage..

@sean-user
Copy link

The first one is cut off, but the the second one, starting at 0db9 = 12, 0dba = 19 (don't care) and 0dbc = 0a so the parser should find it. The trailing character is "0" so this is a TRUE/FALSE key, and the value is 00 (FALSE). But the next one is different, there are no starting bytes and no trailing delimiter, so the parser won't find that this one (PVS_a019_MciStringC) or the rest of them. Looks like they may be a list with no values?

@darryllee
Copy link
Contributor Author

Sorry, yeah, I cut it off too soon:

00000d10: 396e 4012 170a 0950 5653 5f53 7461 7465  9n@....PVS_State
00000d20: 2a0a 5056 535f 4163 7469 7665 1224 0a11  *.PVS_Active.$..
00000d30: 5056 535f 5365 6c66 5465 7374 5374 6174  PVS_SelfTestStat
00000d40: 652a 0f50 5653 5f53 656c 6654 6573 744f  e*.PVS_SelfTestO
00000d50: 6666 1214 0a10 5056 535f 456e 6162 6c65  ff....PVS_Enable
00000d60: 4f75 7470 7574 3001 1219 0a15 5056 535f  Output0.....PVS_
00000d70: 5374 7269 6e67 415f 436f 6e6e 6563 7465  StringA_Connecte
00000d80: 6430 0112 190a 1550 5653 5f53 7472 696e  d0.....PVS_Strin
00000d90: 6742 5f43 6f6e 6e65 6374 6564 3001 1219  gB_Connected0...
00000da0: 0a15 5056 535f 5374 7269 6e67 435f 436f  ..PVS_StringC_Co
00000db0: 6e6e 6563 7465 6430 0012 190a 1550 5653  nnected0.....PVS
00000dc0: 5f53 7472 696e 6744 5f43 6f6e 6e65 6374  _StringD_Connect
00000dd0: 6564 3000 1a13 5056 535f 6130 3139 5f4d  ed0...PVS_a019_M
00000de0: 6369 5374 7269 6e67 431a 1350 5653 5f61  ciStringC..PVS_a
00000df0: 3032 305f 4d63 6953 7472 696e 6744 1a16  020_MciStringD..
00000e00: 5056 535f 6130 3331 5f4d 6369 3350 7656  PVS_a031_Mci3PvV
00000e10: 6f6c 7461 6765 1a16 5056 535f 6130 3332  oltage..PVS_a032
00000e20: 5f4d 6369 3450 7656 6f6c 7461 6765 0aa5  _Mci4PvVoltage..
00000e30: 0f0a 9f01 0a9c 010a 260a 2454 4553 594e  ........&.$TESYN

If I'm reading this right, I think PVS_a031_Mci3PvVoltage and PVS_a031_Mci3PvVoltage both start with: 1a16, and well, then there's a bunch of data after that that I cut off in my last post. Sorry about that.

@sean-user
Copy link

The 16 is the string length (22 characters). The 1a is unknown because it immediately follows the previous key, which should have been a delimiter and a value, but they are not there. It's possible they added these new keys without fully implementing them, since no values appear after them like all the others. Or they are just text which is part of PVS_StringD_Connected.

@wiedmann
Copy link

wiedmann commented Sep 1, 2021

I'll just add here that putting my output through a protobuf decoder yields something at least half-way sensible, so this may be a serialization format based on a similar principle. Try this: https://protogen.marcgravell.com/decode

The strings and varints seem to decode correctly, but the fixed64 values don't appear correct, so it's probably some other format. In terms of unpacking the data, though, this might be helpful.

I'm still on version 21.20.6, FWIW.

@brianhealey
Copy link

brianhealey commented Sep 21, 2021

Not perfect.. but close. It's definitely protobuf.

I am able to parse my file using this but I still have a few defects.

The "root" is the DevicesWithVitals entity which contains multiple devices. The c script above was dumping the aggregates across multiple devices.

@sean-user

@brianhealey
Copy link

`syntax = "proto3";
option go_package = "github.com/example/path/gen;gen";
package teslapower;

import "google/protobuf/timestamp.proto";

message DeviceVital {
optional string name = 1;
oneof value {
int64 intValue = 3;
double floatValue = 4;
string stringValue = 5;
bool boolValue = 6;
}
}

message StringValue {
string value = 1;
}

message UInt32Value {
uint32 value = 1;
}

message ConnectionParameters {
optional string ipAddress = 1;
optional string serialPort = 2;
optional string serialBaud = 3;
optional uint32 modbusId = 4;
}

message TeslaHardwareId {
optional UInt32Value pcbaId = 1;
optional UInt32Value assemblyId = 2;
optional UInt32Value usageId = 3;

}

message TeslaEnergyEcuAttributes {
optional int32 ecuType = 1;
optional TeslaHardwareId hardwareId = 2;

}

message GeneratorAttributes {
optional uint64 nameplateRealPowerW = 1;
optional uint64 nameplateApparentPowerVa = 2;
}

message PVInverterAttributes {
optional uint64 nameplateRealPowerW = 1;
}

message MeterAttributes {
repeated uint32 meterLocation = 1;
}

message DeviceAttributes {
oneof deviceAttributes {
TeslaEnergyEcuAttributes teslaEnergyEcuAttributes = 1;
GeneratorAttributes generatorAttributes = 2;
PVInverterAttributes pvInverterAttributes = 3;
MeterAttributes meterAttributes = 4;
};
}

message Device {
optional StringValue din = 1;
optional StringValue partNumber = 2;
optional StringValue serialNumber = 3;
optional StringValue manufacturer = 4;
optional StringValue siteLabel = 5;
optional StringValue componentParentDin = 6;
optional StringValue firmwareVersion = 7;
optional google.protobuf.Timestamp firstCommunicationTime = 8;
optional google.protobuf.Timestamp lastCommunicationTime = 9;
optional ConnectionParameters connectionParameters = 10;
repeated DeviceAttributes deviceAttributes = 11;
}

message SiteControllerConnectedDevice {
optional Device device = 1;
}

message SiteControllerConnectedDeviceWithVitals {
repeated SiteControllerConnectedDevice device = 1;
repeated DeviceVital vitals = 2;
repeated string alerts = 3;
}

message DevicesWithVitals {
repeated SiteControllerConnectedDeviceWithVitals devices = 1;
}`

@fkhera
Copy link

fkhera commented Sep 21, 2021 via email

@mir0810
Copy link

mir0810 commented Nov 7, 2021

I logged into my TEG on its internal wifi with installer access. the "System" page shows all the vital statistics nicely represented. The reason I was in there was to figure out how much electricity each solar string was generating. But it would be nice to parse out the additional values like temp and frequency as well.
Since the website shows it clear text, I assume there is a javascript on the page that does the decoding but I haven't looked at it in detail yet :)

@jasonacox
Copy link
Contributor

jasonacox commented Nov 23, 2021

@sean-user Thanks for your code. I ported a version to python here but it isn't quite right. The protobuf path seems very interesting.

@brianhealey Were you able to parse with that? Any quick tips for someone just learning protobuf? :) I just brew install protobuf and checking out how to get a proto definition into python to parse these vitals payloads. Going through protoc now...

I was able to use protoc to get the raw decode, so this is clearly a protobuf payload:

protoc --decode_raw < vitals.bin

It represents the data in { ... } groupings with number prefixing and key values in hex, strings or 1/0 boolean - example snip:

1 {
  1 {
    1 {
      1 {
        1: "TETHC--1234567-25-E--TG123456789012"
      }
      2 {
        1: "1234567-25-E"
      }
      3 {
        1: "TG123456789012"
      }
      4 {
        1: "TESLA"
      }
      6 {
        1: "STSTSM--1234567-00-E--TG123456789012"
      }
      7 {
        1: "67f943cb05d12d"
      }
      9 {
        1: 1637708577
      }
      11 {
        1 {
          1: 224
        }
      }
    }
  }
  2 {
    1: "THC_State"
    5: "THC_STATE_AUTONOMOUSCONTROL"
  }
  2 {
    1: "THC_AmbientTemp"
    4: 0x403899999999999c
  }
}

I'm particularly interested in getting power data on strings. This shows up in the PVAC section - example:

 2 {
    1: "PVAC_PVMeasuredVoltage_A"
    4: 0x406cb66666666667
  }
  2 {
    1: "PVAC_PVMeasuredVoltage_B"
    4: 0xc003333333333332
  }
  2 {
    1: "PVAC_PVMeasuredVoltage_C"
    4: 0x406c09999999999a
  }
  2 {
    1: "PVAC_PVMeasuredVoltage_D"
    4: 0x406c09999999999a
  }
  2 {
    1: "PVAC_PVMeasuredPower_A"
    4: 0x4069c00000000000
  }
  2 {
    1: "PVAC_PVMeasuredPower_B"
    4: 0x0000000000000000
  }
  2 {
    1: "PVAC_PVMeasuredPower_C"
    4: 0x4078f00000000000
  }
  2 {
    1: "PVAC_PVMeasuredPower_D"
    4: 0x4078b00000000000
  }
  2 {
    1: "PVAC_LifetimeEnergyPV_Total"
    4: 0x4133401600000000
  }

Boolean examples:

 2 {
    1: "ISLAND_L1MicrogridOk"
    6: 1
  }
  2 {
    1: "ISLAND_L2MicrogridOk"
    6: 1
  }
  2 {
    1: "ISLAND_L3MicrogridOk"
    6: 0
  }
  2 {
    1: "ISLAND_ReadyForSynchronization"
    6: 1
  }
  2 {
    1: "ISLAND_GridConnected"
    6: 1
  }

@jasonacox
Copy link
Contributor

jasonacox commented Nov 28, 2021

I used the great work by @brianhealey (tesla.proto definition) to process the vitals payload and produce a json representation. Work in progress: https://github.com/jasonacox/pypowerwall/tree/main/examples/vitals

UPDATE: I added the protobuf handling to the pypowerwall python library to make it easy to pull the vitals data. My goal was to get the solar panel "String" data so I added a function to parse that data from the vitals payload. Both functions, vitals() and strings() return python dictionary or JSON payloads.

# Install pyPowerwall package
pip install pypowerwall
import pypowerwall

# Credentials for your Powerwall - Customer Login Data
password='password'
email='email@example.com'
host = "localhost"                # Change to the IP of your Powerwall
timezone = "America/Los_Angeles"  # Change to your local timezone/tz

# Connect to Powerwall
pw = pypowerwall.Powerwall(host,password,email,timezone)

# Display Vitals (returns details as dictionary or JSON)
vitals = pw.vitals(jsonformat=False)
print(vitals)

# Display String Data 
strings = pw.strings(jsonformat=False)
print(strings)

@reli3
Copy link

reli3 commented Dec 5, 2021

What would be the best way to format the output a bit better?

I am Pre-PTO I think there are issues with my MCI - Maybe once a week / day etc. I have to keep an eye on my strings.
Sometimes 1 or maybe 2 strings are not functioning so I have to do a reboot of the inverters to make them show up again.

If getting the fromat correct and comparing it from a baseline everyday if something turns False for a string some type of email notifaction would be amazing for a lot of people I believe when they think there solar is under producing but in the end it could be a MCI problem or a string issue since people complain alot about under performance this could be the reason why!

After executing this is what I get for the Dispaly String Data:

{'A': {'Current': 1.1300000000000001, 'Voltage': 141.20000000000002, 'Power': 173.0, 'State': 'PV_Active', 'Connected': True}, 'B': {'Current': 0.54, 'Voltage': 339.0, 'Power': 178.0, 'State': 'PV_Active', 'Connected': True}, 'C': {'Current': 0.59, 'Voltage': 324.6, 'Power': 197.0, 'State': 'PV_Active', 'Connected': True}, 'D': {'Current': 0.48, 'Voltage': 99.4, 'Power': 64.0, 'State': 'PV_Active', 'Connected': True}, 'A1': {'Current': 1.1, 'Voltage': 302.90000000000003, 'Power': 327.0, 'State': 'PV_Active', 'Connected': True}, 'B1': {'Current': 1.09, 'Voltage': 303.5, 'Power': 320.0, 'State': 'PV_Active_Parallel', 'Connected': True}, 'C1': {'Current': 0.0, 'Voltage': -1.299999999999999, 'Power': 0.0, 'State': 'PV_Active', 'Connected': False}, 'D1': {'Current': 0.0, 'Voltage': -0.29999999999999893, 'Power': 0.0, 'State': 'PV_Active', 'Connected': False}}

@jasonacox
Copy link
Contributor

jasonacox commented Dec 5, 2021

Hi @reli3 - I want to keep an eye on my strings as well, which is why I wrote this. I also wanted to keep an eye on the Neruio solar meter which is also in the vitals payload. It looks like you have 6 active strings in your system (A, B, C, D, A1, and B1). For string data, if you want to see it in human readable format, just specify jsonformat as True like this:

strings = pw.strings(jsonformat=True)
print(strings)

Output:

{
    "A": {
        "Connected": true,
        "Current": 0.48,
        "Power": 115.0,
        "State": "PV_Active",
        "Voltage": 251.8
    },
    "B": {
        "Connected": false,
        "Current": 0.0,
        "Power": 0.0,
        "State": "PV_Active",
        "Voltage": -2.3999999999999995
    },
    "C": {
        "Connected": true,
        "Current": 1.82,
        "Power": 526.0,
        "State": "PV_Active",
        "Voltage": 293.6
    },
    "D": {
        "Connected": true,
        "Current": 1.81,
        "Power": 521.0,
        "State": "PV_Active_Parallel",
        "Voltage": 294.0
    }
}

I push that into a tiny installation of Splunk (see TinySplunk) and get a graph like this:

image

I have a lot of trouble with the Neruio solar meter that Tesla installed in my Powerwall+ so I also have scripts to watch that. It shows up in the vitals payload as a device that I monitor and capture to create alerts (e.g. if it doesn't show up in the vitals payload that means it has lost connection with the gateway and shut down solar production):

"NEURIO--VAHxxxxxxxxxx": {
        "NEURIO_CT0_InstRealPower": 1333.5606370817025,
        "NEURIO_CT0_Location": "solar",
        "Parent": "STSTSM--xxxxxxx-00-E--TGxxxxxxxxxxxx",
        "firmwareVersion": "1.6.1-Tesla",
        "lastCommunicationTime": "1638720520",
        "manufacturer": "NEURIO",
        "partNumber": "",
        "serialNumber": "VAHxxxxxxxxxx"
    }

@reli3
Copy link

reli3 commented Dec 5, 2021

@jasonacox omg thata is amazing exactly what I think I am looking for....any chance for a write up to set this up via tinysplunk on how you are getting the data in? Does it pull the string data every minute?

This output will help in the morning and ill run it via cron to put to what I need instead of having to login via installer and flipping the powerwall switch when I think its not producing the correct amount of solar as expected!

I loved your other dashboard with grafana was thinking of making a PI display just for that eventually.

I believe I have the NEURIO as well since I have Two Powerwall+ and 4x Powerwalls. I haven't had any issues with the Neurio at least so far besides sometimes the inveter likes to discount from the Wi-Fi every once in a while which what I found was to re join it via the App surpisingly and it fixes it.

Thanks for your amazing work so far with this!

@jasonacox
Copy link
Contributor

Thanks @reli3 ! I cross posted this as an issue under pypowerwall for Strings Data- (see jasonacox/pypowerwall#1) since I don't know how relevant it would be for this issue. I included the script I use to gather and push the string data into Splunk. I plan to add this to the Grafana dashboard as well. :)

@brianhealey
Copy link

Glad to see that the protocol buffer example helped. :) There were some errors in what I put together, so I forked your branch to update it. Hopefully this update will help flush out any issues your might be seeing and/or include fields I didn't have before that might be silently ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants