-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add support for e-volve/fitdays smart scale #503
Comments
Hi @bbbart
The perfect situation for Decoder.
Correct, Theengs Decoder is the BLE decoding library used for OpenMQTTGatewaym Theengs Gateway, the Theengs App for mobile phones and several other projects …
We have an Adding Decoders section for anyone wanting to add new decoders for devices. It can possibly be a bit overwhelming when going through it all for the first time, but we are happy to help with any queries or working on a new decoder together. You might want to have a look at the existing Xiaomi Body Composition Scale decoder as a starting point, as you scale has much the same functionality. https://github.com/theengs/decoder/blob/development/src/devices/XMTZC05HMLB_json.h |
Alright, let me look into it. Thanks for the pointers I currently have it working with a completely custom written, Nimble and PubSubClient based approach, but I'd rather be standing on some giant's shoulders here than completely relying on myself. :-D |
Just a quick question: part of the decoding requires and XOR operation of selected bytes with a given, static value. All other operations and conditions required seem fine, but I couldn't seem to find how to get that XOR to happen. Is this a supported operation for a |
(I know that A XOR B = (A NAND (A NAND B)) NAND (B NAND (A NAND B)) and that NOT and AND are available, but is that the best way forward? :-)) |
Do you have your custom solution online as a repo? Or could you post some undecoded raw data you a receiving with OpenMQTTGateway from the scale, when you have Advertisement and advanced data set to true? Also indicating which part of the advertising data you already know is the weight in kg, with any additionally required calculations? Do you build OpenMQTTGateway for an ESP32 yourself with PlatformIO? |
Could you possibly explain a bit further how this specifically applies to decoding the scale data? I would assume that what you need to do might be to have alternative calculations for the same property, but with different conditions. Have a look at the APPLEDEVICE decoder, with its sole unlocked property having two conditional alternatives https://github.com/theengs/decoder/blob/development/src/devices/APPLEDEVICE_json.h So creating two conditions, one with true AND false, the other with false AND true, you should be able to define your XOR situation. |
This is what I figured out how to get weight data from the advertised manufacturer data: if byte at position 8 is 0x20, then Here's a snippet from my current code: if (cManufacturerData[8] == 0x20) {
uint32_t rawWeight = 0;
rawWeight |= ((uint32_t)cManufacturerData[9] << 16);
rawWeight |= ((uint32_t)cManufacturerData[10] << 8);
rawWeight |= ((uint32_t)cManufacturerData[11]);
uint32_t decodedWeight = rawWeight ^ 0x2CA0A0;
float weight = round(decodedWeight / 1000.0f * 10) / 10.0f;
} I feel I know how to get the condition, the concatenation and the division implemented, but not the xorring. |
Could you give some example manufacturerdata along with its decoded weight? |
sure:
-> byte at position 8 is 0x20, so you calculate (0x2DB052 ^ 0x2CA0A0) / 1000 = 69.874, which gives 69.9 when rounded. |
I suppose we'd have to add an XOR post_proc for this ;) quite an unusual weight calculation. May I ask where you got this from, as for only by reverse engineering it is not something immediately obvious?
Does this mean that with octet 8 being something other than 0x20 there is also a different XOR calculation? |
I reckon this is what they refer to as encryption on the packaging...
I put two and two together after stumbling upon https://stackoverflow.com/questions/63761622/get-weight-data-from-bluetooth-le-scale-by-disassemble-manucaftures-library and figuring out what the data was for 0.0kg.
No. If there's no 0x20 there, it either means that:
|
As long as this 'encryption' is generally applicable to all the e.volve scales and not some changing value set when first pairing the scale to the app, which I assume you have previously done. Would be also good to know if the scale always broadcasts BLE advertising data, or only if or if not provisioned with the app, as some devices do.
Good to know, as we also have a similar indicator with the Xiaomi scale, and only decode and publish the weight with the 'settled' indicator. I suppose there is a similar indicator to show if the weighing has been done barefoot so that any impedance value broadcast is validated as well. Let me have a look into adding an XOR post_proc the next days, unless you also want to have a go at adding this yourself 😉 https://github.com/theengs/decoder/blob/development/src/decoder.cpp#L670 |
In started a bit if you want to have a peak ;) |
Ah, these are good questions. I indeed paired the app with the scale after unpacking it. However, I suspect the scale being 'dumb' enough to not being able to have anything set or written to it. The same app supports 4G, WIFI and Bluetooth scales. I'm doubtful it is doing more than just looking for and registering data (I'm not sure, obviously). I do know at least that even with the app not running, I can get the values. Also, with the app running, my ESP32 and a BLE sniffer on, all three devices get the advertising data simultaneously. There is absolutely no talk of connecting or pin codes or other security measures.
Ah. Then this 0x20 is the 'settled' indicator. FYI, the same decoding works for packages having a 0xA0 at position 8, but those aren't settled then.
yup. The impedance value is set with indicator 0x85, but I haven't figured out yet what the corresponding data bytes mean or how to interpret them. I do know that the packet is always there, even when the app says that no impedance data was received (when not weighing barefoot, for example). |
It's not so much the app running or not etc. but just having provisioned/paired the scale with the app once. Some devices, e.g. the Qingping Air Monitor Lite, only broadcast BLE advertising data after they have been provisioned/paired with their app. Then the app can not run at all or even be deleted, but once they get deleted from the app again they don't broadcast any more BLE advertising data - hence we have made a note of this in the relating doc https://decoder.theengs.io/devices/CGDN1.html It would be good if you could just test this with the scale being deleted from the app as well, just for verification. Hopefully though, when provisioning/pairing the scale with the app again it won't set a different encryption/XOR key to the current one ;) Also it is very likely that the scale can have some small bits set/written to it, probably at least the unit, if the display on the scale shows kg or lbs for example. So an encryption/XOR key might also be an option - but let's hope not.
I don't see much point in publishing the unsettled weight and only to publish the weight once it has settled down as indicated by the 0x20, that's why I put this verification into the actual model condition of the quick test decoder. We do the same for the Xiaomi scales.
Again much the same as with the Xiamo scale, where impedance data is always being broadcast, but when wearing shoes or socks there is a 'not valid impedance' indicator which changes to a different 'valid' indicator when a proper impedance reading has been achieved standing barefoot on the scale. Putting this as a condition of the impedance property makes sure that only valid impedance readings are being published. FYI - I merged the XOR post_proc |
Okay. I played a little more with the scale. Here are some findings:
So, I still don't know if the 'encryption' is set at init, nor if the scale will work if never paired in the first place, but I still feel somehow that the device is very dumb and simply sends out advertisement packages whenever it's on (still just a feeling though).
Yay! I have yet to install/use Theengs for the first time. Do I start from source, or is there a release out already with this change? Thanks so much for your responsiveness and efforts! |
The best would be to get the source for OpenMQTTGateway, you should want to take the latest development branch version there, open it in Visual Studio Code and make sure to have the PlatformIO extension installed for VSC, so you can then build and upload the Or install the web upload of the development nightly build of OpenMQTTGateway from Or if you want to use Theengs Gateway, you can do a pip install of the latest release version, which will automatically also install the letest release of Theengs Decoder as one of its dependencies. You can then get the latest Decoder source to build and install the latest dev with the XOR commit. which in turn is then also possible with any further decoder test branch you will create. Let us know how you get on. |
I believe I just (kind of) figured this one out. I am getting 100% consistent results using the following rule regarding the byte at position 13. When expressed in hex, the first symbol (so, the first four bits) denote the unit. The rest (so, the last four bits) are a checksum. Let's disregard the checksum. When the first symbol of byte 13 is A or B, the unit is kg, when it is E or F, the unit it st:lb, and when it is 8 or 9, the unit is lb. The same, but better structured:
Do you have support for bits in the Theengs decoder already, or only for bytes? Or we could solve it with simple comparisons too, since we're talking about the most significant bits. |
This is what I did. After, I updated to the latest nightly build. This is reported by Home Assistant for the
When using the scale, I can see in the esp32 console that it is being detected. The log contains the name and the MAC address and the claim it published something:
(I enabled publishing of manufacturerdata). But... now what? I don't seem to be getting any of this information in Home Assistant. Sorry if I'm missing something obvious here. |
From
So with that we can implement the selected unit. I'm wondering if the changing bit[0] state with every unit alternative might indicate the different weighing with or without impedance or such, as again it is similar to the Xiaomi scales.
Sorry, I wasn't too clear then ;) the development nightly build is linking the decoder library to the development branch of Decoder, which has the XOR implementation included after I merged it, but knows nothing of the e.volve scale test decoder I have started at my own fork, linked further above.
Perfect, that's the next thing I would have asked you to do :) so we can investigate the undecoded published data further. One thing I see with your above MQTT message is that the manufacturerdata includes the scale's Bluetooth MAC address at the beginning in reverse order, after what is usually the company ID of then device manufacturer, which I cannot find as being registered though, not unusual, but hopefully the same with every e.volve scale. This is how I usually go about reverse engineering and separating unknown advertising data. With your sample above though, does the ad instead of a 20 mean the weight was not settled yet or could it be the impedance indicator followed by impedance data then, but both weighings were with the kg unit?
As stated above, the development test build you have installed only had the XOR post_proc commit cinluded, but knows nothing of the test decoder - maybe wasn't such a good idea of me to suggest that build for testing. I have started another test build form you to try, this time with an OpenMQTTGateway branch which links the decoder library to my started test decoder branch, as you can see as the only difference to the development branch of OMG 1technophile/OpenMQTTGateway@development...decoder-test This is also how I suggested above on how you would create and install your own OpenMQTTGateway builds, linkin g to your own or my test decoder. It would only required a few simple steps to install Visual Studio Code and the PlatformIO extension, as described here https://platformio.org/install/ide?install=vscode Then opening the downloaded OpenMQTTGateway source code in VSC and on the left side in the PlatformIO section under PROJECT TASKS you can see the |
Hello again! Sorry for the long radio silence. Life got in the way, as it tends to do. :-) Today, I took some time for some more structured measurements, but I'm not sure what we can learn from them. Here's the raw data (grabbed with nRF Connect on my phone, with duplicate lines removed and in reverse chronological order).
What else would be useful at this time? Is it perhaps already a good thing to simply pull this into the codebase with only support for the weight data and ignoring impedance for now? |
Hi @bbbart Yeah, life does that sometimes, doesn't it ;)
One thing we did learn is that the actual broadcast weight is always in kg, no matter what the unit on the scale is set to, but with the scale display unit also being broadcast. So with the current decoder I am getting
So apart from the confusing additional 20 final weight measurement in your second data set the weight and unit are recognised fine, although the keys might need to be renamed to weight_kg and display_unit for clarification. Thinking about the impedance some more I am pretty certain that it would also need to be sent with the same broadcast, and not a different separate broadcast, as it would need to be available at the same time as the weight to be able to calculate all the other smart scale properties like body fat, bone mass etc. from these two values. The fact that the weight is encoded in the 3 octets, but with an XOR supports this theory, that there is more information contained in these three octets. I tried XORing with the binary inverse, but that also didn't produce any results which looked sensible to me. Maybe you have additional ideas of how the remaing data in these octets might be encoded. You're right, we could initially release this decoder with the weight_kg and display_unit only, but it would really require for you to test the actual decoder with OpenMQTTGateway a bit more, especially with the rogue final weight measurement 15.14 kg vs. the actual 40.0 kg. Did you get any further with setting up Visual Studio Code with PlatformIO?
Actually, the development test builds do work fine, it's just a performance test which fails frequently with recent builds which makes them appear failed. So let us know when you have the time to install a pre-built test binary with the e.volve decoder included on your ESP32. |
This issue is stale because it has been open for 60 days with no activity. |
This issue was closed because it has been inactive for 14 days since being marked as stale. |
The past few days I have worked on reverse engineering a smart scale I happen to own. The brand is e-volve and the companion app is Fitdays.
I figured out that
I would like to process and store this information in Home Assistant, and I believe Theengs OpenMQTTGateway is a good use for an ESP32 achieving just that.
However, if I am not mistaken, I first need to get decoding support for this particular scale supported here before I program my board with Theengs. Correct?
I'd be more than happy to share my findings here and perhaps even prepare a pull request to get it sorted.
I didn't come across any pointers yet on how to get started however. Perhaps I can get some here? :-)
Thanks to @koenvervloesem for pointing me in this direction!
The text was updated successfully, but these errors were encountered: