Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support to Beewi Smart Door as broadcaster. #22

Closed
wants to merge 3 commits into from
Closed

Added support to Beewi Smart Door as broadcaster. #22

wants to merge 3 commits into from

Conversation

KipK
Copy link
Contributor

@KipK KipK commented Jan 30, 2019

Works well.

It needs to blacklist the Beewi mac in the "services:" config first or ble2mqtt will connect to it and enable notifications. ( there are but they are useless )

Interesting data are broadcasted in the advertising payload, not served as service characteristics.
Device will be locked if connected on it and won't broadcast anything.

Also strange bug in the published mqtt side, the Type field displays :

Beewi Smart Door�]
@�]
@Mijia Temp+Hum

Looks like an overflow. I haven't found yet where is the fonction that displays this Type field.

As ble2mqtt default tries to connect to ble device, Beewi device needs to be blacklisted in "ble/services" or device will not broadcast data anymore.
@shmuelzon
Copy link
Owner

Thanks for the PR.

Also strange bug in the published mqtt side, the Type field displays...

This is because you used "Beewi Smart Door" as the name of the broadcaster, which is 16 bytes longs.
The name field of the broadcasters is limited to 16 bytes so there was no room for the nul-terminator and the string functions continued on merrily. This shouldn't happed with the current code as you shortened the name and also placed it after the MiJia broadcaster.
Can you confirm that you don't see it now?

BTW - I find it strange that the door status isn't available as a characteristic, can you please share which services/characteristics are available for this device?

@@ -356,7 +356,6 @@ static void mijia_temp_hum_metadata_get(uint8_t *adv_data, size_t adv_data_len,
uint8_t len;
mijia_temp_hum_t *mijia_temp_hum = mijia_temp_hum_data_get(adv_data,
adv_data_len, &len);

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be a part of this commit.

@@ -389,12 +388,78 @@ static broadcaster_ops_t mijia_temp_hum_ops = {
.is_broadcaster = mijia_temp_hum_is_broadcaster,
.metadata_get = mijia_temp_hum_metadata_get,
};
/* Beewi Smart Door */
#define BEEWI_SMART_DOOR_COMPANY_ID 0x000D
#define BEEWI_SMART_DOOR_SERVICE_ID 0x08
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to align define values

typedef struct {
uint16_t company_id;
uint8_t service_id;
uint8_t type_stat;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You have here and a few other places inconsistent indentation. Could you please make sure to use only spaces and remove tabs? (Yes, I'm one of those guys...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix that

#define BEEWI_SMART_DOOR_COMPANY_ID 0x000D
#define BEEWI_SMART_DOOR_SERVICE_ID 0x08
#define BEEWI_SMART_DOOR_DATA_TYPE_STAT 0x0C
#define BEEWI_SMART_DOOR_DATA_TYPE_BATT 0x06
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see where this define is used

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in beewi_smart_door_metadata_get:


 if ((beewi_smart_door->service_id == BEEWI_SMART_DOOR_SERVICE_ID) &&
	    (beewi_smart_door->type_stat == BEEWI_SMART_DOOR_DATA_TYPE_STAT)

I've added the TYPE_BATT check too in a incoming commit.

uint8_t battery;
} __attribute__((packed)) beewi_smart_door_t;

static beewi_smart_door_t *beewi_smart_door_data_get(uint8_t *adv_data, uint8_t adv_data_len,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure you don't exceed 80 characters per line (there are a few other places as well)

{
char s[6];
beewi_smart_door_t *beewi_smart_door = beewi_smart_door_data_get(adv_data, adv_data_len, NULL);
if ((beewi_smart_door->service_id == BEEWI_SMART_DOOR_SERVICE_ID) &&
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This validation should be done as a part of beewi_smart_door_is_broadcaster(), by the time this function is called you should already be sure the device type is a Beewi smart door sensor and just purse the relevant data.

uint8_t status;
uint8_t type_batt;
uint8_t battery;
} __attribute__((packed)) beewi_smart_door_t;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this is the exact format of the payload?
Could it be that the status and battery fields may either be included or not and according to the type field you should check?
Do you have any documentation you can share regarding the advertisement payload?

Copy link
Contributor Author

@KipK KipK Jan 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No doc at all they refused to communicate specs. This is just reverse enginering.
Status and Batt are always present in this frame.

Here is a complete packet :
02 01 06 02 0A 00 08 FF 0D 00 08 0C 01 06 64 11 09 42 65 57 69 20 53 6D 61 72 74 20 44 6F 6F 72 => Open
02 01 06 02 0A 00 08 FF 0D 00 08 0C 00 06 64 11 09 42 65 57 69 20 53 6D 61 72 74 20 44 6F 6F 72 => Closed

As there's always 0C before Status and 06 before Battery I've defined them as identifier.
0D 00 part before is the Company ID ( not from the device but from the bt chipset i to the standard, Texas Instrument here ( 00 0D ) .
Here is complete wireshark analysis of the packet :

Frame 435: 44 bytes on wire (352 bits), 44 bytes captured (352 bits)
Encapsulation type: Bluetooth H4 with linux header (99)
Arrival Time: Jan 22, 2019 11:33:07.735169000 Paris, Madrid
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1548153187.735169000 seconds
[Time delta from previous captured frame: 0.259482000 seconds]
[Time delta from previous displayed frame: 0.259482000 seconds]
[Time since reference or first frame: 41.804692000 seconds]
Frame Number: 435
Frame Length: 44 bytes (352 bits)
Capture Length: 44 bytes (352 bits)
[Frame is marked: False]
[Frame is ignored: False]
Point-to-Point Direction: Received (1)
[Protocols in frame: bluetooth:hci_h4:bthci_evt:btcommon]
Bluetooth
[Source: controller]
[Destination: host]
Bluetooth HCI H4
[Direction: Rcvd (0x01)]
HCI Packet Type: HCI Event (0x04)
Bluetooth HCI Event - LE Meta
Event Code: LE Meta (0x3e)
Parameter Total Length: 41
Sub Event: LE Extended Advertising Report (0x0d)
Num Reports: 1
Event Type: 0x0013, Connectable, Scannable, Legacy, Data Status: Complete
Peer Address Type: Public Device Address (0x00)
BD_ADDR: TexasIns_e3:a4:e6 (10:ce:a9:e3:a4:e6)
Primary PHY: LE 1M (0x01)
Secondary PHY: No packets on the secondary advertising channel (0x00)
Advertising SID: 0xff (not available)
TX Power: 127dBm (not available)
RSSI: -65dBm
Periodic Advertising Interval: 0x0000 (no periodic advertising)
Direct Address Type: Public Device Address (0x00)
Direct BD_ADDR: 00:00:00_00:00:00 (00:00:00:00:00:00)
Data Length: 15
Advertising Data
Flags
Length: 2
Type: Flags (0x01)
000. .... = Reserved: 0x0
...0 .... = Simultaneous LE and BR/EDR to Same Device Capable (Host): false (0x0)
.... 0... = Simultaneous LE and BR/EDR to Same Device Capable (Controller): false (0x0)
.... .1.. = BR/EDR Not Supported: true (0x1)
.... ..1. = LE General Discoverable Mode: true (0x1)
.... ...0 = LE Limited Discoverable Mode: false (0x0)
Tx Power Level
Length: 2
Type: Tx Power Level (0x0a)
Power Level (dBm): 0
Manufacturer Specific
Length: 8
Type: Manufacturer Specific (0xff)
Company ID: Texas Instruments Inc. (0x000d)
Data: 080c010664
[Expert Info (Note/Undecoded): Undecoded]
[Undecoded]
[Severity level: Note]
[Group: Undecoded]

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, in that case, using the company ID to distinguish these devices isn't good enough.
I suggest you try to read the ESP_BLE_AD_TYPE_NAME_CMPL part of the advertisement data and compare it with BeWi Smart Door.
That should be specific enough. I would also suggest marking the type_stat and type_batt fields as tbd1 and tbd2 as we don't really know yet what they are.
Since we know for sure this is the smart door sensor (as we read its name), we can forgo checking these stat fields and just assume the values are where we expect them to be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll compare with advertised device name. That's what I'm already doing in another project.
But we also nee to check the type_stat as device broadcast another frame starting also with 00 D0 08 when in pairing mode. Couldn't figure what data there's in. Probably only an identifier string to say it's in pairing state.

Copy link
Contributor Author

@KipK KipK Jan 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying this but it doesn't seems to work :

snprintf(dev_name, strlen(dev_name), "%s", esp_ble_resolve_adv_data(adv_data, ESP_BLE_AD_TYPE_NAME_CMPL, &dev_len ));
if (strncmp(dev_name, "Beewi Smart Door", dev_len) == 0)
return 1;
else return 0;

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's probably because the device sends BeWi Smart Door and not BeeWi Smart Door.
Also, esp_ble_resolve_adv_data() might return NULL so watch out for that. Since you're using strncmp() anyways, there really is no need to put it in a buffer first.

char s[6];
beewi_smart_door_t *beewi_smart_door = beewi_smart_door_data_get(adv_data, adv_data_len, NULL);
if ((beewi_smart_door->service_id == BEEWI_SMART_DOOR_SERVICE_ID) &&
(beewi_smart_door->type_stat == BEEWI_SMART_DOOR_DATA_TYPE_STAT)) {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's not a BEEWI_SMART_DOOR_DATA_TYPE_STAT, what else could it be?
Is there some other information we can extract?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dunno it's a common char that always come before status, as 0x06 for battery.
There's no other informations in the broadcasted part, compared to their offical app.

beewi_smart_door_t *beewi_smart_door = beewi_smart_door_data_get(adv_data, adv_data_len, NULL);
if ((beewi_smart_door->service_id == BEEWI_SMART_DOOR_SERVICE_ID) &&
(beewi_smart_door->type_stat == BEEWI_SMART_DOOR_DATA_TYPE_STAT)) {
sprintf(s,"%i",beewi_smart_door->status );
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As status (and battery below) are uint8_t's, the format should be %hhu

@shmuelzon
Copy link
Owner

Please also add the Beewi to the README.md file

@KipK
Copy link
Contributor Author

KipK commented Jan 31, 2019

Thanks for the PR.

Also strange bug in the published mqtt side, the Type field displays...

This is because you used "Beewi Smart Door" as the name of the broadcaster, which is 16 bytes longs.
The name field of the broadcasters is limited to 16 bytes so there was no room for the nul-terminator and the string functions continued on merrily. This shouldn't happed with the current code as you shortened the name and also placed it after the MiJia broadcaster.
Can you confirm that you don't see it now?

yes its ok with shorten name.
Broadcasted name in the advertising packet is "Beewi Smart Door". Perhaps we should extend the name variable a bit?

BTW - I find it strange that the door status isn't available as a characteristic, can you please share which services/characteristics are available for this device?

For sure it's not. We have in the characteristics:

  • many device info ( name, hardware rev, firmware rev, etc... )
  • Battery ( but also broadcasted )
  • 24h history doors
  • history doors day ( you can get 24h history for day - 1 , day - 2 .. day - 6 ) > I find it useless as the controller already log history usually
  • Local time

That's the behavior of their official app also. When connected you onl ave access to battery and history. You have to disconnect to display the opening/close status

Here is a complete dump:

`Device Name: BeeWi Smart Door
Device Address: 30:45:11:E5:A6:C8

Services:--------------------------
GAP (00001800-0000-1000-8000-00805f9b34fb)
Device Name (00002a00-0000-1000-8000-00805f9b34fb)
Appearance (00002a01-0000-1000-8000-00805f9b34fb)
Peripheral Privacy Flag (00002a02-0000-1000-8000-00805f9b34fb)
Reconnection Address (00002a03-0000-1000-8000-00805f9b34fb)
Peripheral Preferred Connection Parameters (00002a04-0000-1000-8000-00805f9b34fb)

GATT (00001801-0000-1000-8000-00805f9b34fb)
Service Changed (00002a05-0000-1000-8000-00805f9b34fb)

Device Information (0000180a-0000-1000-8000-00805f9b34fb)
System ID (00002a23-0000-1000-8000-00805f9b34fb)
Model Number String (00002a24-0000-1000-8000-00805f9b34fb)
Serial Number String (00002a25-0000-1000-8000-00805f9b34fb)
Firmware Revision String (00002a26-0000-1000-8000-00805f9b34fb)
Hardware Revision String (00002a27-0000-1000-8000-00805f9b34fb)
Software Revision String (00002a28-0000-1000-8000-00805f9b34fb)
Manufacturer Name String (00002a29-0000-1000-8000-00805f9b34fb)
IEEE 11073-20601 Regulatory (00002a2a-0000-1000-8000-00805f9b34fb)
PNPID (00002a50-0000-1000-8000-00805f9b34fb)

Unknown service (0000180f-0000-1000-8000-00805f9b34fb)
Unknown characteristic (00002a19-0000-1000-8000-00805f9b34fb)

Unknown service (a8b3fd00-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3fd01-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3fd02-4834-4051-89d0-3de95cddd318)

Unknown service (a8b3fc00-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3fc04-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3fc06-4834-4051-89d0-3de95cddd318)

Unknown service (a8b3ffe0-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3ffe1-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3ffe2-4834-4051-89d0-3de95cddd318)
Unknown characteristic (a8b3ffe3-4834-4051-89d0-3de95cddd318)

Unknown service (f000ffc0-0451-4000-b000-000000000000)
Unknown characteristic (f000ffc1-0451-4000-b000-000000000000)
Unknown characteristic (f000ffc2-0451-4000-b000-000000000000)

--------------------------`

@KipK
Copy link
Contributor Author

KipK commented Jan 31, 2019

Updated it. Looks solid for now.

@shmuelzon
Copy link
Owner

Where's the check for the "BeWi Smart Door" name?

@KipK
Copy link
Contributor Author

KipK commented Jan 31, 2019

Where's the check for the "BeWi Smart Door" name?

Btw, it's "BeeWii" not "BeWi"

in static beewi_smart_door_t *beewi_smart_door_data_get:

char *name = (char *)esp_ble_resolve_adv_data(adv_data, ESP_BLE_AD_TYPE_NAME_CMPL, &name_len); if (strncmp(name, "BeeWi Smart Door", name_len) != 0) return NULL;

@shmuelzon
Copy link
Owner

And that works for you? According to the payload you posted before

02 01 06 02 0A 00 08 FF 0D 00 08 0C 01 06 64 11 09 42 65 57 69 20 53 6D 61 72 74 20 44 6F 6F 72

It sends BeWi Smart Door and not a BeeWi Smart Door. Besides, all of the tests to check if this device is the one we want to handle should be done in beewi_smart_door_is_broadcaster() and should be done only once.

@KipK
Copy link
Contributor Author

KipK commented Jan 31, 2019

And that works for you? According to the payload you posted before

02 01 06 02 0A 00 08 FF 0D 00 08 0C 01 06 64 11 09 42 65 57 69 20 53 6D 61 72 74 20 44 6F 6F 72

It sends BeWi Smart Door and not a BeeWi Smart Door. Besides, all of the tests to check if this device is the one we want to handle should be done in beewi_smart_door_is_broadcaster() and should be done only once.

That's a typo in my copy pasta I think. I get 42 65 65 57 ... so BeeWi it is :)

I've moved the test in broadcaster function. I need to keep a test in beewi_smart_door_metadata_get() as the device can sen other frame type somehow ( only when in pairing state )

@shmuelzon
Copy link
Owner

The same test that you do in beewi_smart_door_metadata_get() can be implemented in beewi_smart_door_is_broadcaster(), they work on the same data. The only difference is that if it's in the former, the BLE2MQTT app will publish the type and RSSI of the device without any metadata while the latter will not publish anything if the message type isn't what you expect.

There are also some additional coding convention issues. I'll modify them tonight and manually merge.

@KipK
Copy link
Contributor Author

KipK commented Feb 1, 2019

Thanks, feel free to optimise, I'm no real good developer :)

Also if you want some packet capture compatible with wireshark of the whole pairing process + connection, I can provide.
I haven't found how the pairing process is done yet.

shmuelzon added a commit that referenced this pull request Feb 4, 2019
shmuelzon added a commit that referenced this pull request Feb 4, 2019
@shmuelzon
Copy link
Owner

Hey,

Could you please test out https://github.com/shmuelzon/esp32-ble2mqtt/commits/KipK-master and make sure it works correctly?
If it does, I'll merge it.

Thanks!

@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

I will asap thanks.
I've experimented a bit with the Gatt part of the device.

You can only connect to the device when in pairing state if not bonded.

There's a pairing sequence to handle. I've captured some packets and tryed to reproduce it with a python script. It switchs ok the device to non pairing mode.at the end of the sequence , but still no connection after. I'm still locked here.

Also I get different results between battery state reported in ad data vs the battery service. Strangely the official apps displays the same battery result than me ( from the ad data ). Perhaps battery sensor report less when connected because it draws more current ? Dunno.

I could see the official app writing in non advertised characteristics. Perhaps there's a hidden door state somewhere. But the broadcast way looks more efficient to me.

@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

Here are are the hci logs you can read with wireshark:
https://www.dropbox.com/s/4k9rrnyeytq9z8j/hci_snoop_2019_02_04_15_55_35.cfa?dl=0
Device was set in pairing mode ( pushing a button ) , then I've connected with the app.
The pairing sequence looks like this following this file.

Write 0x3A - 00 ( set pairing state to 0) ( if you read it it's 1 when pairing is enabled on it )
Write 0x33 - 01 00 ( hidden) ( notify )
Read 0x18 ( firm rev )
Read 0x2B ( time ) => 303030303030303030303030 ( ascii 0 0 0 0 0 ....)
Write 0x32 - 00 00 ( history days )
Read 0x18 ( firm rev )
Read 0x28 ( name )
Write 0x42 - 01 00 ( hidden) ( notify )
Write 0x2b - 31 39 30 32 30 34 31 35 35 39 30 33 ( date&time in ascii ( aammddhhmmss ) : 19 02 04 15 59 03 )
Read 0x18 ( firmware revision )
Write 0x41 - 00 00 00 00 ( ?? ) => answer always: 00 00 00 7c 41 41 41 41
0x42 - 00 00 ( hidden) ( notify )

By reproducing this, the client is still not recognised at next connection when pairing mode is off, so there's something else that I've missed.

@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

Tested right now. Everything works as it should.

@shmuelzon
Copy link
Owner

Thanks!
Merged manually.

@shmuelzon shmuelzon closed this Feb 5, 2019
@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

Tell me when you've grabbed the log so I can delete the link . Do you have any clue on what I've could miss on the pairing process? I don't know enough BLE standards perhaps it's some common thing. But no trace of it on wireshark

@shmuelzon
Copy link
Owner

I've downloaded the capture and verified I can open so you can take down the link.
I've yet to take a deep look at it but hope to get around to it over the weekend.

@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

You can give a look to how I'm integrating it in Jeedom here:
https://github.com/jeedom/plugin-blea/pull/50/files

Still a Wip as pairing doesn't work yet.

@KipK
Copy link
Contributor Author

KipK commented Feb 5, 2019

there's many devices implemented here you can grab info from:
https://github.com/jeedom/plugin-blea/tree/beta/resources/blead/devices

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants